Sunday, December 08, 2013

Follow up to my IPv6 and ULA post plus some thoughts on design and IPv6 behavior

I've gotten some interesting comments and questions in regards to my IPv6 ULA (Unique Local Address) post and I wanted to outline the questions and then follow up with my thoughts and some high level design discussion.

The first question is around the situation of multiple ISP connections without using BGP and Provider Independent (PI) addresses or alternately switching from one ISP to another. The argument is pretty straightforward, you don't want to have to renumber your servers, network links and other infrastructure when changing ISP's because the IPv6 address space you have is allocated from that ISP. That IPv6 address space will no longer be yours when you change from ISP A to ISP B. So, alternately you can use ULA for your network, server and even client devices and only use that ISP IPv6 address space at the edge, similar to how you implement IPv4 today. This requires the use of some sort of NAT translation at the edge.

The second question is a bit of a refinement of the above that argues you should use ULA for all internal hosts and your LAN and WAN resources so you don't have to renumber if you change ISP's but also so you have a consistent address prefix preference for internal traffic to register with DNS.

I think those really are the main pro ULA arguments I have heard so far. Let's tackle the first part which is really all about flexibility in choosing and changing your ISP and reducing the IPv6 renumbering costs to a fractional degree. The second one I will cover in a different post.

The following diagrams shows the multiple ISP connection configuration which is no different than migrating from one ISP to another. We start with ISP A on the left and will be moving to ISP B on the right. Our initial network configuration will use ISP A's IPv6 Provider Assigned (PA) address block and our goal is the move to ISP B's IPv6 Provider Assigned (PA) address block. For a duration of time you will have both ISP A's and ISP B's IPv6 address blocks active on your network. The next diagrams (Figure 2 and 3) are what everyone thinks they want to deploy. It utilizes some sort of NAT technology at the Internet edge and assumes you will only use ULA for the "internal" network.

Figure 1. Dual connections while migrating service providers

Figure 2. ISP 1 with NAT66

Figure 3. ISP 2 with NAT66

The primary problem with the ULA solution people want is that there are no good production quality NPTv6 implementations out there to run on a firewall or edge Internet device today. This limits your ability to actually deploy that solution so you are left with NAT66 (not a good option in my opinion). Furthermore, NPTv6 still breaks many end to end features for applications relying on the underlying networking protocol not doing NAT. Application developers will still have to build NAT traversal mechanisms into their apps even though we were not supposed to need them with IPv6 at all. The only advantage you will gain is that with NPTv6 you have a stateless solution for NAT as only the prefix is actually changed. This reduces the amount of resources required on that Internet edge device greatly compared to the same solution with IPv4. An additional option is to go with NAT66 (which is available from some network manufactures) but which make IPv6 even more brittle and introduces every one of the NAT/PAT problems we have with IPv4 only on a much larger scale. I consider this solution a huge mistake and using it roles back all the progress and design work that the architects of IPv6 put into the protocol.

The slightly more difficult dual ISP PA migration (Figure 1) solution will actually work just fine without ULA. You will have to do some planning to do renumbering however depending on your design this could potentially be relatively easy. The most difficult resources to renumber would be your LAN and WAN infrastructure and your servers. Client devices should be dynamically obtaining their IP addresses (IPv4 or IPv6) and should not care greatly if their network address prefix changes over a weekend or even within the day. IPv6 has no issue with running both PA IPv6 prefixes at the same time nor does it require ULA to do this function. You might gain some time back on larger LAN and WAN networks on not having to renumber and this could be true for servers also however you will have to write additional firewall rules and you will still have to give them global unicast addresses if they need to talk to the public IPv6 Internet at all (remember, limited or no NPTv6 support today). This means you still have to touch the device to change it over from one IPv6 prefix to another. If it does not have the requirement for being able to get to the public Internet at all then there is a use case for using ULA. I see this as a secure network and I argued in the previous blog post that a secure network would be a legitimate use case for ULA.

ULA may buy you a small amount of stability in avoiding some address renumbering but I would argue that a proper DDI solution solves almost all the issues using DDNS and name to IP address resolution. What you gain from ULA to avoid renumbering you will likely have to pay back 10 fold in NAT traversal issues and pain.

A quick question, how often are people changing ISP's for small to medium sized businesses? I think some of the renumbering argument assumes this happens far more often than it does. For many medium to larger enterprise customers they will likely get Provider Independent IPv6 address space and utilize BGP to advertise to multiple ISP's at once (Figure 4). They do not require NPTv6 and even if they need local Internet hop off at a branch location they would simply deploy both PI and PA IPv6 prefixes at that site and let RFC 6724 solve the source/destination matching (Figure 5). They can gain more granular control of the routing behavior if they modify the prefix policy tables.

Figure 4. Dual homed IPv6 BGP peering

Figure 5. IPv6 PI and PA with local Internet hop off

As for those that are dual homing but not running BGP they do not require ULA or a NAT solution, simply run both IPv6 PA address space on the network (Figure 1) and let RFC 6724 solve the source/destination matching. If you want more control you can manage the prefix policy tables on your hosts or do some routing policy work within your network. Even so, if you are doing dual homed but no BGP you likely don't have a large enough network to care or bother with those efforts. Either way, I believe it is still simpler to use both service providers IPv6 PA space than to deal with ULA and NAT.

Thoughts? What other ways to you see ULA being an advantage over global unicast IPv6 addresses in your network?
- Ed

My book is available for pre-order on Amazon or you can order directly from the publisher Apress


rtaccon said...



I have been working with multiple Enterprise customers developing IPv6 addressing schemes and I can tell you that dual homing and asymmetric routing are REAL issues with PI space. For example if you have a /48 from ARIN and advertise that block out of multiple DC’s to the global Internet, there is the real possibility that inbound return traffic could come through a different link than outbound. This gets even more complicated when you have more than 2 DC’s (one customer I am working with has 4 US DC’s). Outbound can be influenced with BGP metrics, but inbound can be an issue.

With NAT you can easily send a more specific route out from your regional firewalls and ensure that return traffic comes back into the same stateful firewall. The problem now with going to this especially over my MPLS VRF around the country for remote sites, the traffic can come into either of my two core sites in CO and MD for redundancy then exit through my firewalls to the internet but the return traffic can’t be controlled as to which firewall it comes back into. Without some way of controlling this routing (like via NAT) it will be broken if servers are not in the same area.

Ed Horley said...

Yes, I have read both those threads previously. You can still do more specific /48 site prefix advertisements today and have some more granular control for inbound via BGP (same as IPv4). The reality is that BGP issues are the same, regardless of IPv4 or IPv6. As far as stateful and asymmetric routing, these problems exist in IPv4 with or without NAT so IPv6 in not unique there either.

You are not required to run PI space from all your sites, you can run regional PA space and simple do route filters and firewall ACL's - see Figure 5.

If you are controlling routing via NAT you are doing something wrong in my book. I know some folks like to think NAT solves these sort of issues, it really does not. Furthermore, if we are future looking, with the moves to NFV and SDN these issue will become less impactful over time as policy enforcement moves closer and closer to the end host device.

garegin said...

stupid question. let's say that you are in a v4 or v6 environment were you have enough addresses for everyone. But you don't want your camera or printer to be routed on the internet. Could you just make that host non-routable in your CPE? What if your internet connection suddenly goes down. Do all your hosts loose IP addresses?

Ed Horley said...

Those are fair questions. It really is not any different in IPv4 verse IPv6 for those use cases. If you want those devices to have specific IPv6 addresses you can either use DHCPv6 reservations and make sure the A flag is off on your CPE or you can statically assign them. You would then use your stateful packet inspection firewall to write rules to block/permit access to resources on your network. If your CPE device drops its external connection you will still have a valid lifetime on the use of the IPv6 address if you are using DHCPv6. If it is static then you will use it for as long as the you leave it on the host. If the CPE is attempting to set up a new prefix you will need to change the hosts. That is where SLAAC becomes an advantage, it allows your service provider to renumber your local CPE network without much customer interaction if any. They can allocate a new prefix and your CPE will renumber (if it supports multiple prefix delegation) and expire out the old prefix.
- Ed

Carol Chisholm said...

Still trying to figure out how small security-oriented businesses need to implement this.

Talking about a /24 LAN a /24 DMZ, one firewall/router, no SIEM.

All IPv4 addresses currently fixed or DHCP reservations for control and reporting purposes.

ISPs keep merging and renumbering - not our choice!

An because we're small we can't get our own block, we have to wait for our ISP to sort itself out.

Carol Chisholm said...

And we're not putting our data in the cloud any time soon...