Monday, June 01, 2015

IPv6, Docker and building for scale

I've had some interesting conversations lately around some of my ideas about why Docker would be fundamentally better with IPv6 and IPv6 only. You can check out the podcast I did with Matt Oswalt and Jon Langemak for the ClassC Block.

Let's jump right into it. Some of the constraints around Docker is the IPv4 networking stack, how to do port forwarding, NAT and dealing with RFC 1918, even routing. There is a lot of state and management that goes into all that code to just deal with basic networking.

What if we could use a new paradigm to make Docker easier, with less state and dependencies and best of all, not having to remap any ports at all?

What if we gave every Docker host a routed /64 IPv6 address prefix and allowed it to preallocate IPv6 addresses from that /64 block to any Docker container that wanted to be run on that platform.

Better yet, let's never reuse that IPv6 address again - every (what?!? are you crazy?!?). How long would it take to burn through that /64 of public IPv6 address space?

As  Leonard Hofstadter's mom on Big Bang Theory said, "I'd like to do the math." So here we go:
Let's assume a crazy number of containers on a single host in a second, something so large no one will argue with us about it not being large enough.
  • How about 10,000,000 per second <-- yes, 10 million per second
  • A standard /64 prefix in IPv6 is 18,446,744,073,709,600,000 addresses.
 And the math:
18,446,744,073,709,600,000 IPv6 addresses / (10,000,000 IPv6 addresses/second * 60 sec/min * 60 min/hr * 24 hr/day * 365 days/yr) = 58,494 years

To consume a single /64 of address space on a single Docker host that is generating 10 million containers per second it would take more than 58 thousand years to consume all the IPv6 addresses.

A single /48 that you would allocate to a data center has 65,536 /64's in it. So, if you allocate an entire /48 to just your Docker hosts (this means you are running a data center with 65,536 servers) then you will not run out of unique IPv6 addresses on your server for (more math):
58,494 years * 65,536 /64 per /48 = 3,833,478,626

So, at a run rate of 10 million containers per second, a standard /48 that you would allocate to a data center for docker hosts it would take you 3.8 billion years to consume all the IPv6. I think that will cover almost every company that ever needs to run Docker containers and have them be unique at a point in time. Best part, we don't have to modify port numbers, they are globally unique, we can lay down a predictive algorithm for building out the lower /64 and we don't have to deal with any layer 2 at all, it is all routing! Seems like Nirvana to me.
- Ed