Thursday, April 25, 2013

Some thoughts from the 2013 North American IPv6 Summit

After spending the last few days in Denver, CO participating in the largest IPv6 event in North America (and freezing due to a cold front moving through and dumping snow - in April no less!) I have come up with some take aways.

First, there are several dilemmas in the industry affecting adoption of IPv6. Specifically, the lack of content providers having their content available via IPv6. The disparate performance differences on the public Internet for IPv6 vs. IPv4 and the fact that it is not being accurately measured or even being paid attention to by the majority of service providers. And finally the challenge that enterprises are having in deciding between DHCPv6 stateful vs. DHCPv6 stateless vs. wanting to use SLAAC w/ RFC 6106 (no thanks to Android for making that problem worse.)

Second, the lack of interest, press coverage and buzz around IPv6 is upsetting, even the home page of Network World did not have a single mention of the word IPv6 after the last day of the Summit (actually any day of the summit.) I was surprised this year by how many colleagues and friends felt it more important to attend OpenStack in Portland and ONS in San Jose rather than attend the IPv6 Summit. What is even more alarming is the fact that both those events are around technologies that MUST have robust IPv6 support or they will fail long term, in other words they will have no market if they don't have an IPv6 solution. Perhaps it was just unfortunate timing that all three events were the same week but there were some noticeable gaps in attendance and in some key luminaries who chose to attend some of the other events. Granted, the weather, American Airlines having issues which grounded its fleet and the horrible news from Boston and it's subsequent lock down likely had an impact on attendance. Even with that I still feel the messaging on how important IPv6 will be to those in the IT community can't be over emphasized and I find it alarming more people are not participating and learning about IPv6. This event is a rare opportunity to learn arguably one of the most important basic skills you will need in your career as an IT Professional regardless of role. As a side note, both OpenStack and ONS sold out but the NA IPv6 Summit still had room for more attendees. I am not sure why that is and would love to hear what others think about that.

Third, the slow realization that IPv6 is not accelerating but is actually slowing down in adoption. The application rate for Provider Independent (PI) space is slowing, the role outs of IPv6 to service provider customers is not happening at an impactful rate and because of the lack of content its use will languish until more Enterprises feel it is important to have as a transport option. There is serious case to wonder if IPv6 will make it or if everyone will decide to make horrible IPv4 NAT on NAT on NAT (NAT444) solutions and just deal with brokenness and performance problems indefinitely. Perhaps they are hoping that eventually it will be somebody else's problem to deal with.

So, in summary, while I am a huge IPv6 advocate I am getting tired of being ridiculed by manufactures, peers, industry pundits and customers whenever I bring up the topic of IPv6 and adoption and everyone's response is "isn't that dead" or "no one actually uses IPv6" or "I hope to retire before I have to learn that" as quips back to my inquiry. Honestly, I am to the point now where my response is going to be "be part of the solution or get the heck out of the way" with perhaps some expletives in the mix depending on the audience. As an anecdotal side note, I have had several long time IPv6 colleagues and industry leaders tell me they are not focusing as much on IPv6 due to the incredible frustration of it being ignored and the frightening thought that IPv6 might not actually make it as a legitimate technology solution due to the lack of interest and adoption. Apparently growth for the Internet just really isn't that important.

- Ed

Co-chair, CA IPv6 Task Force

Wednesday, April 10, 2013

Second Annual TechDays happening May 2-3, 2013 - only 3 weeks left to sign up!

Pacific IT Professionals will be hosting the second annual TechDays conference in San Francisco May 2-3, 2013. Really excited this is happening, specifically because the speaker line up this year is fantastic. The topics range from PowerShell (with two separate PowerShell MVP's coming to present - Jeff Hicks and Aleksandar Nikolic) to Cloud (Private and Public) to Windows Server 2012 and Windows 8.

Mark Minasi is returning and will be incredible as always and we will be welcoming some DevOp sessions with Glenn Block and Richard Campbell. Darren Mar-Elia is returning so those that need their Group Policy fix should be sure to attend. To add to the fun we will be having a Women In Technology (WIT) panel with Christa Anderson, Jennelle Crothers, Jessica DeVita and Laura Hunter and being moderated by Richard Campbell to be recorded for RunAs Radio. Almost all the speakers presenting at this conference are either Microsoft blue badge or Microsoft MVP's and are regular speakers at other conferences like Microsoft TechEd, TechMentor and Windows Connections.

Overall, I can't emphasis enough what an incredible bargain this conference is given what the quality of sessions (4 tracks running over 2 days) will be so go register now. It is limited to the first 200 registered attendees and that is it! The sessions are all available up on EventBoard which is available for iPhone, Android and Windows Phone in addition to a native Windows 8 app. A summary of the sessions is also up on the website.

I hope I will see you there!
- Ed

Monday, April 08, 2013

Some early lessons in PowerShell

I've been working on some practical uses of PowerShell for some testing in our lab (and as an excuse to start learning PowerShell too.) Specifically, we are testing the connection limitations of some firewalls (Cisco ASA's running in multi-context mode) and needed to use a cost effective tool (free) to make this all happen. I decided I can make all this work with iperf and use PowerShell to manage it all. So the challenge was to start an iperf session that increments the TCP port number for each instance which would also us to specifically measure the connection counts and get discrete sessions through the firewall.

I started off thinking I could use the Start-Job cmdlet to manage everything and built out the following working PowerShell:

# - use a for loop to open 100 iperf server listeners
for (; $i -le 100; $i++)
{ Start-Job -ScriptBlock { C:\users\ehorley\Downloads\java\jperf-2.0.0\bin\iperf.exe -s -p (5000 + $i) }

This sort of worked however there were some pretty big flaws that became obvious when executing it. First, was the fact that the PowerShell didn't do what I wanted, it didn't increment the iperf port number at all. I had no clue why until I started reading about global variable and scopes. It turns out that my variable wasn't passing into the ScriptBlock at all. So, after a bit of digging I figured out how to fix that and came up with:

# - just name the file iperf-servers.ps1 and you can execute it from PS by:
# - .\iperf-servers.ps1
# - starts $portcount # of iperf server sessions as background jobs
# - set the
$portcount = 1
$portcountmax = 100
for (; $portcount -le $portcountmax; $portcount++)
    { Start-Job -ScriptBlock {
    cd "C:\users\ehorley\Downloads\java\jperf-2.0.0\bin"
    ./iperf.exe -s -p (5000 + $portcount)
    } -ArgumentList $portcount

This script actually worked! It incremented the TCP port number of the iperf session and I was able to verify that with a simple netstat -an -p TCP and I was able to remove all the jobs started by the script in a rather brute force as with:

Get-Job | Remove-Job -force

There were some big issues with this method though. Because I was using the start-job cmdlet PowerShell was starting a new PowerShell instance for each job that it runs. In addition, it is starting a cmd.exe process and an iperf.exe process which resulted in a lot of memory being consumed for a very small number of TCP ports being opened. In our testing on a Windows 7 client VM with 4GB of memory we could not push much past 150 to 200 jobs, really not very useful when you want to test 20,000 concurrent connections through a firewall.

So it was back to the drawing board to figure out a better way to do this. I decided that the jobs cmdlets was not the way to go and I had to figure out a more reasonable way that did not start multiple PowerShell sessions but instead just ran from the single PowerShell session but ran all the cmd.exe and iperf.exe processes.

After awhile I figured out that I could use the Start-Process cmdlet instead and came up with:

# - just name the file iperf-servers-2.ps1 and you can execute it from PS by:
# - .\iperf-servers-2.ps1
# - starts $portcount # of iperf server sessions as background jobs
# - set the $portcountmax variable to the high end of sessions
# - correct the path to iperf executable before running
# - modify the @iperfargs to match client or server role
# - to see all the jobs run:
# - get-process iperf
# - to kill all the sessions:
# - get-process iperf | stop-process
$portcount = 1
$portcountmax = 1000
for (; $portcount -le $portcountmax; $portcount++)
    $iperfargs = "-s -p " + (5000 + $portcount)
    # $iperfargs = "-c <host ip> -t -p " + (5000 + $portcount) - or something like that for the client
    Start-Process -windowstyle Hidden -WorkingDirectory "C:\users\ehorley\Downloads\java\jperf-2.0.0\bin\" .\iperf.exe $iperfargs

This script worked as expected and I was able to execute 1,000 sessions and confirm the TCP ports on my laptop without any issue. As is indicated in the script you could confirm the processes are running with:

Get-Process iperf


and you could kill all the sessions with:

Get-Process iperf | Stop-Process

Clearly there is a lot of work that could be done to automate the process to do this even more efficiently and perhaps I will do that but the scripts were functional enough for our engineers to use to do the testing. The Windows 7 VM's with 4GB of RAM were able to run over 1,750 TCP ports each which allowed for us to build out a reasonable test farm in our lab of virtual machines and test the firewalls.

Yes, I know I could likely do this easier with a simple javac or even just use linux and some perl or python but remember I was using the opportunity to learn PowerShell! I was grateful for having the incredibly useful book from Don Jones and Jeff Hicks - Learn Windows PowerShell 3 in a Month of Lunches, Second Edition by Manning. If you are at all interested in learning PowerShell I highly recommend it.
- Ed