Thursday, April 29, 2010

Windows BranchCache OS requirements

I was asked at the Windows Intelligence event I presented at on Monday if there were specific requirements on what version of Windows 7 could be used with BranchCache. It turns out there are, you can only use Windows 7 Enterprise and Ultimate Editions to take advantage of BranchCache - even if you are doing a workgroup configuration and not a domain.

The step by step guide has the requirements for both the client and the server and can be found here.

It appears that Microsoft is assuming small and medium sized companies should be using Enterprise or Ultimate Editions to take advantage of newer technologies like BranchCache and DirectAccess. I will ask around to see if this will be changing at all in the future and post back up what I hear. I am a bit surprised to read that Windows 7 Professional didn't make the cut for BranchCache as a feature but licensing is a complicated thing that honestly I think very few folks truly understand, even within Microsoft.
- Ed

Tuesday, April 27, 2010

Windows Intelligence DirectAccess and BranchCache presentations

I had a fun time yesterday presenting on Microsoft DirectAccess and BranchCache at the Windows Intelligence NorCal event. Thank you to everyone who attended the event, I appreciate the discussion and feedback and as promised here are the slide decks for download.



If you attended my sessions you might have missed sessions by Jennelle Crothers and Steve Evens who both gave excellent presentations from what I heard (I was giving mine so I couldn't be in two places at the same time!) and should be making their presentations available on their sites also. Doug Spindler also presented but I am not sure where his presentations are posted but I will find out and update here shortly.

It was also nice to see the West Coast Microsoft DPE IT Pro Evangelists at the event, Chris Henley, Chris Avis and Harold Wong. They are all wonderful presenters and totally passionate about what they do - a big shout out to them - you guys rock!
- Ed

Friday, April 23, 2010

Cisco IPSec VPN Client - 5.0.7 Released- Win7 64-bit support

Cisco has officially released a 64-bit version (non-beta) of the IPSec VPN Client, version 5.0.7 (vpnclient-winx64-msi- available for download. Finally a Cisco supported 64-bit version of the IPSec client for Windows 7! It is available for download on CCO but requires a valid CCO login and current contract to get the code.

Do read the release notes, there are caveats for Windows Vista in the release notes so if you are planning on running it on 64-bit Vista make sure you review those. Also, it is important to note there is a specific section titled "Windows 7 and Vista Window Auto-tuning Feature Might Cause Network Timeout Problems" that states:
"Windows 7 and Vista support a feature called "Receive Window Auto-Tuning" that continually adjusts the receive Windows size, based upon the changing network conditions.

Some people reported that auto-tuning causes network timeout problems with some applications and routers. If you have experienced such problems, you can turn it off using the following procedure:

Step 1 Open an elevated command prompt.
Step 2 Enter the following command to disable auto-tuning:
netsh interface tcp set global autotuninglevel=disabled

If this solution does not fix the problem, you can turn it back on, as follows:
Step 1 Open up an elevated command prompt.
Step 2 Enter the following command to enable auto-tuning
netsh interface tcp set global autotuninglevel=normal

To view the states of the TCP global parameters, use the following command:
netsh interface tcp show global "

So for all of you that have to support Windows 7 64-bit and still have Cisco VPN Concentrators, PIX's and ASA's doing IPSec only you have a valid supported solution now! I don't know why it took so long but it is nice to know you don't have a pay the AnyConnect essentials license if you don't want to. I do like many of the features and advantages with AnyConnect, it just seemed wrong that to get VPN for a 64-bit OS you HAD to use AnyConnect only and pay the premium for the client.
- Ed

Wednesday, April 21, 2010

Cisco ASA - PAT performance and logging

It isn't a common issue for most small to medium businesses but occasionally you have to remember that when doing port address translation (PAT) or overloading a single IP address with multiple IP's behind it that there are actual limitation on how many session you can do simultaneously. Remember that for a single IP address can only do 65536 TCP sessions. If you remove the first 1024 you are down to 64512.

So why is this important? Turns out if you have enough hosts generating requests behind the ASA you can run out of ports and if you generate traffic quickly enough you will start getting the following error in the log files:

%ASA-3-305006: portmap translation creation failed for tcp src XXX dst XXX

The message description from Cisco is here and shows the format as:
%PIX|ASA-3-305006: {outbound static|identity|portmap|regular)
translation creation failed for protocol src interface_name:source_address/source_port
dst interface_name:dest_address/dest_port

I don't think this will be a common issue as it takes a lot of hosts to generate this sort of load doing thousands of requests a second. To exceed the threshold for instance you would need 250 hosts generating approximately 260 requests per second sustained per host and they would have to keep those sessions open for the ASA to start having this issue.

Luckily it is a rather fast and easy fix if you have more than one Public IP address available. You can simply add another global statement to the ASA with more public IP's to overload and the ASA will happily start spreading the load across the new IP addresses that you add. In terms of syslog, this is severity level 3 (error) so it should show up in most syslog data that is collected. Definitely recommend running Splunk, it makes finding these sort of errors so much easier!

Also, while it may be saying a portmap translation failed the ASA itself doesn't run out of resources from a CPU or memory standpoint. The ASA will continue to run along nicely it just won't be able to generate new TCP sessions as fast as your hosts may be requesting. This makes it really important to be looking at your log files to know what is happening. The problem is a function of how you are doing overloading, not a limitation of how the device performs that function.
- Ed

Friday, April 16, 2010

Cisco ASA - how to see your pre-shared-key

One of the annoying things about managing pre-shared keys for both site to site vpn tunnels and group pre-shared keys for client vpn tunnels is the fact that if you do a show run they are starred out (*) in the configuration file.

So you will see something like:
pre-shared-key *

If you need to recover back your keys because you have lots of folks running around with Cisco IPSec VPN clients with a standard PCF file and you can't remember what the group pre-shared-key is or don't have it documented you can do the following command.

more system:running-config

This will output your running-config file with the pre-shared-key variable in clear text.

Obviously this is useful for site to site keys also because you might have a tunnel set up with a third party vendor or a partner and you can't remember the key at all because you made it up on the fly with the other network engineer on the phone (That never happens... really.)

This will NOT show you the enable or passwd values because those are actually encrypted. You will have to use other tools to break those or do a standard password recovery process.
- Ed

Tuesday, April 13, 2010

Cisco ASA syslog over TCP - Potential to stop forwarding traffic!

I ran into an interesting configuration issue with the Cisco ASA and doing syslog via TCP to a syslog server. It turns out that Cisco has a feature in the ASA that when pushing logging information to a sylog server via TCP that if the server does not respond it will stop the ASA from building out new session flows and therefore it will stop forwarding traffic. It seems that existing TCP sessions continue to allow traffic to flow, the rational being that the logging of that flow being established was sent to the syslog earlier and therefore it is audited event. Apparently this was a specific request from some government agencies who did not want their firewalls to pass traffic if logging was not possible. Reasonable enough but it isn't particularly well documented and does not alert you of it's default behavior at all when you enable the syslog command using TCP.

You can find all the details in the Cisco ASA 5500 command line configuration guide (this one is for 8.2 code release) and look under "Sending Syslog Messages to a Syslog Server" to see the parameters. They do provide a optional command switch that allows the ASA to continue working even if the syslog server is down.

So how can this be a problem? It turns out that a lot of shops rotate their syslog files and will write a quick script to stop their daemon for syslog, change their logfile and then start the syslogd daemon again. This causes the TCP session to fail because the syslogd daemon is no longer keeping the socket open. Depending on the thresholds you have built on the network this can cause your ASA to stop forwarding traffic. Worse, it is difficult to figure out what is happening because existing log flow sessions (perhaps and ssh or database session) will still be working but anything new you attempt to send through the ASA will not. Also, the ASA does not consider this a flag to attempt to failover to a standby unit - in fact, there is no way to even monitor the syslog process to flag it as such. I think that is something that needs to be added so the failover unit can then test to see if it can connect to the syslog server and if so, failover so that your firewall continues to forward traffic as expected.

The error you will get in the local log is:
%ASA-3-201008: Disallowing new connections.

It tells you what is happening - it doesn't tell you why it is happening though - very frustrating.

This is not an issue if you are using UDP for syslog traffic on the ASA's as they assume if you are doing UDP to the syslog server it is not guaranteed delivery and therefore you don't have to meeting the same standards for logging or security.
- Ed

Friday, April 02, 2010

Cisco Nexus 5k and 2148t caveats

I ran into a few more items I think warrant a post regarding the Cisco Nexus 5k series and the 2k fabric extenders (fex.) For those that are evaluating the Nexus 5010 and 5020's with the 2148t fex for layer 2 switchport capacity there are two gotchas that I think failed to get mentioned often enough in the pre-sales engagement.

1. The 2148t fex is not capable of doing a port-channel (ether-channel) with two or more ports on a single chassis. This is not a limitation that can be fixed with software either, it is a limitation in the ASIC's themselves on the 2148t. I think this is a huge failure. That being said, I believe it is road mapped as a feature fix for the new 2248TP and 2232PP models.

2. If you do need to do a port-channel to a single host from a 2148t you need two 2148t's and you have to run a Virtual Port-channel (vPC) using the 2148t's that are each single homed to the 5k's upstream (in what is called single homed fabric extender vPC topology.) You can only do a maximum of 1 Gigabit Ethernet port to each 2148t (because of the limitation above) and then the 5k's will do the logical vPC to make them a port-channel. So a 2 Gigibit max port-channel with the 2148t single homed to the 5k's. Again, because of the limitation above I see this as poor engineering work around to the failure in design.

So, with those limitation on the 2148t fex's there is something else from a design standpoint that is frustrating. You have two recommended options when designing the 5k/2k deployment, you can do a single homed fabric extender vPC topology or a dual homed fabric extender vPC topology. Here are the diagrams from the Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide.

The single homed:

and the dual homed:

What is not mentioned or documented anywhere about this configuration is if you can mix and match the topology between single homed and dual homed 2k's on a single pair of 5k's. I can only find one reference that mentions this vPC mixed topology option. Even with this option you are still burning up a second 2148t just to get port-channeling as an option if you don't want to use the 5k's directly. Burning up the few 10/1 Gig mix use ports on the 5k's for 1 Gig port-channels seems to be a huge waste and very expensive.

So there you have it, some items to mull over while planning out your data center design with Nexus and some hard hitting questions to ask your Cisco SE's when they show up talking about cost per port and maximizing functionality and flexibility per port. On top of the above items the 2148t also only does 1Gig, it doesn't support 10/100 at all, not a show stopper but another nagging item that limits some deployments. Don't get me wrong, I'm a fan of the new data center design and topology model that Cisco is building and with OTV coming out in the 5.0 NX-OS code release some really exciting things are going to be possible. Just keep in mind you might be buying an extra one or two 2k's to accommodate some of the systems that have the port-channel requirements for higher availability.
- Ed