Tuesday, March 23, 2010

Cisco Nexus command - Checkpoints and rollbacks

Using the Cisco Nexus 7010, 5010 and 2148's has changed some of the habits I have traditionally used for the Cisco IOS command set. Some of the new Nexus commands have become second nature and I now miss them on IOS. Being able to use grep is one I really wish was incorporated into IOS. I am used to having it with the ASA platform and now with the Nexus platform - going back to IOS 12.x and not having it there is annoying.

A new command that is really useful on the Nexus platform is checkpoint. There are several things that are unique about checkpoints and how you can use them. First, checkpoints are primarily used for rollback situations. They allow you to make changes on the system and if required due to an error rollback to a known good configuration on the system. There are three rollback types.
Atomic rollback is done when the configuration can be applied with NO errors.
Best Effort rollback will ignore errors and push the configuration onto the system.
Stop At First Failure will process the rollback request until it hits an error and then stops.

The default rollback type is Atomic and this is likely the most common rollback method you would use on a production environment. I am not aware of many folks wanting to rollback to a "Stop At First Failure" or "Best Effort" scenario situation unless true desperation has kicked in. There might be a case of the order of rollback if you are using VDC's and moving physical resources from one VDC to the other in which case perhaps Best Effort might be useful.

Also of note, the rollback feature must be used per Virtual Device Context (VDC), in other words, you have to run the command in each VDC. This is expected behavior as each VDC is it's own NX-OS instance and you have to run all the same commands to get the desired behavior out of the NX-OS platform.

The command itself is very simple:
checkpoint {checkpoint name} description {a description} | filename {path and filename}
Example: checkpoint cp-running-config-known-good-2010-03-22 description checkpoint of running config

There are some restrictions on the checkpoint name (max length 80 characters) and there are restrictions on the filename (max length of 75 characters and filename can't start with the word "system") but otherwise it is pretty straightforward process to get this going. I am using this on NX-OS version 4.3.1, earlier versions had more restrictions on file names and such so read the documentation if you are on an earlier release.

To see what the checkpoint command does you can use the show commands. To see all the checkpoints that are in a given VDC:
show checkpoint all
show checkpoint summary

The checkpoint command basically keeps a small database of checkpoints to allow you to rollback to a specific one and calculates the differences between a current state or checkpoint and that checkpoint you want to move to. It will generate a rollback script when you use the rollback command. If you want to see the differences that are being generated you can do that too:
show diff rollback-patch {checkpoint source name | running-config | startup-config | file filename} {checkpoint destination name | running-config | startup-config | file filename}
Example: show diff rollback-patch running-config checkpoint cp-running-config-known-good-2010-03-22

To actually do a rollback:
rollback running-config {checkpoint cp name | running-config | startup-config | file filename} {atomic | best-effort | stop-at-first-failure}
Example: rollback running-config checkpoint cp-running-config-known-good-2010-03-22 atomic

To see the status of rollbacks:
show rollback log

You can also clear out the checkpoint history and files, use the command with caution.
clear checkpoint database

This is a VERY useful command to build into your scripts prior to pushing out production changes on gear. It allows you to have a well known state stored locally and be able to rollback to it quickly in case of problems in your scripts. Awesome!
- Ed

Friday, March 19, 2010

Cisco IPSec VPN Client - 5.0.7 BETA - Win7 64-bit support

Cisco has a beta version of the IPSec VPN Client out, version 5.0.7 BETA (vpnclient-winx64-msi- available for download. It appears they got the message about the need for a 64-bit version of the IPSec client for Windows 7! It is available for download on CCO but requires a valid CCO login and current contract to get the code.

This is great news for Microsoft customers that have Cisco ASA's, PIX's or VPN 3000 concentrators deployed and their IT team is migrating their client OS's to Windows 7. Many OEM's are shipping the default OS as Windows 7 64-bit to take advantage of the all the RAM systems can support today.

Cisco is still pushing their AnyConnect client + ASA platform to compete with Microsoft's DirectAccess solution on Forefront UAG. Cisco is positioning the AnyConnect client as an always ready vpn client that auto-reconnects and is seamless in that process. I don't know if I am buying that position given the flexibility of what DirectAccess can do and how you can manage the policies with GPO's in AD in the Microsoft solution. Plus the fact that once DirectAccess is deployed there is no VPN "client" per-say, it is more of a policy function which is a very unique position in terms of product positioning.

At least if you have a Cisco IPSec solution deployed today you can now leverage it with Windows 7 without requiring any third party software to get around the problem of being on a 64-bit OS.
- Ed

Wednesday, March 17, 2010

Cisco ASA 8.3 code - read the release notes!

Cisco released version 8.3(1) for the ASA on March 8th. The release notes have some interesting items. Specifically I think the important ones are outlined below and are quoted from the release notes or related documents.

Before you just upload 8.3(1) READ THE MIGRATION GUIDE!! They have made some BIG changes in how NAT and PAT work. The new code will convert your configuration, you have been warned! Save your work prior to the upgrade. Let me repeat that, save it someplace other than the ASA so you have a clean copy prior to the migration.

From the migration document:
"The major changes in Version 8.3 that require migration are:

Real IP addresses in access lists, where access lists are used in supported features—When using NAT or PAT, you used to have to specify the mapped addresses and ports in an access list for all features that use access lists. Now, for several supported features, you must use the real, untranslated IP address and ports. (Other features continue to use the mapped IP address).

NAT—The NAT feature has been redesigned for increased flexibility and functionality. All NAT and NAT-related commands have been redesigned.

Named Network and Service Objects—Network and service objects are automatically created and used for several features, including NAT and access lists that are used for access rules. "

If you have existing static entries in your ASA configuration, and I would guess most folks do, if you are allowing any inbound services then get ready for a migration. The static statements stay the same but the ACL and object-groups change! The important things is to figure out what NAT configuration you match and how it will look after the migration. The migration guide has a table that outlines this for you. Read the migration guide to make sure you understand how your configuration is going to change, this is not a minor change!

Also, in the release notes:
"To run Version 8.3 in a production environment, you need to upgrade the memory on the Cisco ASA 5505, 5510, 5520, or 5540"

The memory requirements are outlined in the doc, needless to say this is not a minor release as they are requiring a pretty big memory uplift even for the smallest of the ASA's. Make sure you order the memory upgrade kits if you plan to run the 8.3x version code at all.

"You can now create named network objects that you can use in place of a host, a subnet, or a range of IP addresses in your configuration and named service objects that you can use in place of a protocol and port in your configuration. You can then change the object definition in one place, without having to change any other part of your configuration. This release introduces support for network and service objects in the following features:


Access lists

Network object groups

The following commands were introduced or modified: object network, object service, show running-config object, clear configure object, access-list extended, object-group network."

The change to allow the use of objects is actually a good thing but it is going to take awhile before people are used to using it in NAT statements for instance. I am actually excited about this one as it will make life much easier for those that plan out their network and resources well. It also allows for much easier change control and scripting to get hosts in and out of rules on the firewall.

"Displays the timestamp, along with the hash value and hit count, for a specified access list. The following command was modified: show access-list. "

For those of us who spend a lot of time looking at access-list hit counts having the timestamp and hash value is a huge plus. It is the simple things. Now if only they would allow multiple | and grep commands.

There are some items about licensing that are worth noting and if you are doing UC they have made changes to improve UC-IME. There is plenty to read up on, definitely a lot to keep up on.
- Ed

Friday, March 12, 2010

New Windows networking blog and other blogs to check out

Joe Davies is maintaining a new blog location that I think will be useful for those that are into network topics about the Windows platform. The blog is up on technet and is The Windows Server User Assistance Networking (WSUAN) team's blog. It covers all things networking related including subjects like DNS, DHCP, DirectAccess, BranchCache, Windows Firewall and my personal favorite, IPv6.

Also worth checking out is the Springboard Series blog that Stephen Rose runs - good stuff there for IT Pros.

There are several other personal blogs worth mentioning. My short list, for those that are interested, of ones I look at.
Jennelle Crothers at http://www.techbunny.com/
Susan Bradley at http://msmvps.com/blogs/bradley/Default.aspx
Colin McNamara at http://www.colinmcnamara.com/
Scott Lowe at http://blog.scottlowe.org/
Steve Riley at http://stvrly.wordpress.com/
Jeremy Stretch at http://packetlife.net/blog/
Jessica Deen at http://win7geek.wordpress.com/

There are a ton more out there that are really good, when I get a moment I will dump down my list to another post.
- Ed

Thursday, March 11, 2010

Cisco Nexus vPC port channel notes

Ran into an interesting issue with turning up vPC's between a Cisco Nexus 7010 and Cisco Nexus 5010's in regards to have the underlying port channels are configured. It turns out that if the 7k and 5k pairs have different port channel configurations they do not appropriately block the way you would think they should but instead continue to bounce the port. The mistake in my configuration was simple, the 5k was configured as such:
int e2/1
channel-group 10 mode active

int e2/2
channel-group 10 mode active

int p10
switchport mode trunk
vpc 10

The 7k was set as:
int e1/1
channel-group 10

int e1/2
channel-group 10

int p10
switchport mode trunk
vpc 10

The 5k was going LACP and the 7k was forced on. What was odd was that the vPC reported down as expected but the ports kept bouncing up and down which then forced spanning-tree to go into effect, not something I desired to have happen in this misconfiguration situation. I would have been ok with the port(s) staying down due to a port channel type mismatch but it kept trying to bring the ports up which was causing loops and causing spanning-tree to do its thing.

What I found interesting was that if I simply disabled one of the 7k ports, for instance int e1/2 everything would stabilize out since there was not a loop (no spanning tree) and the vPC still reported down. Expected behavior when you think about it but not intuitive when debugging the problem.

Once I corrected the port-channel configuration and forced everything on the vPC came up and I was able to fail interfaces with no interruption of traffic. My concern regarding this is the impact it had on all the vlans associated with that trunk, so it is critical to make sure that adding any new 5k's to the 7k's that the port configurations are double checked or an outage is possible due to this problem. I sort of wish the ports would error disable out due to vPC determining that the port channel type doesn't match to help prevent the L2 loop situation. This would keep the ports from bouncing and therefore prevent spanning-tree from having to go into effect to do loop prevention.
- Ed

Friday, March 05, 2010

Cisco Nexus 7000 - 32 port 10Gig linecard caveats

I have been working on a set of Cisco Nexus 7010's that both have 32 port 10G line cards. I have been able to get virtual PortChannel (vPC) and Virtual Device Context (VDC) set up and working on the platform, definitely some of the coolest technology out there for the data center outside of server virtualization. That being said, there are some interesting caveats that folks should pay attention to when planning out the total number of usable 10G ports on the 32 port line card.

First, the ports are "grouped" in sets of four (1,3,5,7 for instance) - each group of four is capable of doing 10G in shared mode or a single port in the group (the first one) is capable of 10G in dedicated mode and the other 3 are shutdown. Effectively the card is over subscribed 4 to 1 - not horrible. This means it has a total of 80Gbps into the backplane. All this is pretty well documented by Cisco so it's not new or an unknown.

So, what isn't obvious when you are doing planning and design? Turns out if you are designing a network to utilize VDC's in the 7k's than you need to pay closer attention to the number of usable 10G interfaces you get per VDC. Because the ports are grouped and sharing ASIC resource they cannot be individually allocated to a VDC but have to be allocated as a group.

So if you plan on needing just a single 10G interface in a VDC in reality you will have to allocate a full group of 4 10G ports (and in the correct grouping order.) Even if you decide you want to use the dedicated mode to allow full 10G for a single 10G port and leave the other 3 in the group shutdown you still have to allocate all 4 10G ports to the VDC. When you think about it this makes sense but often in the design phase while white boarding out a solution an engineer will simply allocate a single 10G interface for a VDC because that is all that will be "used" but in reality you will be allocating 4 not 1.

This can throw off your 10G port count by a lot depending on how many VDC's you plan to run. You can only run up to 4 VDC's on the 7k's today though in theory there is no reason they couldn't raise that as it is only a software limitation. So, in reality all VDC's should be allocated 4 10G ports even if you only need 1 10G port for something. That way you don't run into any surprises when trying to deploy a solution. Also, for deployments that make use of two 7k's (which really is how you should do it to gain the benefit of vPC) I would trade a dual supervisor per chassis configuration for an additional 32 port 10G line card if I had to chose. It makes more sense to have more flexibility in the number and allocation of 10G interfaces with a single supervisor per chassis solution since you will have two chassis' anyway for redundancy. Just a thought when planning and budgeting.
- Ed