Monday, August 23, 2010
VMworld 2010 - August 30 - Sept 2nd
Another VMworld is around the corner, just a week away now - it is again in San Francisco at Moscone Center and should be an interesting event as always. Cisco, EMC and NetApp will all have big presence at the show for sure and it is nice to see what they have in terms of product lines and what they are doing in the virtualization space. What makes me want to attend the expo every year though is the small niche companies that have interesting products or way of solving problems that the big boys haven't figured out or haven't thought about.
Often it is the case that these folks get purchased and integrated into one of the bigger players so part of the fun is getting to see them while they are still small, still agile and still interesting plus getting their swag while they still have their own logos and tag lines and such.
I expect to see some announcements associated with the show regarding partnerships, new product launches and perhaps some heated debate too. It looks like Microsoft will once again be relegated to a regular small booth which is a shame considering the majority of deployments of VMware's products are to support Microsoft platforms. I think VMware is making a mistake not allowing Microsoft to have a bigger presence at the event and simply asking them to live within some guidelines regardings Hyper-V.
- Ed
Often it is the case that these folks get purchased and integrated into one of the bigger players so part of the fun is getting to see them while they are still small, still agile and still interesting plus getting their swag while they still have their own logos and tag lines and such.
I expect to see some announcements associated with the show regarding partnerships, new product launches and perhaps some heated debate too. It looks like Microsoft will once again be relegated to a regular small booth which is a shame considering the majority of deployments of VMware's products are to support Microsoft platforms. I think VMware is making a mistake not allowing Microsoft to have a bigger presence at the event and simply asking them to live within some guidelines regardings Hyper-V.
- Ed
Friday, August 06, 2010
Splunk User Conference - August 9-11, 2010
One of the tools I use to help me wade through the mass of data that networking devices spit out is Splunk. Splunk is easily one of the most useful apps to run if you have to try and figure out what is going on with regards to firewalls, ids, network logging and anything else that will output to syslog.
If you happen to live in the San Francisco Bay Area you are in luck too, next week is the first annual Splunk User Conference and you can still sign up. I wish I could make it myself, especially since I live local to the event.
Splunk has really expanded the capabilities of the product but to be honest, the simple search and filtering to help me write better access control lists for firewalls is a huge time saver, just being able to do that quickly and efficiently makes it worth it's weight in gold.
- Ed
If you happen to live in the San Francisco Bay Area you are in luck too, next week is the first annual Splunk User Conference and you can still sign up. I wish I could make it myself, especially since I live local to the event.
Splunk has really expanded the capabilities of the product but to be honest, the simple search and filtering to help me write better access control lists for firewalls is a huge time saver, just being able to do that quickly and efficiently makes it worth it's weight in gold.
- Ed
Thursday, July 29, 2010
Why paying attention to IPv6 is now important
As a personal interest I have been working with and following developments of IPv6 for several years. I started presented on IPv6 back in 2006 because of what was happening with Windows Vista and the changes that Microsoft was doing with the OS and their new networking stack. Here we are in 2010 and I think we are past early adopters in regards to IPv6. In fact, I now think if you are not paying attention to what is happening with IPv6 it could start impacting your ability to perform your job soon, especially if you are an IT Professional.
So, who do I think will be impacted the most by the transition and more prolific use of IPv6? I think you might be surprised.
The standard answer is network engineers and granted they indeed will be rolling out and maintaining dual IPv4/IPv6 networks for years to come but I actually don't think IPv6 will be as much of a challenge for network engineers to get up and running assuming they have moderately newer network equipment. Granted, there are issues with lack of feature parity but that will be resolved over time and will be fast tracked when equipment manufactures realize they are losing sales due to the lack of the parity.
Next on the list is system admins. I think many will find IPv6 to be a bit more of a challenge in regards to the differences in behavior of the protocol and getting worked out the behavior differences of applications as a result. This is a huge issue for client machines in terms of what OS you are running on your desktop and what the server does or does not support. I would argue that the majority of system admins know enough IPv4 networking to allow them to do their job but likely will have some challenges with differences in IPv6. I know there are some great system admins out there who could run networks also so obviously this a wildly general statement but I still feel there is going to be a bigger learning curve for system admins than they care to admit. Perhaps it is time for Microsoft to bring back a dedicated networking exam - like the old MCP TCP/IP exam?
The surprise group is application developers and database admins. Just think about how much code has been written out there to account for IPv4 addresses. IPv4 addresses are 32bit and I would imagine the majority of applications out there are storing that value under a declared INTEGER. I could be wrong - maybe they are all stored as a STRING instead but I have a feeling that isn't the case. IPv6 addresses on the other hand are 128bit and likely the majority of applications will have to be modified to account for the new size, difference in how they are represented (in HEX not DEC) and also the fact that the application might potentially have to pay attention to which interface it is directing traffic through. This doesn't even cover all the databases out there that are storing IPv4 information and the SQLNET statements all based around IPv4 to query those databases.
To top it all off, IPv6 can represented an address in multiple ways due to the zero compression option. So searching through logs or analyzing output could be an additional issue unless some standards are agreed to in advance in terms of how to store and represent an IPv6 address. So imagine trying to correlate information from multiple systems and they can't match stuff because the IPv6 addresses are represented differently in each system. I think some of these issues will be the biggest road blocks to overcome in the months and years ahead for IPv6.
So, why is it important to pay attention to IPv6 now? It is important because the adoption and momentum behind the protocol has already begun. Major content providers like Facebook and network providers like Comcast and content delivery providers like Limelight have all deployed IPv6 already and are doing their trials now. If you have no knowledge or understanding of IPv6 how will you address your business needs when you need to either access content, deliver content or work with a network when you don't understand the protocol they are using to move traffic?
In short, if you don't have a working understanding of IPv6, you are already behind. Take a quick quiz. Do you know what behavior Windows 7 has when it has a public IPv4 address? What is different if it gets a public IPv6 address? Which protocol does it use for DNS resolution if it has both an IPv4 and IPv6 address? Does the type of IPv6 address it has matter to the default behavior? This is all just for Windows 7, now do this for OSX, Windows Vista, Windows XP, Windows Server 2003, 2008, 2008R2, Linux and Solaris. How did you do?
- Ed
So, who do I think will be impacted the most by the transition and more prolific use of IPv6? I think you might be surprised.
The standard answer is network engineers and granted they indeed will be rolling out and maintaining dual IPv4/IPv6 networks for years to come but I actually don't think IPv6 will be as much of a challenge for network engineers to get up and running assuming they have moderately newer network equipment. Granted, there are issues with lack of feature parity but that will be resolved over time and will be fast tracked when equipment manufactures realize they are losing sales due to the lack of the parity.
Next on the list is system admins. I think many will find IPv6 to be a bit more of a challenge in regards to the differences in behavior of the protocol and getting worked out the behavior differences of applications as a result. This is a huge issue for client machines in terms of what OS you are running on your desktop and what the server does or does not support. I would argue that the majority of system admins know enough IPv4 networking to allow them to do their job but likely will have some challenges with differences in IPv6. I know there are some great system admins out there who could run networks also so obviously this a wildly general statement but I still feel there is going to be a bigger learning curve for system admins than they care to admit. Perhaps it is time for Microsoft to bring back a dedicated networking exam - like the old MCP TCP/IP exam?
The surprise group is application developers and database admins. Just think about how much code has been written out there to account for IPv4 addresses. IPv4 addresses are 32bit and I would imagine the majority of applications out there are storing that value under a declared INTEGER. I could be wrong - maybe they are all stored as a STRING instead but I have a feeling that isn't the case. IPv6 addresses on the other hand are 128bit and likely the majority of applications will have to be modified to account for the new size, difference in how they are represented (in HEX not DEC) and also the fact that the application might potentially have to pay attention to which interface it is directing traffic through. This doesn't even cover all the databases out there that are storing IPv4 information and the SQLNET statements all based around IPv4 to query those databases.
To top it all off, IPv6 can represented an address in multiple ways due to the zero compression option. So searching through logs or analyzing output could be an additional issue unless some standards are agreed to in advance in terms of how to store and represent an IPv6 address. So imagine trying to correlate information from multiple systems and they can't match stuff because the IPv6 addresses are represented differently in each system. I think some of these issues will be the biggest road blocks to overcome in the months and years ahead for IPv6.
So, why is it important to pay attention to IPv6 now? It is important because the adoption and momentum behind the protocol has already begun. Major content providers like Facebook and network providers like Comcast and content delivery providers like Limelight have all deployed IPv6 already and are doing their trials now. If you have no knowledge or understanding of IPv6 how will you address your business needs when you need to either access content, deliver content or work with a network when you don't understand the protocol they are using to move traffic?
In short, if you don't have a working understanding of IPv6, you are already behind. Take a quick quiz. Do you know what behavior Windows 7 has when it has a public IPv4 address? What is different if it gets a public IPv6 address? Which protocol does it use for DNS resolution if it has both an IPv4 and IPv6 address? Does the type of IPv6 address it has matter to the default behavior? This is all just for Windows 7, now do this for OSX, Windows Vista, Windows XP, Windows Server 2003, 2008, 2008R2, Linux and Solaris. How did you do?
- Ed
Tuesday, July 20, 2010
Cisco ASA NetFlow configuration
I have been setting up a lot more NetFlow on Cisco ASA's recently. Mainly due to the request for more visibility into the traffic that is consuming Internet bandwidth and for compliance reasons. It seems that even with proxy services and other solutions many IT organizations still have a poor understanding of the actual traffic traversing their network.
Since NetFlow is limited in terms of platform support (specifically the Cisco Catalyst 3k/2k switches do NOT support it) but the Cisco ASA does I have been asked to turn it on the ASA to have a better idea what is going across the network. Cisco has a nice Introduction to Cisco IOS NetFlow if you need to run it on the routing or 4500/6500 platforms which is a great way to go in addition to the ASA as you can then see what is happening between devices on the network also.
Chapter 75 in the Cisco ASA 8.2 CLI Configuration Guide covers how to set up a NetFlow configuration. Here is a short script to get it up and running quickly. A couple of caveats, read the config guide because it covers the parameters for timing and limiting what you are collecting. This script is a "let's get this going, send me everything" sort of solution. Not optimal for heavily loaded ASA's. Should be good enough to get you going though.
!#####
! - NetFlow script for Cisco ASA
!
! - ACL to catch all IP traffic - to specify the traffic you are interested in
access-list flow_export_acl extended permit ip any any
!
! - set up the destination server ip and template rate
flow-export destination {interface name} {IP address} {port #}
flow-export template timeout-rate 1
!
! - build out the class-map for the flow that matches the ACL
class-map flow_export_class
match access-list flow_export_acl
! - or don't use an ACL by using
match any
!
! - build out the policy-map
policy-map flow_export_policy
class flow_export_class
flow-export event-type all destination {IP address}
!
! - apply the policy-map to whatever global policy you have or make one
service-policy flow_export_policy global
!
! - if you have an existing policy-map apply the class-map into that one
! - for instance the default ASA service-policy for global is global_policy
! - so you could add the class-map to it by doing
policy-map global_policy
class flow_export_class
flow-export event-type all destination {IP address}
!
!#####
You can get information about what the ASA is doing in terms of the flow output by using the following commands:
show flow-export counters
show service-policy global flow ip host {source IP} host {dest IP}
show access-list
Obviously you need some sort of NetFlow collector. There are a lot of professional and free tools to do this and there are some great vendors doing this. That being said, I have used Plixer's Scrutinizer free product to at least get folks up and working and have a functional tool to look at until they can decide what tool they want to use. It does and excellent job of showing what is possible in terms of reporting and information gathering. That being said the following companies also have NetFlow commercial products you should consider or have free offerings that can be used.
AdventNet
NetQoS - part of CA now
NTOP - opensource tool
SolarWinds
I am leaving off a ton of vendors in the list but I have found if I list everyone who is doing a solution then folks who are trying it out freeze up and can't pick one. I know, not a great reason but I rather have them use something than nothing at all.
Cisco owns the NetFlow name but there is a standards version of NetFlow supported by many other networking vendors called sFlow. Basically it provides the same sort of function but on other vendor equipment. This means that almost all NetFlow collectors can work with sFlow. So if desired you can collect from non Cisco devices that support sFlow to the same collector to get a more complete view from around your network.
If you have not deployed and made use of NetFlow I really recommend doing a quick trial run. You may be surprised by what you find. I've had clients discover employee's watching video and tv shows being pulled from foreign countries (some of questionable content), others consuming high bandwidth across tunneled links they did not know they were running and lots of other interesting items. Many have been surprised how much IPv6 and tunneled IPv6 they are running on their network. It is a great tool so check it out.
- Ed
Since NetFlow is limited in terms of platform support (specifically the Cisco Catalyst 3k/2k switches do NOT support it) but the Cisco ASA does I have been asked to turn it on the ASA to have a better idea what is going across the network. Cisco has a nice Introduction to Cisco IOS NetFlow if you need to run it on the routing or 4500/6500 platforms which is a great way to go in addition to the ASA as you can then see what is happening between devices on the network also.
Chapter 75 in the Cisco ASA 8.2 CLI Configuration Guide covers how to set up a NetFlow configuration. Here is a short script to get it up and running quickly. A couple of caveats, read the config guide because it covers the parameters for timing and limiting what you are collecting. This script is a "let's get this going, send me everything" sort of solution. Not optimal for heavily loaded ASA's. Should be good enough to get you going though.
!#####
! - NetFlow script for Cisco ASA
!
! - ACL to catch all IP traffic - to specify the traffic you are interested in
access-list flow_export_acl extended permit ip any any
!
! - set up the destination server ip and template rate
flow-export destination {interface name} {IP address} {port #}
flow-export template timeout-rate 1
!
! - build out the class-map for the flow that matches the ACL
class-map flow_export_class
match access-list flow_export_acl
! - or don't use an ACL by using
match any
!
! - build out the policy-map
policy-map flow_export_policy
class flow_export_class
flow-export event-type all destination {IP address}
!
! - apply the policy-map to whatever global policy you have or make one
service-policy flow_export_policy global
!
! - if you have an existing policy-map apply the class-map into that one
! - for instance the default ASA service-policy for global is global_policy
! - so you could add the class-map to it by doing
policy-map global_policy
class flow_export_class
flow-export event-type all destination {IP address}
!
!#####
You can get information about what the ASA is doing in terms of the flow output by using the following commands:
show flow-export counters
show service-policy global flow ip host {source IP} host {dest IP}
show access-list
Obviously you need some sort of NetFlow collector. There are a lot of professional and free tools to do this and there are some great vendors doing this. That being said, I have used Plixer's Scrutinizer free product to at least get folks up and working and have a functional tool to look at until they can decide what tool they want to use. It does and excellent job of showing what is possible in terms of reporting and information gathering. That being said the following companies also have NetFlow commercial products you should consider or have free offerings that can be used.
AdventNet
NetQoS - part of CA now
NTOP - opensource tool
SolarWinds
I am leaving off a ton of vendors in the list but I have found if I list everyone who is doing a solution then folks who are trying it out freeze up and can't pick one. I know, not a great reason but I rather have them use something than nothing at all.
Cisco owns the NetFlow name but there is a standards version of NetFlow supported by many other networking vendors called sFlow. Basically it provides the same sort of function but on other vendor equipment. This means that almost all NetFlow collectors can work with sFlow. So if desired you can collect from non Cisco devices that support sFlow to the same collector to get a more complete view from around your network.
If you have not deployed and made use of NetFlow I really recommend doing a quick trial run. You may be surprised by what you find. I've had clients discover employee's watching video and tv shows being pulled from foreign countries (some of questionable content), others consuming high bandwidth across tunneled links they did not know they were running and lots of other interesting items. Many have been surprised how much IPv6 and tunneled IPv6 they are running on their network. It is a great tool so check it out.
- Ed
Labels: Cisco ASA, NetFlow, sFlow
Thursday, July 15, 2010
The four datacenter horsemen - who will they be?
I've been watching some of the network manufacture market transitions happening right now and the rapid changes going on in the data center and networking market space. I think there will be four main players who I am calling the four datacenter horsemen who will be calling the shots in the future - leading players of the apocalypse I guess. This is all me having fun guessing what will go on so don't hold me to it and of course, this is only my opinion.
Team #1 is made up of Cisco/EMC/VMware due to their partnership arrangement. This partnership for taking over the data center is what caused all sorts of realignments in the industry in the first place. Specifically, with the falling out of HP and Cisco over servers and HP now having a full portfolio of network, servers and storage to go after the data center space that Cisco has traditionally shared with HP it begs the question - what next?
Granted I agree that HP, team #2, does not have a fully competitive network/storage solution when compared to the Cisco Nexus + UCS solution but they are close enough that folks are picking sides. Certainly it can be argued that HP has much more depth in the server market space and a longer deeper relationship with Microsoft to cover the hypervisor gap.
So, what other teams are out there? We can't leave IBM out of the game, they have a partner arrangement with Juniper and I would not be surprised if IBM considered buying Juniper up to cover the gap in the network portfolio. IBM has storage covered and has been doing professional services and large scale data center work since the beginning of the industry - so I say they are team #3.
Who is team #4? I think Oracle with their recent Sun acquisition is the answer. I also think since they have a storage arm from Sun that they could easily fill the networking gap by buying Brocade and then potentially developing in house (Sun team) a networking solution complement to the Foundry arm of Brocade or picking up a smaller Ethernet vendor like Force10 or Extreme to help round out the portfolio. Oracle already has a hypervisior and so does Sun so they have lots of software to leverage and they can strong arm customers into buying a "blessed" data center deployment solution that runs Oracle top to bottom and compete against everyone else.
So, who is left out of this game? I guess Dell falls back to a distant #5, and while they technically have server, storage (EqualLogic) and network their story is incomplete and their solution do not align with a complete data center story. Also marginalized are SAP, Microsoft, Novell, RedHat, Citrix and several others that used to have very strategic partnerships which now will not be as important for those four horsemen in the immediate future. And then there are folks like NetApp, F5, Riverbed, Infoblox and others who will have to fit into this ecosystem partner arrangement without being swallowed up.
I think the next 3-4 years are going to be some of the most interesting and fast moving for the data center market space. To think, we haven't even addressed the SaaS/Cloud market space that Amazon (AWS), Microsoft (Azure) and others are rapidly pushing forward. Perhaps Microsoft will leapfrog everyone at the end of the day and the four datacenter horsemen won't even be relevant because you won't need a data center anymore. Hard to imagine? Is that the real apocalypse for data center, where the four horsemen aren't even relevant to the process for most customers?
- Ed
Team #1 is made up of Cisco/EMC/VMware due to their partnership arrangement. This partnership for taking over the data center is what caused all sorts of realignments in the industry in the first place. Specifically, with the falling out of HP and Cisco over servers and HP now having a full portfolio of network, servers and storage to go after the data center space that Cisco has traditionally shared with HP it begs the question - what next?
Granted I agree that HP, team #2, does not have a fully competitive network/storage solution when compared to the Cisco Nexus + UCS solution but they are close enough that folks are picking sides. Certainly it can be argued that HP has much more depth in the server market space and a longer deeper relationship with Microsoft to cover the hypervisor gap.
So, what other teams are out there? We can't leave IBM out of the game, they have a partner arrangement with Juniper and I would not be surprised if IBM considered buying Juniper up to cover the gap in the network portfolio. IBM has storage covered and has been doing professional services and large scale data center work since the beginning of the industry - so I say they are team #3.
Who is team #4? I think Oracle with their recent Sun acquisition is the answer. I also think since they have a storage arm from Sun that they could easily fill the networking gap by buying Brocade and then potentially developing in house (Sun team) a networking solution complement to the Foundry arm of Brocade or picking up a smaller Ethernet vendor like Force10 or Extreme to help round out the portfolio. Oracle already has a hypervisior and so does Sun so they have lots of software to leverage and they can strong arm customers into buying a "blessed" data center deployment solution that runs Oracle top to bottom and compete against everyone else.
So, who is left out of this game? I guess Dell falls back to a distant #5, and while they technically have server, storage (EqualLogic) and network their story is incomplete and their solution do not align with a complete data center story. Also marginalized are SAP, Microsoft, Novell, RedHat, Citrix and several others that used to have very strategic partnerships which now will not be as important for those four horsemen in the immediate future. And then there are folks like NetApp, F5, Riverbed, Infoblox and others who will have to fit into this ecosystem partner arrangement without being swallowed up.
I think the next 3-4 years are going to be some of the most interesting and fast moving for the data center market space. To think, we haven't even addressed the SaaS/Cloud market space that Amazon (AWS), Microsoft (Azure) and others are rapidly pushing forward. Perhaps Microsoft will leapfrog everyone at the end of the day and the four datacenter horsemen won't even be relevant because you won't need a data center anymore. Hard to imagine? Is that the real apocalypse for data center, where the four horsemen aren't even relevant to the process for most customers?
- Ed
Labels: Brocade, Cisco, HP, IBM, Microsoft, Oracle, Sun
Thursday, July 01, 2010
Microsoft MVP renewal - this time Desktop Experience
I am happy to say I have been renewed as a Microsoft MVP. This year I've been awarded in the Desktop Experience category. This is my third category so far in the program. A brief look back as I was first award in July 2004.
2004 - Windows Server - Networking
2005 - Windows Server - Networking
2006 - Windows Server - Networking
2007 - Windows Server - Networking
2008 - Enterprise Security - Networking
2009 - Enterprise Security - Networking
2010 - Desktop Experience
I still plan on being actively involved and paying attention to what is going on in Security and more specifically in networking. Most of the focus will be around IPv6 as I am now a co-chair on the California IPv6 Task Force.
I needs to extend some thanks to some Microsoft folks:
Jake Grey - MVP Lead
Emily Freet - MVP Regional Manager, Americas
Stephen Rose - Sr. Community Manager - Windows OS
Chris Avis, Harold Wong and Chris Henley - IT Pro Evangelist - DPE - West Region team
Also, thanks to Jennelle Crothers - fellow Microsoft MVP and PacITPros member for helping to make the User Group experience what it is, couldn't have made it this far without your help.
- Ed
2004 - Windows Server - Networking
2005 - Windows Server - Networking
2006 - Windows Server - Networking
2007 - Windows Server - Networking
2008 - Enterprise Security - Networking
2009 - Enterprise Security - Networking
2010 - Desktop Experience
I still plan on being actively involved and paying attention to what is going on in Security and more specifically in networking. Most of the focus will be around IPv6 as I am now a co-chair on the California IPv6 Task Force.
I needs to extend some thanks to some Microsoft folks:
Jake Grey - MVP Lead
Emily Freet - MVP Regional Manager, Americas
Stephen Rose - Sr. Community Manager - Windows OS
Chris Avis, Harold Wong and Chris Henley - IT Pro Evangelist - DPE - West Region team
Also, thanks to Jennelle Crothers - fellow Microsoft MVP and PacITPros member for helping to make the User Group experience what it is, couldn't have made it this far without your help.
- Ed
Labels: Microsoft MVP
Tuesday, June 29, 2010
Cisco Live!
I can't attend Cisco Live! this week but you can see some of the keynotes and other interesting items on the website. Looks like Cisco is announcing a bunch of stuff this year which is good news. The only thing I wish I was attending for was to get involved and be able to ask the tough questions about road map and support for IPv6. A lot of that was done at the Google IPv6 Implementors Event a few weeks ago but upper Cisco management and in particular - the product groups, were NOT at that event. Hard to see how important something is if key folks aren't paying attention.
I hope Cisco starts to push internally feature parity for IPv6 - there is a huge gap in "supporting" IPv6 and in having feature parity to their existing product portfolio. It will come over time but I think a bigger push needs to happen... and soon.
- Ed
I hope Cisco starts to push internally feature parity for IPv6 - there is a huge gap in "supporting" IPv6 and in having feature parity to their existing product portfolio. It will come over time but I think a bigger push needs to happen... and soon.
- Ed
Monday, June 14, 2010
NANOG 49 - San Francisco
For those that do not follow what is happening in the Internet Service Provider space much you might find it interesting that the North American Network Operators' Group (NANOG) is hosting their meeting in San Francisco this week. Many folks came in early to attend the Google IPv6 Implementors Conference that happened last Thursday and Friday in Mt. View and are now having fun in San Francisco.
The agenda has lots of IPv6 on it also, I am sort of bummed I will not have time to make it to event but I hope all those attending are enjoying it.
I also got to sit in on a conference call today about how the new North American IPv6 Task Force and the other regional task forces (like the California one) will be structuring and building out resources. Stay tuned for some cool events, labs facilities and meetings to happen across the country soon based out of this work. In the mean time, if you are interested in starting to hear about what will be happening with IPv6 in California you can follow http://twitter.com/CAv6TF and we will let you know when things are going on.
- Ed
The agenda has lots of IPv6 on it also, I am sort of bummed I will not have time to make it to event but I hope all those attending are enjoying it.
I also got to sit in on a conference call today about how the new North American IPv6 Task Force and the other regional task forces (like the California one) will be structuring and building out resources. Stay tuned for some cool events, labs facilities and meetings to happen across the country soon based out of this work. In the mean time, if you are interested in starting to hear about what will be happening with IPv6 in California you can follow http://twitter.com/CAv6TF and we will let you know when things are going on.
- Ed
Thursday, June 10, 2010
Google IPv6 Implementors Conference
Attending the Google IPv6 Implementors Conference - the slide decks are available on the site to download. Great presentations on what is happening in the IPv6 space.
Definitely check out the presentations, especially the Facebook presentation - they turned up their experimental IPv6 service in short order and are only using 3 IPv6 addresses to get it deployed, remarkable. To top it off they spend no new capital to make it happen.
I have a feeling the content here is a bit better than what is going on at TechEd but perhaps the gauntlet has been thrown - come on Microsoft, where are you on this?
- Ed
Definitely check out the presentations, especially the Facebook presentation - they turned up their experimental IPv6 service in short order and are only using 3 IPv6 addresses to get it deployed, remarkable. To top it off they spend no new capital to make it happen.
I have a feeling the content here is a bit better than what is going on at TechEd but perhaps the gauntlet has been thrown - come on Microsoft, where are you on this?
- Ed
Labels: IPv6
Friday, June 04, 2010
California IPv6 Task Force
I am excited to announce that I have been appointed a co-chair on the California IPv6 Task Force handling the Bay Area. The California IPv6 Task Force (CAv6TF) is chartered with doing advocacy and education regarding IPv6 in the state of California.
With my new appointment comes the obvious questions, what sort of events should the CAv6TF do to cover both advocacy and education? I would love to hear from people about what they feel are the best ways to go about this prior to driving off with the bus half full. Specifically, what format and topics would be the best for the Bay Area crowd who definitely have their own unique requirements.
If you are interested in participating please follow us on twitter and keep an eye out for the relaunch of our website. We believe this will be a big year for IPv6 in general and I am personally excited to start working on the events!
- Ed
With my new appointment comes the obvious questions, what sort of events should the CAv6TF do to cover both advocacy and education? I would love to hear from people about what they feel are the best ways to go about this prior to driving off with the bus half full. Specifically, what format and topics would be the best for the Bay Area crowd who definitely have their own unique requirements.
If you are interested in participating please follow us on twitter and keep an eye out for the relaunch of our website. We believe this will be a big year for IPv6 in general and I am personally excited to start working on the events!
- Ed
IF-MAP - Infoblox and Juniper pushing a new standard
I attended an Infoblox and Juniper joint presentation on IF-MAP and it looks interesting. I think they are attempting to rally support around the standard so a great deal is self serving (asking folks to push for IF-MAP inclusion in RFP's for instance.) That being said, I do think the standard addresses a real need requirement in the industry regarding getting multi-vendor solutions to utilize a common state engineer (which is what IF-MAP really is) which anyone can leverage for different purposes. I primarily see it for a single database location for NAC and policy authentication and enforcement systems to share information in a transparent way and to subscribe to changes in the "state" of something and then take action based on those states.
Vendors who are doing policy enforcement should keep an eye on the standard to determine if they need to have a solution that will work with IF-MAP. What will be really interesting is if monitoring and alerting systems start utilizing the IF-MAP standard to learn what is happening on the network to drive alerting from layer 1 all the way up to application attributes. This already happens on the enforcement side but there is little in the way of alerting and monitoring notification that can do something similar.
I don't think this will change the landscape of the data center models being developed today by the likes of HP or Cisco but this is a critical issue to enterprise and commercial customers as a pain point and if vendors choose to implement IF-MAP then they certainly might gain an advantage in terms of interoperability.
- Ed
Vendors who are doing policy enforcement should keep an eye on the standard to determine if they need to have a solution that will work with IF-MAP. What will be really interesting is if monitoring and alerting systems start utilizing the IF-MAP standard to learn what is happening on the network to drive alerting from layer 1 all the way up to application attributes. This already happens on the enforcement side but there is little in the way of alerting and monitoring notification that can do something similar.
I don't think this will change the landscape of the data center models being developed today by the likes of HP or Cisco but this is a critical issue to enterprise and commercial customers as a pain point and if vendors choose to implement IF-MAP then they certainly might gain an advantage in terms of interoperability.
- Ed
Labels: IF-MAP, Infoblox, Juniper
Thursday, May 20, 2010
Cisco ASA - failover history
If you spend time working with Cisco ASA's in a failover configuration and you want to get a history of failures on the device the you should use the "show failover history" and "show failover" commands. I typically only use the "show failover" command when working on interface status (what is being watched for failover criteria) and the state of the interfaces themselves plus which unit is active and which is standby or failed. The nice thing about the "show failover history" command is it tells you when things happened in terms of failover status. Here is some sample output:
fw-1# sh fail history
==========================================================================
From State To State Reason
==========================================================================
07:49:22 PST Feb 4 2010
Not Detected Negotiation No Error
07:50:07 PST Feb 4 2010
Negotiation Just Active No Active unit found
07:50:07 PST Feb 4 2010
Just Active Active Drain No Active unit found
07:50:07 PST Feb 4 2010
Active Drain Active Applying Config No Active unit found
07:50:07 PST Feb 4 2010
Active Applying Config Active Config Applied No Active unit found
07:50:07 PST Feb 4 2010
Active Config Applied Active No Active unit found
14:04:39 PST Feb 11 2010
Active Failed Interface check
14:04:41 PST Feb 11 2010
Failed Standby Ready Interface check
14:04:46 PST Feb 11 2010
Standby Ready Just Active Failover state check
14:04:46 PST Feb 11 2010
Just Active Active Drain Failover state check
14:04:46 PST Feb 11 2010
Active Drain Active Applying Config Failover state check
14:04:46 PST Feb 11 2010
Active Applying Config Active Config Applied Failover state check
14:04:46 PST Feb 11 2010
Active Config Applied Active Failover state check
==========================================================================
So you can see that you get a history and state changes of the failover status. What is nice is that you don't have to do a debug to capture these status changes and it is in a nice summary table.
- Ed
fw-1# sh fail history
==========================================================================
From State To State Reason
==========================================================================
07:49:22 PST Feb 4 2010
Not Detected Negotiation No Error
07:50:07 PST Feb 4 2010
Negotiation Just Active No Active unit found
07:50:07 PST Feb 4 2010
Just Active Active Drain No Active unit found
07:50:07 PST Feb 4 2010
Active Drain Active Applying Config No Active unit found
07:50:07 PST Feb 4 2010
Active Applying Config Active Config Applied No Active unit found
07:50:07 PST Feb 4 2010
Active Config Applied Active No Active unit found
14:04:39 PST Feb 11 2010
Active Failed Interface check
14:04:41 PST Feb 11 2010
Failed Standby Ready Interface check
14:04:46 PST Feb 11 2010
Standby Ready Just Active Failover state check
14:04:46 PST Feb 11 2010
Just Active Active Drain Failover state check
14:04:46 PST Feb 11 2010
Active Drain Active Applying Config Failover state check
14:04:46 PST Feb 11 2010
Active Applying Config Active Config Applied Failover state check
14:04:46 PST Feb 11 2010
Active Config Applied Active Failover state check
==========================================================================
So you can see that you get a history and state changes of the failover status. What is nice is that you don't have to do a debug to capture these status changes and it is in a nice summary table.
- Ed
Labels: Cisco ASA
Monday, May 17, 2010
Cisco ASA flag descriptions
I'm always trying to remember the flag codes for the ASA connection command. Turns out it is in the help for the "show connection all" command, you just have to add the keyword "detail" so that you can see it. So, the command is:
show connection all detail
or for those that like shorter commands:
sh conn all d
The output for the flags (a bit better organized then what is displayed in the help) is:
Flags:
A - awaiting inside ACK to SYN
a - awaiting outside ACK to SYN
B - initial SYN from outside
b - TCP state-bypass or nailed
C - CTIQBE media
D - DNS
d - dump
E - outside back connection
F - outside FIN
f - inside FIN
G - group
g - MGCP
H - H.323
h - H.225.0
I - inbound data
i - incomplete
J - GTP
j - GTP data
K - GTP t3-response
k - Skinny media
M - SMTP data
m - SIP media
n - GUP
O - outbound data
P - inside back connection
p - Phone-proxy TFTP connection
q - SQL*Net data
R - outside acknowledged FIN
R - UDP SUNRPC
r - inside acknowledged FIN
S - awaiting inside SYN
s - awaiting outside SYN
T - SIP
t - SIP transient
U - up
V - VPN orphan
W - WAAS
X - inspected by service module
I am not sure why there are two "R" values - I am assuming sunrpc is listing UDP only and an outside acknowledged FIN would only be for a TCP session so there isn't a conflict in having the same flag value in use. Anyway, got tired of looking for this all the time when debugging problems so I am posting it here so I can find it for myself later.
- Ed
show connection all detail
or for those that like shorter commands:
sh conn all d
The output for the flags (a bit better organized then what is displayed in the help) is:
Flags:
A - awaiting inside ACK to SYN
a - awaiting outside ACK to SYN
B - initial SYN from outside
b - TCP state-bypass or nailed
C - CTIQBE media
D - DNS
d - dump
E - outside back connection
F - outside FIN
f - inside FIN
G - group
g - MGCP
H - H.323
h - H.225.0
I - inbound data
i - incomplete
J - GTP
j - GTP data
K - GTP t3-response
k - Skinny media
M - SMTP data
m - SIP media
n - GUP
O - outbound data
P - inside back connection
p - Phone-proxy TFTP connection
q - SQL*Net data
R - outside acknowledged FIN
R - UDP SUNRPC
r - inside acknowledged FIN
S - awaiting inside SYN
s - awaiting outside SYN
T - SIP
t - SIP transient
U - up
V - VPN orphan
W - WAAS
X - inspected by service module
I am not sure why there are two "R" values - I am assuming sunrpc is listing UDP only and an outside acknowledged FIN would only be for a TCP session so there isn't a conflict in having the same flag value in use. Anyway, got tired of looking for this all the time when debugging problems so I am posting it here so I can find it for myself later.
- Ed
Labels: Cisco ASA
Tuesday, May 11, 2010
Cisco ASA - Resource Allocation for Multiple Context
When setting up virtual contexts within an ASA it is a good practice to build out a resource allocation plan (classes) and then apply it to all the context that you have configured. By doing this you can safeguard the important FW context that need more resources ( a production FW) vs a less important FW context (think development or test.)
The reason I bring this up is after reviewing a new development FW context that I turned up for a client it was sending a huge volume of syslog data and the template that was built for dev limited the amount of syslog transactions it could send which protected the production FW. The output looked like:
fw-sys# sh resource usage all
Resource Current Peak Limit Denied Context
(...)
Syslogs [rate] 4 999 1000 449480 dev
Conns 282 2266 200000 0 dev
(...)
Syslogs [rate] 12 2558 unlimited 0 prod
Conns 20631 71951 unlimited 0 prod
Xlates 111601 145005 unlimited 0 prod
(...)
As you can see, there was a lot of denied syslog traffic. This was due to the initial configuration and leaving the dev FW context doing debug logging to the syslog server. The traffic dropped substantially after that was removed but it easily could have started consuming more than its fair share of resources on the ASA. In this particular case it is a 5580-20 so there wasn't a lot of concern but on a smaller platform (5510 or even a 5520) where memory and cpu are more constrained it could be a much bigger issue.
Your templates can be rather simple, a default is built automatically and applied to all context out of the box. This default gives maximum resources to all contexts, effectively sharing equally all the resources of the ASA. While this is ok to get thing built out and stood up I would not recommend keeping all the contexts in this configuration if you are going to use the ASA for multiple purposes like production, qa and development.
You can build it out like QoS templates. From a practical standpoint it makes sense to do a default, gold, silver and bronze configuration because I find it unlikely that most organizations would need more than that number to cover all their configuration options. Please speak up if you think I am wrong (but I doubt it.) Cisco doesn't seem to have a lot of documentation on recommendations for how to build this out. They do provide a class overview in their CLI configuration guide for 8.2, though it does not have enough details IMHO. You can apply the class to multiple contexts within the ASA so depending on your resource allocation you can easily oversubscribe resources because you set to many context as "gold" or forget to change the default and then have unlimited resources for all contexts that you build that are automatically given that default class.
Below are some sample templates to review - I set some arbitrary values based off of loads that were being observed. You should adjust the values you require for your environment since perhaps your development or qa context has a lot more hosts then what is reflected below. In other words, don't cut and copy, figure out what is right for your deployment which might require some experimenting to find out what you are actually consuming. This isn't a bad process to go through because it allows you to know where you are at, just remember to account for growth otherwise stuff will stop working or appear to have problems. Remembering that you have classes that might be limiting stuff won't be at the top of your list to check when you are troubleshooting - trust me on this one.
!
class default
limit-resource Xlates 100
limit-resource ASDM 2
limit-resource SSH 2
limit-resource Telnet 2
limit-resource Conns 5.0%
limit-resource Hosts 100
limit-resource rate Syslogs 1000
limit-resource rate Inspects 100
!
class gold
limit-resource All 0
limit-resource ASDM 5
limit-resource SSH 5
limit-resource Telnet 5
!
class silver
limit-resource Xlates 10000
limit-resource ASDM 2
limit-resource SSH 5
limit-resource Telnet 2
limit-resource Conns 20.0%
limit-resource Hosts 10000
limit-resource rate Syslogs 10000
limit-resource rate Inspects 5000
!
class bronze
limit-resource Xlates 1000
limit-resource ASDM 2
limit-resource SSH 2
limit-resource Telnet 2
limit-resource Conns 10.0%
limit-resource Hosts 1000
limit-resource rate Syslogs 1000
limit-resource rate Inspects 1000
!
To apply the class to the context it is as simple as:
!
context prod
member gold
!
context admin
member default
!
context lab
member bronze
!
Because the gold class has the "limit-resource All 0" command it effectively has no limit which means it can starve even the admin context in this example. This might not be ideal in all situations and I will often set limits on the gold class for production FW contexts because there will be more than one FW context that needs to run at the highest class level. In this particular case, just note it is getting everything it wants without question. Sometimes it is good to be king.
- Ed
The reason I bring this up is after reviewing a new development FW context that I turned up for a client it was sending a huge volume of syslog data and the template that was built for dev limited the amount of syslog transactions it could send which protected the production FW. The output looked like:
fw-sys# sh resource usage all
Resource Current Peak Limit Denied Context
(...)
Syslogs [rate] 4 999 1000 449480 dev
Conns 282 2266 200000 0 dev
(...)
Syslogs [rate] 12 2558 unlimited 0 prod
Conns 20631 71951 unlimited 0 prod
Xlates 111601 145005 unlimited 0 prod
(...)
As you can see, there was a lot of denied syslog traffic. This was due to the initial configuration and leaving the dev FW context doing debug logging to the syslog server. The traffic dropped substantially after that was removed but it easily could have started consuming more than its fair share of resources on the ASA. In this particular case it is a 5580-20 so there wasn't a lot of concern but on a smaller platform (5510 or even a 5520) where memory and cpu are more constrained it could be a much bigger issue.
Your templates can be rather simple, a default is built automatically and applied to all context out of the box. This default gives maximum resources to all contexts, effectively sharing equally all the resources of the ASA. While this is ok to get thing built out and stood up I would not recommend keeping all the contexts in this configuration if you are going to use the ASA for multiple purposes like production, qa and development.
You can build it out like QoS templates. From a practical standpoint it makes sense to do a default, gold, silver and bronze configuration because I find it unlikely that most organizations would need more than that number to cover all their configuration options. Please speak up if you think I am wrong (but I doubt it.) Cisco doesn't seem to have a lot of documentation on recommendations for how to build this out. They do provide a class overview in their CLI configuration guide for 8.2, though it does not have enough details IMHO. You can apply the class to multiple contexts within the ASA so depending on your resource allocation you can easily oversubscribe resources because you set to many context as "gold" or forget to change the default and then have unlimited resources for all contexts that you build that are automatically given that default class.
Below are some sample templates to review - I set some arbitrary values based off of loads that were being observed. You should adjust the values you require for your environment since perhaps your development or qa context has a lot more hosts then what is reflected below. In other words, don't cut and copy, figure out what is right for your deployment which might require some experimenting to find out what you are actually consuming. This isn't a bad process to go through because it allows you to know where you are at, just remember to account for growth otherwise stuff will stop working or appear to have problems. Remembering that you have classes that might be limiting stuff won't be at the top of your list to check when you are troubleshooting - trust me on this one.
!
class default
limit-resource Xlates 100
limit-resource ASDM 2
limit-resource SSH 2
limit-resource Telnet 2
limit-resource Conns 5.0%
limit-resource Hosts 100
limit-resource rate Syslogs 1000
limit-resource rate Inspects 100
!
class gold
limit-resource All 0
limit-resource ASDM 5
limit-resource SSH 5
limit-resource Telnet 5
!
class silver
limit-resource Xlates 10000
limit-resource ASDM 2
limit-resource SSH 5
limit-resource Telnet 2
limit-resource Conns 20.0%
limit-resource Hosts 10000
limit-resource rate Syslogs 10000
limit-resource rate Inspects 5000
!
class bronze
limit-resource Xlates 1000
limit-resource ASDM 2
limit-resource SSH 2
limit-resource Telnet 2
limit-resource Conns 10.0%
limit-resource Hosts 1000
limit-resource rate Syslogs 1000
limit-resource rate Inspects 1000
!
To apply the class to the context it is as simple as:
!
context prod
member gold
!
context admin
member default
!
context lab
member bronze
!
Because the gold class has the "limit-resource All 0" command it effectively has no limit which means it can starve even the admin context in this example. This might not be ideal in all situations and I will often set limits on the gold class for production FW contexts because there will be more than one FW context that needs to run at the highest class level. In this particular case, just note it is getting everything it wants without question. Sometimes it is good to be king.
- Ed
Tuesday, May 04, 2010
Root signing and DNSSEC
Time to start paying attention to DNSSEC and its potential impact on your ability to query against root servers through your firewalls. You can keep up to date at Root DNSSEC's website regarding the status and potential issues they are seeing.
Of immediate importance is the fact that your firewall may have a default configuration to discard UDP DNS traffic larger then 512 bytes in size. This was a common practice and it is now going to be an issue with the larger payloads that DNSSEC utilizes. Basically you just need to turn off this restriction and everything should work fine, for now. Platforms like the Cisco PIX, ASA and Microsoft ISA and TMG should all be checked.
Many people haven't noticed any issues with the first few root servers being signed and doing this behavior because their servers would not get a response back from the queried root server (blocked by the firewall for the payload size issue) and the server simply went to the next root server in the list which likely wasn't doing DNSSEC and therefore everything worked as expected. This is going to change starting this month going into June and IT Pros should be aware of the issue and review their firewall configurations to make sure they are not creating a problem for themselves.
Also, some of the TLD's are starting to sign also and I believe sometime in 2011 .com should be signed. If you are running DNS on Microsoft Windows 2008 R2 then you should read the DNSSEC Deployment Guide, it can walk you through requirements of what to do to get your infrastructure signed and working.
- Ed
Of immediate importance is the fact that your firewall may have a default configuration to discard UDP DNS traffic larger then 512 bytes in size. This was a common practice and it is now going to be an issue with the larger payloads that DNSSEC utilizes. Basically you just need to turn off this restriction and everything should work fine, for now. Platforms like the Cisco PIX, ASA and Microsoft ISA and TMG should all be checked.
Many people haven't noticed any issues with the first few root servers being signed and doing this behavior because their servers would not get a response back from the queried root server (blocked by the firewall for the payload size issue) and the server simply went to the next root server in the list which likely wasn't doing DNSSEC and therefore everything worked as expected. This is going to change starting this month going into June and IT Pros should be aware of the issue and review their firewall configurations to make sure they are not creating a problem for themselves.
Also, some of the TLD's are starting to sign also and I believe sometime in 2011 .com should be signed. If you are running DNS on Microsoft Windows 2008 R2 then you should read the DNSSEC Deployment Guide, it can walk you through requirements of what to do to get your infrastructure signed and working.
- Ed
Unless otherwise expressly stated, all original material of whatever nature created by Ed Horley and included in this weblog and any related pages, including the weblog's archives, is licensed under a Creative Commons License.
