[c-nsp] One Cat6k/Sup2T is software switching, its identical partner is not

[Matthew Huff]

Traffic destined to the switch rather than through the switch is generally software switched. Too much software switched traffic causing CPU issues could cause other traffic not to be installed in the ASIC causing a cascade failure.

Also, since you are running HSRP, what do you have your "mac-address-table aging-time" set to on both boxes? You might be running into asymmetrical traffic flows causing unicast flooding. Depending on your traffic levels, this could cause major issue. The recommended setting is around 14400, which is not the default. Google for either the command or 6500 unicast flooding for details.

During a maintenance window, I probably would suggest doing a complete power off, reseat the cards, and see if that makes any difference. I know of a few ASIC bugs that have been resolved over the years that are only fixed with either a linecard reset or a complete powerdown. The multicast ASIC bug we ran into on route topology changes was finally fixed in 12.2(33)SXI14 or later. Make sure you are running the latest linecard firmware release as well (at least 12.2(18r)S1)

----
Matthew Huff             | 1 Manhattanville Rd
Director of Operations   | Purchase, NY 10577
OTA Management LLC       | Phone: 914-460-4039
aim: matthewbhuff        | Fax:   914-694-5669

-----Original Message-----
From: Jeroen van Ingen [mailto:jeroen at zijndomein.nl] 
Sent: Monday, April 20, 2015 11:16 AM
To: Matthew Huff
Cc: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] One Cat6k/Sup2T is software switching, its identical partner is not

Hi,

> Haven't followed this thread too carefully, so I apologize if I
> duplicated anyone's suggestion:
>
> Have you checked for:
>
> 1) Are these systems setup for NHRP (hsrp, glbp, vrrp)? If so, is
> the box that is having the issue the active node?

Running HSRP, but doesn't make a difference which of the two routers is 
active. The second router is currently active for all interfaces and groups.

> 2) Are you running PIM? Is this box the DR?

Yes, running PIM-SM. This box is DR on some subinterfaces and not on a 
few others, but changing DR priority doesn't seem to make a difference.

> 3) Have you checked to see if there is traffic headed to the box,
> rather than through it that is causing the issue?

I don't see how it could be related to any traffic coming into the 
interfaces, because the interfaces are programmed to handle certain 
features in software even when they are not receiving any traffic.


Regards,

Jeroen van Ingen
ICT Service Centre
University of Twente, P.O.Box 217, 7500 AE Enschede, The Netherlands