[c-nsp] FWSM vlans down after host SSO

[Bernhard Schmidt]

Hello everyone,

we are having a pretty serious problem with one of our boxes. 

6509
2* WS-SUP720-BASE + WS-F6K-PFC3B running 12.2(33)SXH1 modular
1* WS-X6704-10GE
2* WS-X6724-SFP
2* WS-X6408A-GBIC
1* WS-SVC-NAM-2
1* WS-SVC-FWM-1 running 3.1(4)

The FWSM has 10 contexts in routing mode and 4 contexts in transparent
mode. One of the routed contexts has IPv6 enabled.

Every few days the 6500 does a SSO failover without much explaination.
Console output of the formerly active Sup just starts with the System
Bootstrap again, there is nothing really useful in the remote syslog,
other than a lot UPDOWN messages the first message is 

May 24 13:37:04 CEST: %OIR-SP-3-PWRCYCLE: Card in module 5, is being
power-cycled (RF request)

(module 5 was the active Sup before, so it doesn't match CSCsh34467
which should be resolved in SXH1 anyway).

This is all very inconvenient, but SSO is fast enough for this network
and everything comes back as it should. Except for the FWSM, while the
failover happens every transport VLAN (between the hosting 6500 and the
FWSM) goes to up/down state and stays there. Interestingly the traffic
does not stop immediately, while the failover and the final
"%LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan3500, changed state
to down" was at 13:37, the system monitoring the IPv6 customer did not
see outages before 14:20. 

The only thing that seems to help in this mess is to reboot the FWSM.
Reload on the FWSM console does not work by the way (it seems to hang),
I had to use "hw-module module 9 reset" every time this happened so far.

Anyone having any ideas? I can get to the test kit in the lab on Monday
earliest unfortunately.

Bernhard