[c-nsp] 7600 / SRC4 shutting down it's interfaces

[Andrei Radu]

Hello everybody,

 Yesterday we did a network wide upgrade on our 7600 routers from SRB4
to SRC4. After the upgrade we were hit pretty hard by one of the
weirdest 7600 behaviours ever.

 Between a pair of 7600 routers, let's say R1 and R2 (both with
SUP720-3BXL and 6704-10GE with DFC3BXL or DFC3CXL line-cards) we have
5 10GE links configured for IGP equal-cost load-balancing (the IGP
being OSPF in our case). After about 3-4 hours from the upgrade R1
started shutting down one of it's interfaces from the 5 10GE
interfaces group, the bringing it back up (it was the Te11/1
interface). When I say shutting down I mean that in the logs appeared
"changed state to administratively down" and in the
running-configuration the "shutdown" command appeared under that
interface and after a couple of seconds the "shutdown" command
disappeared. I am 100% sure that no person was issuing any commands on
R1 (we have even disabled SNMP just to make sure).

 We use a mixture of Cisco and non-Cisco optical modules so first we
tried exchanging the XENPAKs with new/original ones and replacing the
fiber optic patches. Then we tried moving the configuration to a new
port (Te10/1, on a different line-card) and the problem persisted.
After a couple hours the situation deteriorated with overruns on some
interfaces and packet loss for traffic transiting R1. After we have
rebooted module 10 (so the second line-card with the problem, not the
first one) the problem cleared.

 We have opened a TAC case and we are waiting for some feedback, in
the mean time has anyone else encountered this behaviour ? The only
thing I could find is CSCsv17989 (interface in SIP200 show "admin
down" when it is physical down) but according to the bug description
that is only a cosmetic problem.

 Best regards,
-- 
Andrei

"2+2=5, for extremely large values of 2 !"