[c-nsp] 7609 SP fails over on KPA failure

[Trevor Korthuis]

We have 2 7609's that are failing over to the redundant SP approximately every 5 days. These boxes are in network with PIM-SM multicast and IPV4 routed traffic running sup720's with SRA2. These failovers  started after 2 6516 GE cards were removed and replaced with 2 6704 10GE cards. When the 6516 cards were pulled, the boxes changed from Ingress Multicast Replication Mode to Egress mode. We have now forced the boxes back into Ingress Mode and have pulled the new 6704 cards.

We have a number of boxes with this configuration and these are the only 2 boxes with these symptoms. Only difference with these boxes is that they have 2 6704 cards running 4.7 firmware whereas the majority of 6704 cards in our network have 4.6 firmware.  In the lab, we have the same configuration and have multiple 6704's with the 4.7 firmware and in the same serial number range and cannot reproduce the issue. Cisco is currently trying to reproduce the issue in their labs. 

Cisco has determined that the ICC channel looks to be congested during one of these failovers, but cannot determine why.  Any chance that there is someone currently running 7609's in a multicast network having a similar sup720 failover issue?

Trevor