[c-nsp] Slave Supervisor for Sup 720 10G Crashing on 6500's

[Richard Chew]

Hi All,

  We have recently deployed 17, 6500's on campus, and about two months 
in we have already had 5 supervisors fail for no apparent reason.  When 
we call TAC they just RMA us a new Sup, but I suspect (cannot prove) 
that something else is causing this problem.  At first I thought it was 
SXH2, but we have recently seen the problem on SXH3, so any help would 
be appreciated.  Thanks.

BTW :

Nov  5 14:38:55.405 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:39:55.437 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:40:55.533 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:41:55.633 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB080EBD
Nov  5 14:42:39.425 PST: %CONST_DIAG-SPSTBY-3-HM_TEST_FAIL: Module 6 
TestSPRPInbandPing consecutive failure count:10
Nov  5 14:42:39.425 PST: %CONST_DIAG-SPSTBY-6-HM_TEST_INFO: CPU 
util(5sec): SP=8% RP=3% Traffic=0%
netint_thr_active[0], Tx_Rate[56], Rx_Rate[0], dev=1[IPv4, fail=10], 
2[IPv4, fail=10], 3[IPv4, fail=10], 4[IPv6, fail=10]
Nov  5 14:42:39.757 PST: %CONST_DIAG-SPSTBY-3-HM_TEST_FAIL: Module 6 
TestSPRPInbandPing consecutive failure count:10
Nov  5 14:42:39.757 PST: %CONST_DIAG-SPSTBY-6-HM_TEST_INFO: CPU 
util(5sec): SP=8% RP=3% Traffic=0%
netint_thr_active[0], Tx_Rate[56], Rx_Rate[0], dev=1[IPv4, fail=10], 
2[IPv4, fail=10], 3[IPv4, fail=10], 4[IPv6, fail=10]
Nov  5 14:42:55.765 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB080EBD
Nov  5 14:43:55.837 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:44:55.925 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:45:55.965 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:46:37.882 PST: %CONST_DIAG-SPSTBY-3-HM_TEST_FAIL: Module 6 
TestSPRPInbandPing consecutive failure count:20
Nov  5 14:46:37.882 PST: %CONST_DIAG-SPSTBY-6-HM_TEST_INFO: CPU 
util(5sec): SP=2% RP=0% Traffic=0%
netint_thr_active[0], Tx_Rate[49], Rx_Rate[0], dev=1[IPv4, fail=20], 
2[IPv4, fail=20], 3[IPv4, fail=20], 4[IPv6, fail=20]
Nov  5 14:46:38.218 PST: %CONST_DIAG-SPSTBY-3-HM_TEST_FAIL: Module 6 
TestSPRPInbandPing consecutive failure count:20
Nov  5 14:46:38.218 PST: %CONST_DIAG-SPSTBY-6-HM_TEST_INFO: CPU 
util(5sec): SP=14% RP=0% Traffic=0%
netint_thr_active[0], Tx_Rate[49], Rx_Rate[0], dev=1[IPv4, fail=20], 
2[IPv4, fail=20], 3[IPv4, fail=20], 4[IPv6, fai=20]
Nov  5 14:46:56.077 PST: %EARL_L2_ASIC-SPSTBY-4-DBUS_HDR_ERR: EARL L2 
ASIC #0: Dbus Hdr. Error occurred. Ctrl1 0xB08D0EBD
Nov  5 14:47:03.241 PST: %PFREDUN-SP-6-ACTIVE: Standby supervisor 
removed or reloaded, changing to Simplex mode
Nov  5 14:47:03.261 PST: %OIR-SP-3-PWRCYCLE: Card in module 6, is being 
power-cycled (RF request)
Nov  5 14:47:13.470 PST: %LINK-3-UPDOWN: Interface GigabitEthernet6/1, 
changed state to down
Nov  5 14:47:13.470 PST: %OSPF-5-ADJCHG: Process 5739, Nbr 128.114.0.4 
on GigabitEthernet6/1 from FULL to DOWN, Neighbor Down: Interface down 
or detached
Nov  5 14:47:13.494 PST: %PIM-5-NBRCHG: neighbor 128.114.1.157 DOWN on 
interface GigabitEthernet6/1 non DR
Nov  5 14:47:13.494 PST: %LINEPROTO-5-UPDOWN: Line protocol on Interface 
GigabitEthernet6/1, changed state to down
Nov  5 14:47:13.606 PST: %SNMP-5-MODULETRAP: Module 6 [Down] Trap
Nov  5 14:47:13.461 PST: %LINK-SP-3-UPDOWN: Interface 
GigabitEthernet6/1, changed state to down
Nov  5 14:47:13.593 PST: %OIR-SP-3-PWRCYCLE: Card in module 6, is being 
power-cycled (Slot disabled)
Nov  5 14:47:13.597 PST: %LINEPROTO-SP-5-UPDOWN: Line protocol on 
Interface GigabitEthernet6/1, changed state to down