[c-nsp] 7500 for DSL aggregation - RSP memory error?

Walter Keen walter.keen at RainierConnect.net
Tue Aug 4 05:51:07 EDT 2009


I've got a 7507 with dual RSP8's attempting to use rsp-jsv-mz.124-8.bin 
configured for rpr-plus, but keep getting this around every 10 minutes 
or so.  It results in a loss of connectivity for end-users of course, 
until the system recovers.

My initial guess is something is wrong with the standby processor (slot 
3) or perhaps the memory in it.  I've had the tech pull it out to see if 
the system stabalizes and will bring it back to the lab if it does.

Anyone else ran into this in the past?



sea-agg-1#              
2w5d: %TBRIDGE-4-NOVCFLOOD: No VC's configured for bridging on ATM4/1/0.669
2w5d: %RSP-3-ERROR: MD error 0080000000010000 -Traceback= 0x40588B14 
0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   Cybus1 parity error (bytes 0:7) 04 -Traceback= 
0x40588CDC 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   bus command write 8bytes (0x7) -Traceback= 
0x40588930 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   physical address (bits 20:12) 0E2000 -Traceback= 
0x40588A50 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 6E0000 -Traceback= 
0x40588A74 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-MVIP_CYBUSERROR_INTERRUPT: A 
Cybus Error occured.                                             
2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYASIC Error Interrupt register 
0xB                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Parity Error internal to 
CYA                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Missing ACK on CyBus 
access                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot5     NACK present on CyBus 
access                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYASIC Other Interrupt register 
0x100                                                               
2w5d: %VIP4-80 RM7000-1-MSG: slot5     QE HIGH Priority 
Interrupt                                                                       

2w5d: %VIP4-80 RM7000-1-MSG: slot5     QE RX HIGH Priority 
Interrupt                                                                    

2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYBUS Error Cmd/Addr 0x8001A80, CYBUS 
Error Data 0x0                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot5 MPUIntfc/PacketBus Error register 
0x0                                                               
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Interrupt Status register 
0x4                                                           
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Address/Command Strobe 
Timeout                                                                  
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Address High 
0x1C01                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Address Low 
0xC                                                                          

2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-SVIP_RELOAD: SVIP Reload is 
called.                                                           
2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-SYSTEM_EXCEPTION: VIP System 
Exception occurred sig=22, code=0x0, context=0x6199A8A8          
                                                                                                                                        

2w5d: %RSP-3-ERROR: End of MEMD error interrupt processing -Traceback= 
0x40589298 0x405892F0 0x4058A978 0x404CFA54                     
2w5d: %DBUS-3-CXBUSERR: Slot 5, CBus 
Error                                                                                              

2w5d: %DBUS-3-DBUSINTERRSWSET: Slot 5, Internal Error due to VIP 
crash                                                                 
2w5d: %OSPF-5-ADJCHG: Process 10, Nbr 74.50.207.83 on FastEthernet5/1/0 
from FULL to DOWN, Neighbor Down: Interface down or detached   
2w5d: %RSP-3-ERROR: CyBus1 error 10 -Traceback= 0x40588DA8 0x405891F0 
0x405892F0 0x4058A978 0x404CFA54                                 
2w5d: %RSP-3-ERROR:     command/address mismatch -Traceback= 0x40588E64 
0x405891F0 0x405892F0 0x4058A978 0x404CFA54                    
2w5d: %RSP-3-ERROR:   bus command read 8bytes (0x1) -Traceback= 
0x40588930 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54      
2w5d: %RSP-3-ERROR:   address offset (bits 3:1) 8 -Traceback= 0x40588A18 
0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54        
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 000000 -Traceback= 
0x40588A74 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54
2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-MVIP_CYBUSERROR_INTERRUPT: A 
Cybus Error occured.                                             
2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYASIC Error Interrupt register 
0xF                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Parity Error internal to 
CYA                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Parity Error in data from 
CyBus                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Missing ACK on CyBus 
access                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot4     NACK present on CyBus 
access                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYASIC Other Interrupt register 
0x80                                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot4     QE HIGH Priority 
Interrupt                                                                       

2w5d: %VIP4-80 RM7000-1-MSG: slot4     Unknown CYA oisr bit 
0x00000080                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot4     QE TX HIGH Priority 
Interrupt                                                                    

2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYBUS Error Cmd/Addr 0x1000198, CYBUS 
Error Data 0x0                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot4 MPUIntfc/PacketBus Error register 
0x0                                                               
2w5d: %VIP4-80 RM7000-1-MSG: slot4 IOBUS Error Interrupt Status register 
0x0                                                           
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                            
2w5d: %HA-5-MODE: Operating mode is hsa, configured mode is 
rpr-plus.                                                                  
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                            
2w5d: %RSP-3-ERROR: CyBus1 error 10 -Traceback= 0x40588DA8 0x405891F0 
0x405892F0 0x4058A978 0x404CFA54                                 
2w5d: %RSP-3-ERROR:     command/address mismatch -Traceback= 0x40588E64 
0x405891F0 0x405892F0 0x4058A978 0x404CFA54                    
2w5d: %RSP-3-ERROR:   bus command write 1byte (0xB) -Traceback= 
0x40588930 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54      
2w5d: %RSP-3-ERROR:   address offset (bits 3:1) 12 -Traceback= 
0x40588A18 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54       
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 000000 -Traceback= 
0x40588A74 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54
2w5d: %DBUS-3-SW_NOTRDY: DBUS software not ready for slot 5 after 
HARD_RESET, elapsed 15048, status 0x0 -Traceback= 0x40388848 0x405BA000 
0x405BD2C0 0x405C3720
2w5d: %DBUS-3-WCSLDERR: Slot 5, error loading WCS, status 0x14 cmd/data 
0x2E pos 
5749518                                                                       

2w5d: %UCODE-3-LDFAIL: Unable to download ucode from system image in 
slot 5, trying rom 
ucode                                                                 
2w5d: %RSP-3-NOSTART: No microcode for VIP4-80 RM7000 card, slot 
5                                                                                             

2w5d: %DBUS-3-WCSLDERR: Slot 5, error loading WCS, status 0x14 cmd/data 
0x2E pos 
5749518                                                                       

2w5d: %UCODE-3-LDFAIL: Unable to download ucode from system image in 
slot 5, trying rom 
ucode                                                                 
2w5d: %OIR-6-INSCARD: Card inserted in slot 5, interfaces 
administratively shut 
down                                                                           

2w5d: %RSP-3-NOSTART: No microcode for VIP4-80 RM7000 card, slot 
5                                                                                             

2w5d: %DBUS-3-SW_NOTRDY: DBUS software not ready for slot 3 after 
dbus_slot_enable(), elapsed 15040, status 0x29 -Traceback= 0x40388848 
0x405BA000 0x405BB054 0x405D2DF4 
0x405C393C                                                                                                                                                             

2w5d: %DBUS-3-WCSPARERR: Slot 3, WCS Controller Parity 
Error                                                                                                            

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-SVIP_RELOAD: SVIP Reload is 
called.                                                                                            

2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-SYSTEM_EXCEPTION: VIP System 
Exception occurred sig=22, code=0x0, 
context=0x6199A8A8                                          
                                                                                                                                                                        

2w5d: %DBUS-3-CXBUSERR: Slot 4, CBus 
Error                                                                                                                              

2w5d: %DBUS-3-DBUSINTERRSWSET: Slot 4, Internal Error due to VIP 
crash                                                                                                  

2w5d: %LINK-3-UPDOWN: Interface BVI3, changed state to 
down                                                                                                             

2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI3, changed 
state to 
down                                                                                       

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %OIR-6-INSCARD: Card inserted in slot 5, interfaces 
administratively shut 
down                                                                                    

2w5d: %CBUS-3-CCBPTIMEOUT: CCB handover timed out, CCB 0xF800FF50, slot 
4 -Traceback= 0x40388848 0x405D6054 0x405D2830 0x405D2EE0 
0x405C393C                           
2w5d: %LINK-3-UPDOWN: Interface FastEthernet5/1/0, changed state to 
down                                                                                                

2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface ATM4/1/0, changed 
state to 
down                                                                                   

2w5d: %LINK-3-UPDOWN: Interface FastEthernet5/1/0, changed state to 
up                                                                                                  

2w5d: %OSPF-5-ADJCHG: Process 10, Nbr 74.50.207.83 on FastEthernet5/1/0 
from LOADING to FULL, Loading 
Done                                                             
2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

sea-agg-1#                                                                                                                                                              

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not found

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not found
2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not found



--------------------



sea-agg-1#                                                     
2w5d: %RSP-3-ERROR: MD error 0080000000010000 -Traceback= 0x40588B14 
0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   Cybus1 parity error (bytes 0:7) 04 -Traceback= 
0x40588CDC 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   bus command write 8bytes (0x7) -Traceback= 
0x40588930 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   physical address (bits 20:12) 0F8000 -Traceback= 
0x40588A50 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 6E0000 -Traceback= 
0x40588A74 0x40588CF8 0x405891CC 0x405892F0 0x4058A978 0x404CFA54
2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-MVIP_CYBUSERROR_INTERRUPT: A 
Cybus Error occured.                                             
2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYASIC Error Interrupt register 
0xB                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Parity Error internal to 
CYA                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Missing ACK on CyBus 
access                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot5     NACK present on CyBus 
access                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYASIC Other Interrupt register 
0x100                                                               
2w5d: %VIP4-80 RM7000-1-MSG: slot5     QE HIGH Priority 
Interrupt                                                                       

2w5d: %VIP4-80 RM7000-1-MSG: slot5     QE RX HIGH Priority 
Interrupt                                                                    

2w5d: %VIP4-80 RM7000-1-MSG: slot5 CYBUS Error Cmd/Addr 0x76F8548, CYBUS 
Error Data 0x0                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot5 MPUIntfc/PacketBus Error register 
0x0                                                               
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Interrupt Status register 
0x4                                                           
2w5d: %VIP4-80 RM7000-1-MSG: slot5     Address/Command Strobe 
Timeout                                                                  
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Address High 
0x1C01                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot5 IOBUS Error Address Low 
0xC                                                                          

2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-SVIP_RELOAD: SVIP Reload is 
called.                                                           
2w5d: %VIP4-80 RM7000-3-MSG: slot5 VIP-3-SYSTEM_EXCEPTION: VIP System 
Exception occurred sig=22, code=0x0, context=0x6199B028          
                                                                                                                                        

2w5d: %RSP-3-ERROR: End of MEMD error interrupt processing -Traceback= 
0x40589298 0x405892F0 0x4058A978 0x404CFA54                     
2w5d: %DBUS-3-CXBUSERR: Slot 5, CBus 
Error                                                                                              

2w5d: %DBUS-3-DBUSINTERRSWSET: Slot 5, Internal Error due to VIP 
crash                                                                 
2w5d: %OSPF-5-ADJCHG: Process 10, Nbr 74.50.207.83 on FastEthernet5/1/0 
from FULL to DOWN, Neighbor Down: Interface down or detached   
2w5d: %RSP-3-ERROR: CyBus1 error 10 -Traceback= 0x40588DA8 0x405891F0 
0x405892F0 0x4058A978 0x404CFA54                                 
2w5d: %RSP-3-ERROR:     command/address mismatch -Traceback= 0x40588E64 
0x405891F0 0x405892F0 0x4058A978 0x404CFA54                    
2w5d: %RSP-3-ERROR:   bus command read 8bytes (0x1) -Traceback= 
0x40588930 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54      
2w5d: %RSP-3-ERROR:   address offset (bits 3:1) 0 -Traceback= 0x40588A18 
0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54        
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 000000 -Traceback= 
0x40588A74 0x40588F68 0x405891F0 0x405892F0 0x4058A978 0x404CFA54
2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-MVIP_CYBUSERROR_INTERRUPT: A 
Cybus Error occured.                                             
2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYASIC Error Interrupt register 
0x4000F                                                             
2w5d: %VIP4-80 RM7000-1-MSG: slot4     CBus read during CBus 
stall                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Parity Error internal to 
CYA                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Parity Error in data from 
CyBus                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Missing ACK on CyBus 
access                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot4     NACK present on CyBus 
access                                                                    
2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYASIC Other Interrupt register 
0x80                                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot4     QE HIGH Priority 
Interrupt                                                                       

2w5d: %VIP4-80 RM7000-1-MSG: slot4     Unknown CYA oisr bit 
0x00000080                                                                 
2w5d: %VIP4-80 RM7000-1-MSG: slot4     QE TX HIGH Priority 
Interrupt                                                                    

2w5d: %VIP4-80 RM7000-1-MSG: slot4 CYBUS Error Cmd/Addr 0x10001A0, CYBUS 
Error Data 0x0                                                
2w5d: %VIP4-80 RM7000-1-MSG: slot4 MPUIntfc/PacketBus Error register 
0x110000A1                                                        
2w5d: %VIP4-80 RM7000-1-MSG: slot4 IOBUS Error Interrupt Status register 
0x4                                                           
2w5d: %VIP4-80 RM7000-1-MSG: slot4     Address/Command Strobe 
Timeout                                                                  
2w5d: %VIP4-80 RM7000-1-MSG: slot4 IOBUS Error Address High 
0x1C01                                                                     
2w5d: %VIP4-80 RM7000-1-MSG: slot4 IOBUS Error Address Low 
0xC                                                                          

2w5d: %RSP-2-QAERROR: reused or zero link error, write at addr 0138 
(QA)                                                               
  log 22013800, data A0F00000 
00000000                                                                                                  

2w5d: %QA-3-DIAG: Failed to enqueue buffer header 
0xA0F0                                                                                

2w5d: %QA-3-DIAG: Approximate stack backtrace prior to 
interrupt:                                                                       

2w5d: 
%QA-3-DIAG:                                                                                                                       

-Traceback= 0x404CD814 0x406B3684 0x406BA5F0 0x406C3770 0x406B6518 
0x406AE188 0x406A5EE0 0x406B1FC4 0x4051C730 0x40642F1C 0x40643104 
0x406432F8
2w5d: %QA-3-DIAG: Buffer 0xA0F0 is element 1 on queue 
0x20                                                                                      

2w5d: %QA-3-DIAG:     Queue 0x20 (E8000100) has 1 
elements                                                                                      

2w5d: %QA-3-DIAG: Buffer 0xA0F0 is element 1154 on queue 
0x27                                                                                   

2w5d: %QA-3-DIAG:     Queue 0x27 (E8000138) has 1154 
elements                                                                                   

2w5d: %QA-3-DIAG: No NULL terminator for queue 
0x28                                                                                             

2w5d: %QA-3-DIAG:     Queue 0x28 (E8000140) has 90 
elements                                                                                     

2w5d: %QA-3-DIAG: No NULL terminator for queue 
0x2C                                                                                             

2w5d: %QA-3-DIAG:     Queue 0x2C (E8000160) has 5 
elements                                                                                      

2w5d: %QA-3-DIAG: At least one QA queue is 
broken                                                                                               

2w5d: %QA-3-DIAG: Buffer header at 0xE000A0F0: 12 1A0 3FEFEFEF 
8A800                                                                            

2w5d: %QA-3-DIAG: Buffer 
contents:                                                                                                              

2w5d: %RSP-3-IDBOFFSET: hwidb = 0x444B8760, Name = AT4/1/0, 
hwidb->rx_offset = 96 Possible datagram start = 0xF808A980 -Traceback= 
0x40388848 0x4059E3A0 0x405A9930 0x405AA064 0x40589324 0x4058A978 
0x404CFA54                                                                                                                                 

2w5d: %SYS-3-DMPMEM: F808A800: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %SYS-3-DMPMEM: F808A818: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %SYS-3-DMPMEM: F808A830: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %SYS-3-DMPMEM: F808A848: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %SYS-3-DMPMEM: F808A860: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %SYS-3-DMPMEM: F808A878: 3FEFEFEF 3FEFEFEF 3FEFEFEF 3FEFEFEF 
3FEFEFEF 
3FEFEFEF                                                                                    

2w5d: %QA-3-DIAG: Global 
queues:                                                                                                                                        

2w5d: %QA-3-DIAG:     3313 buffer 
headers                                                                                                                               

2w5d: %QA-3-DIAG:     RawQ 0xE8000100 (1), ReturnQ 0xE8000108 (0), 
EventQ 0xE8000110 
(0)                                                                                

2w5d: %QA-3-DIAG: IpcackQ 0xE8000118 
(0)                                                                                                                                

2w5d: %QA-3-DIAG: VIP_CrashinfoQ 0xE8000128 
(0)                                                                                                                         

2w5d: %QA-3-DIAG: IpcSlaveackQ 0xE8000120 
(0)                                                                                                                           

2w5d: %QA-3-DIAG:     BufhdrQ 0xE8000158 
(0)                                                                                                                            

2w5d: %QA-3-DIAG:     LovltrQ 0xE8000170 
(0)                                                                                                                            

2w5d: %QA-3-DIAG:     IpcbufQ 0xE8000180 
(0)                                                                                                                            

2w5d: %QA-3-DIAG:     IpcbufQ_classic 0xE8000178 
(0)                                                                                                                    

2w5d: %QA-3-DIAG:     Pool0: 10 buffers, 256 bytes, queue 0xE8000160 
(5)                                                                                                

2w5d: %QA-3-DIAG:     Pool1: 932 buffers, 1536 bytes, queue 0xE8000168 
(0)                                                                                              

2w5d: %QA-3-DIAG:     Pool2: 1357 buffers, 4544 bytes, queue 0xE8000188 
(0)                                                                                             

2w5d: %QA-3-DIAG:     Pool3: 4 buffers, 4576 bytes, queue 0xE8000190 
(0)                                                                                                

2w5d: %QA-3-DIAG: 
Slot3:                                                                                                                                                

2w5d: %QA-3-DIAG: 
Slot4:                                                                                                                                                

2w5d: %QA-3-DIAG:     ATM4/0/0: gfreeq 0xE8000188 (0), lfreeq 0xE8000198 
(0)  (4544 
bytes)                                                                              

2w5d: %QA-3-DIAG:         txq 0xE8001A00 
(0)                                                                                                                            

2w5d: %QA-3-DIAG:     ATM4/1/0: gfreeq 0xE8000188 (0), lfreeq 0xE80001A0 
(0)  (4544 
bytes)                                                                              

2w5d: %QA-3-DIAG:         txq 0xE8001A08 
(0)                                                                                                                            

2w5d: %QA-3-DIAG: 
Slot5:                                                                                                                                                

2w5d: %QA-3-DIAG:     FastEthernet5/1/0: gfreeq 0xE8000168 (0), lfreeq 
0xE80001A8 (0)  (1536 
bytes)                                                                    
2w5d: %QA-3-DIAG:         txq 0xE8001A80 
(0)                                                                                                                            

2w5d: %QA-3-DIAG: 
Slot6:                                                                                                                                                

2w5d: %QA-3-DIAG:     ATM6/0/0: gfreeq 0xE8000188 (0), lfreeq 0xE80001B0 
(0)  (4544 
bytes)                                                                              

2w5d: %QA-3-DIAG:         txq 0xE8001B00 
(0)                                                                                                                            

2w5d: %QA-3-DIAG:     FastEthernet6/1/0: gfreeq 0xE8000168 (0), lfreeq 
0xE80001B8 (0)  (1536 
bytes)                                                                    
2w5d: %QA-3-DIAG:         txq 0xE8001B08 
(0)                                                                                                                            

2w5d: %QA-3-DIAG: Trying to recover from QA 
ERROR.                                                                                                                      

2w5d: %QA-3-DIAG: Removing buffer header 0xA0F0 from all 
queues                                                                                                         

2w5d: %QA-3-DIAG: Buffer 0xA0F0 is on queue 
0x20                                                                                                                        

2w5d: %QA-3-DIAG: At least one QA queue is 
broken                                                                                                                       

2w5d: %QA-3-DIAG: Recovered from QA 
ERROR.                                                                                                                              

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %HA-5-MODE: Operating mode is hsa, configured mode is 
rpr-plus.                                                                                                   

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-ERROR: CyBus1 error 10 -Traceback= 0x40588DA8 0x405891F0 
0x405892F0 0x4058A978 
0x404CFA54                                                                 
2w5d: %RSP-3-ERROR:     command/address mismatch -Traceback= 0x40588E64 
0x405891F0 0x405892F0 0x4058A978 
0x404CFA54                                                    
2w5d: %RSP-3-ERROR:   bus command write 1byte (0xB) -Traceback= 
0x40588930 0x40588F68 0x405891F0 0x405892F0 0x4058A978 
0x404CFA54                                      
2w5d: %RSP-3-ERROR:   address offset (bits 3:1) 12 -Traceback= 
0x40588A18 0x40588F68 0x405891F0 0x405892F0 0x4058A978 
0x404CFA54                                       
2w5d: %RSP-3-ERROR:   virtual address (bits 23:17) 000000 -Traceback= 
0x40588A74 0x40588F68 0x405891F0 0x405892F0 0x4058A978 
0x404CFA54                                
2w5d: %DBUS-3-SW_NOTRDY: DBUS software not ready for slot 5 after 
HARD_RESET, elapsed 15480, status 0x0 -Traceback= 0x40388848 0x405BA000 
0x405BD2C0 0x405C3720        
2w5d: %DBUS-3-WCSLDERR: Slot 5, error loading WCS, status 0x14 cmd/data 
0x2E pos 
5749518                                                                                

2w5d: %UCODE-3-LDFAIL: Unable to download ucode from system image in 
slot 5, trying rom 
ucode                                                                           

2w5d: %RSP-3-NOSTART: No microcode for VIP4-80 RM7000 card, slot 
5                                                                                                      

2w5d: %DBUS-3-WCSLDERR: Slot 5, error loading WCS, status 0x14 cmd/data 
0x2E pos 
5749518                                                                                

2w5d: %UCODE-3-LDFAIL: Unable to download ucode from system image in 
slot 5, trying rom 
ucode                                                                           

2w5d: %OIR-6-INSCARD: Card inserted in slot 5, interfaces 
administratively shut 
down                                                                                    

2w5d: %RSP-3-NOSTART: No microcode for VIP4-80 RM7000 card, slot 
5                                                                                                      

2w5d: %DBUS-3-SW_NOTRDY: DBUS software not ready for slot 3 after 
dbus_slot_enable(), elapsed 15040, status 0x29 -Traceback= 0x40388848 
0x405BA000 0x405BB054 0x405D2DF4 
0x405C393C                                                                                                                                                             

2w5d: %DBUS-3-WCSPARERR: Slot 3, WCS Controller Parity 
Error                                                                                                            

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %TIGER-3-BADADDR_MBE: Invalid MBE dram address: 0xFFFFFFFF latched 
by 
Tiger                                                                                       

2w5d: %RSP-3-ERROR: dbus read at 
3E8410C0                                                                                                                               

-Traceback= 0x405BBE30 0x405BCDA0 0x405C3720  -Traceback= 0x405887C4 
0x4058AFA8 0x404E6EA8 
0x404CDC04                                                                   

2w5d: %DBUS-3-SLOTCOMP: Slot 3, dbus error, slot (0xF) and complement 
(0x0) do not 
match                                                                                

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-SVIP_RELOAD: SVIP Reload is 
called.                                                                                            

2w5d: %VIP4-80 RM7000-3-MSG: slot4 VIP-3-SYSTEM_EXCEPTION: VIP System 
Exception occurred sig=22, code=0x0, 
context=0x6199A8A8                                          
                                                                                                                                                                        

2w5d: %DBUS-3-CXBUSERR: Slot 4, CBus 
Error                                                                                                                              

2w5d: %DBUS-3-DBUSINTERRSWSET: Slot 4, Internal Error due to VIP 
crash                                                                                                  

2w5d: %LINK-3-UPDOWN: Interface BVI3, changed state to down
2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI3, changed 
state to down
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to Non-participant
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to Slave
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to Non-participant
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to Slave
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to Non-participant
2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to Slave
2w5d: %OIR-6-INSCARD: Card inserted in slot 5, interfaces 
administratively shut down
2w5d: %CBUS-3-CCBPTIMEOUT: CCB handover timed out, CCB 0xF800FF50, slot 
4 -Traceback= 0x40388848 0x405D6054 0x405D2830 0x405D2EE0 0x405C393C
2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface ATM4/1/0, changed 
state to down
2w5d: %OSPF-5-ADJCHG: Process 10, Nbr 74.50.207.83 on FastEthernet5/1/0 
from LOADING to FULL, Loading Done
2w5d: %RADIUS-4-RADIUS_DEAD: RADIUS server 69.10.208.10:1645,1646 is not 
responding.
2w5d: %RADIUS-4-RADIUS_ALIVE: RADIUS server 69.10.208.10:1645,1646 has 
returned.
2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not found
sea-agg-1#
sea-agg-1#
sea-agg-1#
sea-agg-1#                                                                                                                                                              

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

sea-agg-1#                                                                                                                                                              

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

2w5d: %IPC_RSP_CBUS-3-NOHWQ: Hardware queue for card at slot 0 not 
found                                                                                                

2w5d: %DBUS-3-SW_NOTRDY: DBUS software not ready for slot 3 after 
dbus_slot_enable(), elapsed 15040, status 0x29 -Traceback= 0x40388848 
0x405BA000 0x405BB054 0x405D2DF4 
0x405C393C                                                                                                                                                             

2w5d: %DBUS-3-WCSPARERR: Slot 3, WCS Controller Parity 
Error                                                                                                            

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %LINK-3-UPDOWN: Interface ATM4/1/0, changed state to 
up                                                                                                           

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface ATM4/1/0, changed 
state to 
up                                                                                     

2w5d: %LINK-3-UPDOWN: Interface BVI3, changed state to 
up                                                                                                               

2w5d: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI3, changed 
state to 
up                                                                                         

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Slave to 
Non-participant                                                                                             

2w5d: %RSP-3-SLAVECHANGE: Slave changed state from Non-participant to 
Slave                                                                                             

2w5d: %TBRIDGE-4-NOVCFLOOD: No VC's configured for bridging on 
ATM4/1/0.669                                                                                             

2w5d: %HA-5-NOTICE: Standby (slave) configured to run HA image 
"disk0:rsp-jsv-mz.124-8.bin"                                                                             

2w5d: %HA-5-NOTICE: Loading standby (slave) image: 
"disk0:rsp-jsv-mz.124-8.bin"                                                                                         

2w5d: %TBRIDGE-4-NOVCFLOOD: No VC's configured for bridging on 
ATM4/1/0.669                                                                                             

2w5d: %RADIUS-4-RADIUS_DEAD: RADIUS server 69.10.201.13:1645,1646 is not 
responding.                                                                                    

2w5d: %RADIUS-4-RADIUS_ALIVE: RADIUS server 69.10.201.13:1645,1646 has 
returned.                                                                                        

2w5d: %RADIUS-4-RADIUS_DEAD: RADIUS server 69.10.201.13:1645,1646 is not 
responding.                                                                                    

2w5d: %RADIUS-4-RADIUS_ALIVE: RADIUS server 69.10.201.13:1645,1646 has 
returned.                                                                                        

2w5d: %RADIUS-4-RADIUS_DEAD: RADIUS server 69.10.201.13:1645,1646 is not 
responding.                                                                                    

2w5d: %RADIUS-4-RADIUS_ALIVE: RADIUS server 69.10.201.13:1645,1646 has 
returned.                                                                                        

2w5d: %HA-5-MODE: Operating mode is hsa, configured mode is 
rpr-plus.                                                                                                   

2w5d: %HA-5-SYNC_NOTICE: Bulk sync 
started.                                                                                                                             

2w5d: %HA-5-SYNC_NOTICE: Bulk sync 
completed.                                                                                                                           

2w5d: %HA-5-SYNC_NOTICE: Config sync 
started.                                                                                                                           

sea-agg-1#                                




More information about the cisco-nsp mailing list