[c-nsp] 7500 Crashes
Jason Berenson
jason at pins.net
Sat Oct 15 12:57:45 EDT 2011
Greetings,
I've got a 7500 running with the following:
NAME: "7507z chassis, ID:76032590", DESCR: "7507z chassis"
PID: 4 , VID: Hardware Version : 1.00 Board Revision :
B0, SN: 76032590
NAME: "Line Card 0", DESCR: "Versatile Interface Processor (VIP4-80)"
PID: VIP4-80 , VID: Hardware Version : 2.04 Board Revision :
A0, SN: 21008718
NAME: "Card Slot 0, Bay 0", DESCR: "Dual Port FastEthernet (RJ45)"
PID: PA-2FE-TX , VID: Hardware Version : 1.0 Board Revision :
B0, SN: MIC050800X2
NAME: "Card Slot 0, Bay 1", DESCR: "Serial T3+ Port Adapter"
PID: PA-T3+ , VID: Hardware Version : 1.0 Board Revision :
B0, SN: 17830995
NAME: "Line Card 1", DESCR: "Versatile Interface Processor"
PID: VIP2-50 , VID: Hardware Version : 2.03 Board Revision :
A0, SN: 17943843
NAME: "Card Slot 1, Bay 0", DESCR: "FastEthernet Port Adapter "
PID: PA-FE-TX-ISL , VID: Hardware Version : 1.0 Board Revision :
A0, SN: 06340347
NAME: "Card Slot 1, Bay 1", DESCR: "Dual Port FastEthernet (RJ45)"
PID: PA-2FE-TX , VID: Hardware Version : 1.0 Board Revision :
C0, SN: JAE070103WM
NAME: "RSP at Slot 3", DESCR: "R5000"
PID: RSP4 , VID: Hardware Version : 1.1 Board Revision :
D0, SN: 11519637
NAME: "Line Card 4", DESCR: "Versatile Interface Processor (VIP4-80)"
PID: VIP4-80 , VID: Hardware Version : 2.04 Board Revision :
B0, SN: 28099188
NAME: "Card Slot 4, Bay 0", DESCR: "Channelized DS3 - single wide, one port"
PID: PA-MC-T3 , VID: Hardware Version : 1.0 Board Revision :
A0, SN: 17832515
NAME: "Card Slot 4, Bay 1", DESCR: "Channelized DS3 - single wide, one port"
PID: PA-MC-T3 , VID: Hardware Version : 1.0 Board Revision :
A0, SN: 17827319
NAME: "Line Card 5", DESCR: "Versatile Interface Processor (VIP4-80)"
PID: VIP4-80 , VID: Hardware Version : 2.03 Board Revision :
A0, SN: 25567743
NAME: "Card Slot 5, Bay 0", DESCR: "Channelized DS3 - single wide, one port"
PID: PA-MC-T3 , VID: Hardware Version : 1.0 Board Revision :
A0, SN: 24501038
NAME: "Card Slot 5, Bay 1", DESCR: "Channelized DS3 - single wide, one port"
PID: PA-MC-T3 , VID: Hardware Version : 1.0 Board Revision :
A0, SN: 15811547
NAME: "Line Card 6", DESCR: "Versatile Interface Processor (VIP6-80)"
PID: VIP6-80 , VID: Hardware Version : 2.01 Board Revision :
A0, SN: 29452845
NAME: "Card Slot 6, Bay 0", DESCR: "GigabitEthernet Port Adapter"
PID: PA-GE , VID: Hardware Version : 1.0 Board Revision :
A1, SN: 18636663
The box keeps switching between the RSPs for some reason. Here's the
relevant logs:
*Oct 15 01:11:07.175: %HA-5-NOTICE: Suspending initialization pending
bulk/config sync...
*Oct 15 01:11:07.251: %HA-5-NOTICE: Command line console will
unavailable until init completes.
*Oct 15 01:11:07.351: %HA-5-NOTICE: Defaulting to hsa mode until sync
completed and system restarted.
*Oct 15 01:11:07.363: %HA-5-SYNC_NOTICE: Bulk sync started.
*Oct 15 01:11:07.675: %HA-5-SYNC_NOTICE: Bulk sync completed.
*Oct 15 01:11:11.899: %HA-5-SYNC_NOTICE: Config sync started.
*Oct 15 01:11:11.975: %HA-5-NOTICE: Resuming initialization...
*Oct 14 20:11:16.987: %SYS-6-CLOCKUPDATE: System clock has been updated
from 01:11:16 UTC Sat Oct 15 2011 to 20:11:16 EST Fri Oct 14 2011,
configured from console by console.
*Oct 14 21:11:16.987: %SYS-6-CLOCKUPDATE: System clock has been updated
from 20:11:16 EST Fri Oct 14 2011 to 21:11:16 EDT Fri Oct 14 2011,
configured from console by console.
Oct 14 21:11:34.274: %HA-5-SYNC_NOTICE: Config sync completed.
Oct 14 21:11:34.926: %SYS-5-RESTART: System restarted --
Cisco IOS Software, RSP Software (RSP-JK9O3SV-M), Version 12.4(25),
RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Tue 21-Apr-09 19:28 by prod_rel_team
Oct 14 21:11:35.046: %SSH-5-ENABLED: SSH 1.99 has been enabled
Oct 14 21:11:39.930: %SNMP-5-COLDSTART: SNMP agent on host CS75-111.1 is
undergoing a cold start
Oct 15 00:24:04.052: %HA-2-CUTOVER_NOTICE: Cutover initiated. Cease all
console activity until system restarts.
Oct 15 00:24:04.052: %HA-2-CUTOVER_NOTICE: Do not add/remove RSPs or
line cards until switchover completes.
Oct 15 00:24:04.052: %HA-2-CUTOVER_NOTICE: Deinitializing subsystems...
Oct 15 00:24:04.052: %OIR-6-REMCARD: Card removed from slot 0,
interfaces disabled
Oct 15 00:24:04.052: %OIR-6-REMCARD: Card removed from slot 1,
interfaces disabled
Oct 15 00:24:04.052: %OIR-6-REMCARD: Card removed from slot 4,
interfaces disabled
Oct 15 00:24:04.052: %OIR-6-REMCARD: Card removed from slot 5,
interfaces disabled
Oct 15 00:24:04.052: %OIR-6-REMCARD: Card removed from slot 6,
interfaces disabled
Oct 15 00:24:04.148: Suspending service policy voice-policy-1024 on
Multilink50021
Oct 15 00:24:04.164: %HA-2-CUTOVER_NOTICE: Reinitializing subsystems...
Oct 15 00:24:04.916: %HA-2-CUTOVER_NOTICE: System preparing to restart...
Oct 15 00:24:04.916: %HA-5-NOTICE: Resuming initialization...
Oct 15 00:24:05.056: %HA-2-CUTOVER_NOTICE: Cutover completed and system
restarted.
Oct 15 00:24:05.056: %HA-2-CUTOVER_NOTICE: You may add/remove RSPs and
line cards after interfaces are fully operational.
Oct 15 00:24:05.100: %HA-5-NOTICE: Standby (slave) will be restarted in
approximately 5 minutes.
Oct 15 00:24:23.412: %RSP-3-SLAVECHANGE: Slave changed state from
Non-participant to Slave
Oct 15 00:29:05.113: %RSP-3-SLAVECHANGE: Slave changed state from Slave
to Non-participant
Oct 15 00:29:07.765: %RSP-3-SLAVECHANGE: Slave changed state from
Non-participant to Slave
Oct 15 00:29:13.449: %RSP-3-SLAVECHANGE: Slave changed state from Slave
to Non-participant
Oct 15 00:29:16.101: %RSP-3-SLAVECHANGE: Slave changed state from
Non-participant to Slave
Oct 15 00:29:21.125: %RSP-3-SLAVECHANGE: Slave changed state from Slave
to Non-participant
Oct 15 00:29:23.777: %RSP-3-SLAVECHANGE: Slave changed state from
Non-participant to Slave
Oct 15 00:30:06.686: %HA-5-NOTICE: Standby (slave) configured to run HA
image "disk0:rsp-jk9o3sv-mz.124-25.bin"
Oct 15 00:30:07.218: %HA-5-NOTICE: Loading standby (slave) image:
"disk0:rsp-jk9o3sv-mz.124-25.bin"
Oct 15 00:31:28.690: %HA-5-MODE: Operating mode is hsa, configured mode
is rpr-plus.
Oct 15 00:31:28.690: %HA-5-SYNC_NOTICE: Bulk sync started.
Oct 15 00:31:28.814: %HA-5-SYNC_NOTICE: Bulk sync completed.
Oct 15 00:31:35.647: %HA-5-SYNC_NOTICE: Config sync started.
Oct 15 00:31:54.067: %HA-5-SYNC_NOTICE: Config sync completed.
Oct 15 00:31:59.483: %HA-5-SYNC_NOTICE: Standby has restarted.
Oct 15 00:31:59.491: %HA-5-MODE: Operating mode is rpr-plus, configured
mode is rpr-plus.
Here's the output of show redundancy:
Redundant System Information :
------------------------------
Available system uptime = 21 hours, 14 minutes
Switchovers system experienced = 2
Standby failures = 0
Last switchover reason = active unit failed
Hardware Mode = Duplex
Configured Redundancy Mode = rpr-plus
Operating Redundancy Mode = rpr-plus
Maintenance Mode = Disabled
Communications = Up
Current Processor Information :
-------------------------------
Active Location = slot 3
Current Software state = ACTIVE
Uptime in current state = 12 hours, 28 minutes
Image Version = Cisco IOS Software, RSP Software
(RSP-JK9O3SV-M), Version 12.4(25), RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Tue 21-Apr-09 19:28 by prod_rel_team
BOOT = disk0:rsp-jk9o3sv-mz.124-25.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
Peer Processor Information :
----------------------------
Standby Location = slot 2
Current Software state = STANDBY COLD-CONFIG
Uptime in current state = 12 hours, 21 minutes
Image Version = Cisco IOS Software, RSP Software
(RSP-JK9O3SV-M), Version 12.4(25), RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Tue 21-Apr-09 19:28 by prod_rel_team
BOOT = disk0:rsp-jk9o3sv-mz.124-25.bin,1;
CONFIG_FILE =
BOOTLDR =
Configuration register = 0x2102
If anymore information is needed let me know and I'll post it. This
router keeps doing this every few hours. Any help would be greatly
appreciated.
Thanks,
Jason
More information about the cisco-nsp
mailing list