[c-nsp] ASR9010 RSP-4G Always rebooting itself.
somphong pokfai
somphong.pokfai at gmail.com
Wed May 29 22:27:35 EDT 2013
Hi,
I have a ASR9010 with a standalone RSP-4G. It is rebooting itself.
Can someone help me please.
logging###########################################
DDR in Interleaved mode
POST 1 : PASSED : code 0 : DDR2 Memory Quick Test
CPU Reset Reason = 0x000d
POST 2 : PASSED : code 0 : FPGA Flash Image CRC Checks
Loading Field Programmable Devices:
FPGA 0-B PROGRAMMED : image: 0xff500028 - 0xff576cca, et: 117ms
FPGA 1-B PROGRAMMED : image: 0xff400028 - 0xff4d1034, et: 206ms
FPGA 2-B PROGRAMMED : image: 0xff100028 - 0xff276358, et: 369ms
FPGA 3-B PROGRAMMED : image: 0xff000028 - 0xff0454a8, et: 69ms
System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],
Copyright (c) 1994-2010 by Cisco Systems, Inc.
Compiled Mon 15-Feb-10 18:14 by saurabja
CPUCtrl: 1.17 [00000001/00000011]
ClkCtrl: 1.18 [00000001/00000012]
IntCtrl: 1.15 [00000001/0000000f]
Punt: 1.5 [00000001/00000005]
CBC: 1.2
BID: 0x0006
PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20080)
M8641 CLKIN: 66 Mhz
Core Clock: 1333 Mhz
MPX Clock: 533 Mhz
LBC Clock: 33 Mhz
POST 3 : PASSED : code 0 : Slot ID/Board Type Validity
PCI-E1: Ready as Root Complex
PCI-E2: Ready as Root Complex
set_chassis_type: chassis_type=0xef02fe found=TRUE
ASR9K (8641D PPC) platform with 4096 Mb of main memory
program load complete, entry point: 0x100000, size: 0x2ac20
program load complete, entry point: 0x100000, size: 0x2ac20
MBI Candidate = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
CARD_SLOT_NUMBER: 5
CPU_INSTANCE: 1
MBI Validation starts ...
Missing or illegal ip address for variable IP_ADDRESS
Mgt LAN 0 interface is selected
tsec_init_hw: configuring FE (port 2) for: Auto Speed, Auto Duplex
tsec_init_interface: hardware initialization completed
Interface link changed state to UP.
Interface link state up.
MBI validation sending request.
HIT CTRL-C to abort
.........
No MBI confirmation received from dSC
AUTOBOOT: Boot string = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm,1;
AUTOBOOT: autobootstate=0, autobootcount=0, cmd=boot
disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
program load complete, entry point: 0x100000, size: 0x2ac20
MBI size from header = 21578140,Bootflash resident MBI filesize = 21578140
...................................................................................
program load complete, entry point: 0x201d9c, size: 0x149329c
Attempting to start second CPU
Config = SMP, Running = SMP
Board type: 0x00100302
Card Capability = 0xffffffff
###################################################################################################################
BSP: Board type : RO-RSP2
tracelogger: starting tracing in background ring mode
tracelogger running with args: -startring -F 1 -F 2
Restricted Rights Legend
Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.
cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706
Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.0
Copyright (c) 2011 by Cisco Systems, Inc.
May 30 02:14:49.132 : Install (Node Preparation): Initializing VS
Distributor...
May 30 02:15:09.292 : Install (Node Preparation): Booting with committed
software
RP/0/RSP1/CPU0:May 30 02:18:34.458: syslogd_helper: [95]:
dsc_event_handler: Got SysMgr dSC event : 1
FPD ltrace_file_name => fpd-agent/fiarsp
FPD ltrace_file_name => fpd-agent/tempo
FPD ltrace_file_name => fpd-agent/longbeach
RP/0/RSP1/CPU0:May 30 02:18:27.815 : [144]: Invalid /bootflash: entry =>
.junk_entity
RP/0/RSP1/CPU0:May 30 02:18:27.815 : [144]: Do not initiate further install
or mirror operations without removing this invalid /bootflash: entry
RP/0/RSP1/CPU0:May 30 02:18:29.191 : spp[92]: %PKT_INFRA-spp-3-ERR : Node
'port3/classify' disposition 4 ('ptp_off_rx_node') not found, using 'drop'
instead
RP/0/RSP1/CPU0:May 30 02:18:36.931 : fiarsp[218]: DEBUG: FPD ltrace
(fpd-agent/fiarsp) Initialization successful for fiarsp
RP/0/RSP1/CPU0:May 30 02:18:42.552 : tempo[425]: DEBUG: FPD ltrace
(fpd-agent/tempo) Initialization successful for tempo
RP/0/RSP1/CPU0:May 30 02:18:45.688 : tempo[425]:
%PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga3 instance 0 is down-rev (V1.18),
upgrade to (V1.23). Use the "upgrade hw-module fpd" CLI in admin mode.
RP/0/RSP1/CPU0:May 30 02:18:54.543 : cfgmgr-rp[162]: ISSU: Starting sysdb
bulk start session
RP/0/RSP1/CPU0:May 30 02:18:56.557 : longbeach[68]: DEBUG: FPD ltrace
(fpd-agent/longbeach) Initialization successful for longbeach
ios con0/RSP1/CPU0 is now available
Press RETURN to get started.
RP/0/RSP1/CPU0:May 30 02:19:13.400 : pfm_node_rp[351]: %FABRIC-FWriting
crashinfo
Active processes:
proc/boot/procnto-booke-smp-instr Thread ID 21 on cpu 0
Active processes:
pkg/bin/pfm_node_rp Thread ID 0 on cpu 1
[0x8eec77bbf] Record Reboot History, reboot cause = 0x2c00001b, descr =
Cause: pfm_dev_sm_perform_recovery_action, Card reset requested by: Process
ID: 254060 (fiarsp), Fault Sev: 0, Target node: 0/RSP1/CPU0, CompId: 0x10,
Device Handle: 0x1013000, CondI[0x8f115f19e] Record crashinfo
[0x8f163eff1] Record Syslog
2013-05-30 02:19:13.498
NOTE: This is NOT a Kernel Crash. This crash was triggered
by the process 'pfm_node_rp', by calling reboot API.
Crash Reason: Cause: pfm_dev_sm_perform_recovery_action, Card reset
requested by: Process ID: 254060 (fiarsp), Fault Sev: 0, Target node:
0/RSP1/CPU0, CompId: 0x10, Device Handle: 0x1013000, CondID: 2545, Fault
Reason: Fabric interface asic ASIC0 encountered fatal faul (Cause Code:
0x2c00001b)
Exception at 0x4c2466d0 signal 5 c=1 f=3
Active process(s):
proc/boot/procnto-booke-smp-instr Thread ID 21 on cpu 0
pkg/bin/pfm_node_rp Thread ID 0 on cpu 1
REGISTER INFO
r0 r1 r2 r3
R0 4c2466cc e7ffdeb0 50013830 00000003
r4 r5 r6 r7
R4 2800001b e7ffe0e9 e7ffde88 00000000
r8 r9 r10 r11
R8 c88f9e00 00000000 1d194d9d ec019570
r12 r13 r14 r15
R12 4c279c08 50013830 e7fff9d0 00000001
r16 r17 r18 r19
R16 e7fff9e4 e7fff9ec e7ffe5a0 00000000
r20 r21 r22 r23
R20 00000000 00000000 ec34c640 01013000
r24 r25 r26 r27
R24 5000d170 e7ffdf88 00000051 01013000
r28 r29 r30 r31
R28 e7ffe0e9 00000000 ec019984 2800001b
cnt lr msr pc
R32 4c1cbe88 4c2466cc 0002d932 4c2466d0
cnd xer
R36 44004084 20000000
SUPERVISOR REGISTERS
Memory Management Registers
Instruction BAT Registers
Index # Value
IBAT0U # 0x1ffe
IBAT0L # 0x12
IBAT1U # 0
IBAT1L # 0
IBAT2U # 0
IBAT2L # 0
IBAT3U # 0xfffc0003
IBAT3L # 0x60011
IBAT4U # 0x4c0007ff
IBAT4L # 0x74000011
IBAT5U # 0
IBAT5L # 0
IBAT6U # 0
IBAT6L # 0
IBAT7U # 0
IBAT7L # 0
Data BAT Registers
Index # Value
DBAT0U # 0x1ffe
DBAT0L # 0x12
DBAT1U # 0x34000002
DBAT1L # 0xdc00002a
DBAT2U # 0x3000001e
DBAT2L # 0xc800002a
DBAT3U # 0xfffc0003
DBAT3L # 0x60011
DBAT4U # 0x4c0007ff
DBAT4L # 0x74000011
DBAT5U # 0
DBAT5L # 0
DBAT6U # 0
DBAT6L # 0
DBAT7U # 0
DBAT7L # 0
Exception Handling Registers
Data Addr Reg # DSISR
0x1c035000 # 0x42000000
SPRG0 # SPRG1 # SPRG2 # SPRG3
0x2800001b # 0xec019984 # 0 # 0x1
SaveNRestore SRR0 # SaveNRestore SRR1
0x4c2466cc # 0x2d932
Miscellaneous Registers
Processor Id Reg # 0x1
HID0 # 0x8493c1bc
HID1 # 0x2cc80
MSSCR0 # 0x8020
MSSSR0 # 0
STACK TRACE
#0 0x4c2466cc
[0x9140ea8f9] Initializing harddisk file system
[0x91873474a] Record TSEC information
!!
Writing TSEC done
!
Writing crashinfo done!
Examine crashinfo file for reboot reason
Writing ppc kernel core file
[0x91a01989e] Kernel core dump start...
fill phdr vaddr=0xf462000, offset=0x43a83d4, size=0xb9e000
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Core dump success. Total_size 83125204
[0x935bc1d88] Successfully dumped Kernel core
[0x93628c375] Record PCDS information
Writing PCDS done
Dump Directory
KD: RSP1.130530-021913.tsec, start = 1000, size = 4000, crc = 0
KD: RSP1.130530-021913.crashinfo.by.pfm_node_rp, start = 5000, size = ae8b,
crc = 0
KD: RSP1.130530-021913.kernel_core.by.pfm_node_rp.Z, start = 10000, size =
ca99f4, crc = 9b470080
KD: RSP1.130530-021913.pcds, start = cba000, size = ff000, crc = 18ff3d46
Writing kernel core file done!
rebooting
Selecting ROMMON Image... B
DDR in Interleaved mode
POST 1 : PASSED : code 0 : DDR2 Memory Quick Test
CPU Reset Reason = 0x000d
POST 2 : PASSED : code 0 : FPGA Flash Image CRC Checks
Loading Field Programmable Devices:
FPGA 0-B PROGRAMMED : image: 0xff500028 - 0xff576cca, et: 117ms
FPGA 1-B PROGRAMMED : image: 0xff400028 - 0xff4d1034, et: 206ms
FPGA 2-B PROGRAMMED : image: 0xff100028 - 0xff276358, et: 369ms
FPGA 3-B PROGRAMMED : image: 0xff000028 - 0xff0454a8, et: 69ms
System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],
Copyright (c) 1994-2010 by Cisco Systems, Inc.
Compiled Mon 15-Feb-10 18:14 by saurabja
CPUCtrl: 1.17 [00000001/00000011]
ClkCtrl: 1.18 [00000001/00000012]
IntCtrl: 1.15 [00000001/0000000f]
Punt: 1.5 [00000001/00000005]
CBC: 1.2
BID: 0x0006
PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20080)
M8641 CLKIN: 66 Mhz
Core Clock: 1333 Mhz
MPX Clock: 533 Mhz
LBC Clock: 33 Mhz
POST 3 : PASSED : code 0 : Slot ID/Board Type Validity
PCI-E1: Ready as Root Complex
PCI-E2: Ready as Root Complex
set_chassis_type: chassis_type=0xef02fe found=TRUE
ASR9K (8641D PPC) platform with 4096 Mb of main memory
program load complete, entry point: 0x100000, size: 0x2ac20
program load complete, entry point: 0x100000, size: 0x2ac20
MBI Candidate = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
CARD_SLOT_NUMBER: 5
CPU_INSTANCE: 1
MBI Validation starts ...
Missing or illegal ip address for variable IP_ADDRESS
Mgt LAN 0 interface is selected
tsec_init_hw: configuring FE (port 2) for: Auto Speed, Auto Duplex
tsec_init_interface: hardware initialization completed
Interface link changed state to UP.
Interface link state up.
MBI validation sending request.
HIT CTRL-C to abort
..........
No MBI confirmation received from dSC
AUTOBOOT: Boot string = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm,1;
AUTOBOOT: autobootstate=0, autobootcount=0, cmd=boot
disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
program load complete, entry point: 0x100000, size: 0x2ac20
MBI size from header = 21578140,Bootflash resident MBI filesize = 21578140
...................................................................................
program load complete, entry point: 0x201d9c, size: 0x149329c
Attempting to start second CPU
Config = SMP, Running = SMP
Board type: 0x00100302
Card Capability = 0xffffffff
###################################################################################################################
BSP: Board type : RO-RSP2
tracelogger: starting tracing in background ring mode
tracelogger running with args: -startring -F 1 -F 2
Restricted Rights Legend
Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.
cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706
Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.0
Copyright (c) 2011 by Cisco Systems, Inc.
May 30 02:20:28.113 : Install (Node Preparation): Initializing VS
Distributor...
May 30 02:20:48.273 : Install (Node Preparation): Booting with committed
software
FPD ltrace_file_name => fpd-agent/fiarsp
RP/0/RSP1/CPU0:May 30 02:24:56.309: syslogd_helper: [95]:
dsc_event_handler: Got SysMgr dSC event : 1
FPD ltrace_file_name => fpd-agent/tempo
RP/0/RSP1/CPU0:May 30 02:24:46.697 : [144]: Invalid /bootflash: entry =>
.junk_entity
RP/0/RSP1/CPU0:May 30 02:24:46.697 : [144]: Do not initiate further install
or mirror operations without removing this invalid /bootflash: entry
RP/0/RSP1/CPU0:May 30 02:24:47.969 : spp[92]: %PKT_INFRA-spp-3-ERR : Node
'port3/classify' disposition 4 ('ptp_off_rx_node') not found, using 'drop'
instead
RP/0/RSP1/CPU0:May 30 02:24:54.398 : fiarsp[218]: DEBUG: FPD ltrace
(fpd-agent/fiarsp) Initialization successful for fiarsp
RP/0/RSP1/CPU0:May 30 02:25:01.409 : tempo[425]: DEBUG: FPD ltrace
(fpd-agent/tempo) Initialization successful for tempo
RP/0/RSP1/CPU0:May 30 02:25:07.616 : tempo[425]:
%PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga3 instance 0 is down-rev (V1.18),
upgrade to (V1.23). Use the "upgrade hw-module fpd" CLI in admin mode.
RP/0/RSP1/CPU0:May 30 02:25:13.315 : cfgmgr-rp[162]: ISSU: Starting sysdb
bulk start session
ios con0/RSP1/CPU0 is now available
Press RETURN to get started.
RP/0/RSP1/CPU0:May 30 02:25:32.211 : pfm_node_rp[351]: %FABRIC-FWriting
crashinfo
Active processes:
pkg/bin/pfm_node_rp Thread ID 0 on cpu 0
Active processes:
pkg/bin/devb-umass Thread ID 13 on cpu 1
[0xa2a02233c] Record Reboot History, reboot cause = 0x2c00001b, descr =
Cause: pfm_dev_sm_perform_recovery_action, Card reset requested by: Process
ID: 254060 (fiarsp), Fault Sev: 0, Target node: 0/RSP1/CPU0, CompId: 0x10,
Device Handle: 0x1013000, CondI[0xa2c509a20] Record crashinfo
[0xa2c9e9450] Record Syslog
2013-05-30 02:25:32.291
NOTE: This is NOT a Kernel Crash. This crash was triggered
by the process 'pfm_node_rp', by calling reboot API.
Crash Reason: Cause: pfm_dev_sm_perform_recovery_action, Card reset
requested by: Process ID: 254060 (fiarsp), Fault Sev: 0, Target node:
0/RSP1/CPU0, CompId: 0x10, Device Handle: 0x1013000, CondID: 2545, Fault
Reason: Fabric interface asic ASIC0 encountered fatal faul (Cause Code:
0x2c00001b)
Exception at 0x4c2466d0 signal 5 c=1 f=3
Active process(s):
pkg/bin/pfm_node_rp Thread ID 0 on cpu 0
pkg/bin/devb-umass Thread ID 13 on cpu 1
REGISTER INFO
r0 r1 r2 r3
R0 4c2466cc e7ffe040 50013830 00000003
r4 r5 r6 r7
R4 2800001b e7ffe279 e7ffe018 00000000
r8 r9 r10 r11
R8 06b8ac00 00000000 10c0e98a ec019570
r12 r13 r14 r15
R12 4c279c08 50013830 e7fffb60 00000001
r16 r17 r18 r19
R16 e7fffb74 e7fffb7c e7ffe730 00000000
r20 r21 r22 r23
R20 00000000 00000000 ec34f640 01013000
r24 r25 r26 r27
R24 5000d170 e7ffe118 00000051 01013000
r28 r29 r30 r31
R28 e7ffe279 00000000 ec019984 2800001b
cnt lr msr pc
R32 4c1cbe88 4c2466cc 0002d932 4c2466d0
cnd xer
R36 44004084 20000000
SUPERVISOR REGISTERS
Memory Management Registers
Instruction BAT Registers
Index # Value
IBAT0U # 0x1ffe
IBAT0L # 0x12
IBAT1U # 0
IBAT1L # 0
IBAT2U # 0
IBAT2L # 0
IBAT3U # 0xfffc0003
IBAT3L # 0x40011
IBAT4U # 0x4c0007ff
IBAT4L # 0x74000011
IBAT5U # 0
IBAT5L # 0
IBAT6U # 0
IBAT6L # 0
IBAT7U # 0
IBAT7L # 0
Data BAT Registers
Index # Value
DBAT0U # 0x1ffe
DBAT0L # 0x12
DBAT1U # 0x34000002
DBAT1L # 0xdc00002a
DBAT2U # 0x3000001e
DBAT2L # 0xc800002a
DBAT3U # 0xfffc0003
DBAT3L # 0x40011
DBAT4U # 0x4c0007ff
DBAT4L # 0x74000011
DBAT5U # 0
DBAT5L # 0
DBAT6U # 0
DBAT6L # 0
DBAT7U # 0
DBAT7L # 0
Exception Handling Registers
Data Addr Reg # DSISR
0x50af3000 # 0x42000000
SPRG0 # SPRG1 # SPRG2 # SPRG3
0x2800001b # 0xec019984 # 0 # 0
SaveNRestore SRR0 # SaveNRestore SRR1
0x4c2466cc # 0x2d932
Miscellaneous Registers
Processor Id Reg # 0
HID0 # 0x8493c1bc
HID1 # 0x2cc80
MSSCR0 # 0x8000
MSSSR0 # 0
STACK TRACE
#0 0x4c2466cc
[0xa4f2659d5] Initializing harddisk file system
[0xa53843f4d] Record TSEC information
!!
Writing TSEC done
!
Writing crashinfo done!
Examine crashinfo file for reboot reason
Writing ppc kernel core file
[0xa5512925b] Kernel core dump start...
fill phdr vaddr=0xf40c000, offset=0x43dd3d4, size=0xbf4000
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Core dump success. Total_size 83694548
[0xa7111d69e] Successfully dumped Kernel core
[0xa717ac541] Record PCDS information
!
Writing PCDS done
Dump Directory
KD: RSP1.130530-022532.tsec, start = 1000, size = 4000, crc = 0
KD: RSP1.130530-022532.crashinfo.by.pfm_node_rp, start = 5000, size = ae7c,
crc = 0
KD: RSP1.130530-022532.kernel_core.by.pfm_node_rp.Z, start = 10000, size =
ce9b82, crc = 795ac33c
KD: RSP1.130530-022532.pcds, start = cfa000, size = ff000, crc = cf5063fb
Writing kernel core file done!
rebooting
Selecting ROMMON Image... B
DDR in Interleaved mode
More information about the cisco-nsp
mailing list