[c-nsp] ASR9010 RSP-4G Always rebooting itself.
Aaron
dudepron at gmail.com
Thu May 30 17:03:53 EDT 2013
Looks like an ASIC issue, open up a tac case
On Wednesday, May 29, 2013, somphong pokfai wrote:
> Hi,
>
> I have a ASR9010 with a standalone RSP-4G. It is rebooting itself.
> Can someone help me please.
>
> logging###########################################
> DDR in Interleaved mode
> POST 1 : PASSED : code 0 : DDR2 Memory Quick Test
>
> CPU Reset Reason = 0x000d
> POST 2 : PASSED : code 0 : FPGA Flash Image CRC Checks
>
> Loading Field Programmable Devices:
> FPGA 0-B PROGRAMMED : image: 0xff500028 - 0xff576cca, et: 117ms
> FPGA 1-B PROGRAMMED : image: 0xff400028 - 0xff4d1034, et: 206ms
> FPGA 2-B PROGRAMMED : image: 0xff100028 - 0xff276358, et: 369ms
> FPGA 3-B PROGRAMMED : image: 0xff000028 - 0xff0454a8, et: 69ms
>
> System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],
> Copyright (c) 1994-2010 by Cisco Systems, Inc.
> Compiled Mon 15-Feb-10 18:14 by saurabja
>
> CPUCtrl: 1.17 [00000001/00000011]
> ClkCtrl: 1.18 [00000001/00000012]
> IntCtrl: 1.15 [00000001/0000000f]
> Punt: 1.5 [00000001/00000005]
> CBC: 1.2
> BID: 0x0006
>
>
> PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20080)
> M8641 CLKIN: 66 Mhz
> Core Clock: 1333 Mhz
> MPX Clock: 533 Mhz
> LBC Clock: 33 Mhz
>
> POST 3 : PASSED : code 0 : Slot ID/Board Type Validity
> PCI-E1: Ready as Root Complex
> PCI-E2: Ready as Root Complex
>
>
> set_chassis_type: chassis_type=0xef02fe found=TRUE
> ASR9K (8641D PPC) platform with 4096 Mb of main memory
>
> program load complete, entry point: 0x100000, size: 0x2ac20
> program load complete, entry point: 0x100000, size: 0x2ac20
> MBI Candidate = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
>
> CARD_SLOT_NUMBER: 5
> CPU_INSTANCE: 1
> MBI Validation starts ...
> Missing or illegal ip address for variable IP_ADDRESS
>
> Mgt LAN 0 interface is selected
> tsec_init_hw: configuring FE (port 2) for: Auto Speed, Auto Duplex
>
> tsec_init_interface: hardware initialization completed
> Interface link changed state to UP.
> Interface link state up.
>
> MBI validation sending request.
> HIT CTRL-C to abort
> .........
> No MBI confirmation received from dSC
>
> AUTOBOOT: Boot string = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm,1;
> AUTOBOOT: autobootstate=0, autobootcount=0, cmd=boot
> disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
> program load complete, entry point: 0x100000, size: 0x2ac20
>
> MBI size from header = 21578140,Bootflash resident MBI filesize = 21578140
>
> ...................................................................................
> program load complete, entry point: 0x201d9c, size: 0x149329c
> Attempting to start second CPU
> Config = SMP, Running = SMP
> Board type: 0x00100302
> Card Capability = 0xffffffff
>
> ###################################################################################################################
> BSP: Board type : RO-RSP2
> tracelogger: starting tracing in background ring mode
> tracelogger running with args: -startring -F 1 -F 2
> Restricted Rights Legend
>
> Use, duplication, or disclosure by the Government is
> subject to restrictions as set forth in subparagraph
> (c) of the Commercial Computer Software - Restricted
> Rights clause at FAR sec. 52.227-19 and subparagraph
> (c) (1) (ii) of the Rights in Technical Data and Computer
> Software clause at DFARS sec. 252.227-7013.
>
> cisco Systems, Inc.
> 170 West Tasman Drive
> San Jose, California 95134-1706
>
>
>
> Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.0
> Copyright (c) 2011 by Cisco Systems, Inc.
> May 30 02:14:49.132 : Install (Node Preparation): Initializing VS
> Distributor...
> May 30 02:15:09.292 : Install (Node Preparation): Booting with committed
> software
> RP/0/RSP1/CPU0:May 30 02:18:34.458: syslogd_helper: [95]:
> dsc_event_handler: Got SysMgr dSC event : 1
> FPD ltrace_file_name => fpd-agent/fiarsp
> FPD ltrace_file_name => fpd-agent/tempo
> FPD ltrace_file_name => fpd-agent/longbeach
> RP/0/RSP1/CPU0:May 30 02:18:27.815 : [144]: Invalid /bootflash: entry =>
> .junk_entity
> RP/0/RSP1/CPU0:May 30 02:18:27.815 : [144]: Do not initiate further install
> or mirror operations without removing this invalid /bootflash: entry
> RP/0/RSP1/CPU0:May 30 02:18:29.191 : spp[92]: %PKT_INFRA-spp-3-ERR : Node
> 'port3/classify' disposition 4 ('ptp_off_rx_node') not found, using 'drop'
> instead
> RP/0/RSP1/CPU0:May 30 02:18:36.931 : fiarsp[218]: DEBUG: FPD ltrace
> (fpd-agent/fiarsp) Initialization successful for fiarsp
> RP/0/RSP1/CPU0:May 30 02:18:42.552 : tempo[425]: DEBUG: FPD ltrace
> (fpd-agent/tempo) Initialization successful for tempo
> RP/0/RSP1/CPU0:May 30 02:18:45.688 : tempo[425]:
> %PLATFORM-UPGRADE_FPD-4-DOWN_REV : fpga3 instance 0 is down-rev (V1.18),
> upgrade to (V1.23). Use the "upgrade hw-module fpd" CLI in admin mode.
> RP/0/RSP1/CPU0:May 30 02:18:54.543 : cfgmgr-rp[162]: ISSU: Starting sysdb
> bulk start session
> RP/0/RSP1/CPU0:May 30 02:18:56.557 : longbeach[68]: DEBUG: FPD ltrace
> (fpd-agent/longbeach) Initialization successful for longbeach
>
>
> ios con0/RSP1/CPU0 is now available
>
>
>
>
>
> Press RETURN to get started.
>
> RP/0/RSP1/CPU0:May 30 02:19:13.400 : pfm_node_rp[351]: %FABRIC-FWriting
> crashinfo
> Active processes:
> proc/boot/procnto-booke-smp-instr Thread ID 21 on cpu 0
>
> Active processes:
> pkg/bin/pfm_node_rp Thread ID 0 on cpu 1
>
> [0x8eec77bbf] Record Reboot History, reboot cause = 0x2c00001b, descr =
> Cause: pfm_dev_sm_perform_recovery_action, Card reset requested by: Process
> ID: 254060 (fiarsp), Fault Sev: 0, Target node: 0/RSP1/CPU0, CompId: 0x10,
> Device Handle: 0x1013000, CondI[0x8f115f19e] Record crashinfo
> [0x8f163eff1] Record Syslog
> 2013-05-30 02:19:13.498
> NOTE: This is NOT a Kernel Crash. This crash was triggered
> by the process 'pfm_node_rp', by calling reboot API.
>
> Crash Reason: Cause: pfm_dev_sm_perform_recovery_action, Card reset
> requested by: Process ID: 254060 (fiarsp), Fault Sev: 0, Target node:
> 0/RSP1/CPU0, CompId: 0x10, Device Handle: 0x1013000, CondID: 2545, Fault
> Reason: Fabric interface asic ASIC0 encountered fatal faul (Cause Code:
> 0x2c00001b)
>
> Exception at 0x4c2466d0 signal 5 c=1 f=3
>
> Active process(s):
> proc/boot/procnto-booke-smp-instr Thread ID 21 on cpu 0
> pkg/bin/pfm_node_rp Thread ID 0 on cpu 1
>
> REGISTER INFO
> r0 r1 r2 r3
> R0 4c2466cc e7ffdeb0 50013830 00000003
> r4 r5 r6 r7
> R4 2800001b e7ffe0e9 e7ffde88 00000000
> r8 r9 r10 r11
> R8 c88f9e00 00000000 1d194d9d ec019570
> r12 r13 r14 r15
> R12 4c279c08 50013830 e7fff9d0 00000001
> r16 r17 r18 r19
> R16 e7fff9e4 e7fff9ec e7ffe5a0 00000000
> r20 r21 r22 r23
> R20 00000000 00000000 ec34c640 01013000
> r24 r25 r26 r27
> R24 5000d170 e7ffdf88 00000051 01013000
> r28 r29 r30 r31
> R28 e7ffe0e9 00000000 ec019984 2800001b
> cnt lr msr pc
> R32 4c1cbe88 4c2466cc 0002d932 4c2466d0
> cnd xer
> R36 44004084 20000000
>
> SUPERVISOR REGISTERS
>
>
>
> Memory Management Registers
>
> Instruction BAT Registers
> Index # Value
> IBAT0U # 0x1ffe
> IBAT0L # 0x12
> IBAT1U # 0
> IBAT1L # 0
> IBAT2U # 0
> IBAT2L # 0
> IBAT3U # 0xfffc0003
> IBAT3L # 0x60011
> IBAT4U # 0x4c0007ff
> IBAT4L # 0x74000011
> IBAT5U # 0
> IBAT5L # 0
> IBAT6U # 0
> IBAT6L # 0
> IBAT7U # 0
> IBAT7L # 0
>
> Data BAT Registers
> Index # Value
> DBAT0U # 0x1ffe
> DBAT0L # 0x12
> DBAT1U # 0x34000002
> DBAT1L # 0xdc00002a
> DBAT2U # 0x3000001e
> DBAT2L # 0xc800002a
> DBAT3U # 0xfffc0003
> DBAT3L # 0x60011
> DBAT4U # 0x4c0007ff
> DBAT4L # 0x74000011
> DBAT5U # 0
> DBAT5L # 0
> DBAT6U # 0
> DBAT6L # 0
> DBAT7U # 0
> DBAT7L # 0
>
>
> Exception Handling Registers
> Data Addr Reg # DSISR
> 0x1c035000 # 0x42000000
> SPRG0 # SPRG1 # SPRG2 # SPRG3
> 0x2800001b # 0xec019984 # 0 # 0x1
> SaveNRestore SRR0 # SaveNRestore SRR1
> 0x4c2466cc # 0x2d932
>
>
> Miscellaneous Registers
> Processor Id Reg # 0x1
> HID0 # 0x8493c1bc
> HID1 # 0x2cc80
>
> MSSCR0 # 0x8020
> MSSSR0 # 0
>
> STACK TRACE
> #0 0x4c2466cc
> [0x9140ea8f9] Initializing harddisk file system
> [0x91873474a] Record TSEC information
> !!
> Writing TSEC done
> !
> Writing crashinfo done!
>
> Examine crashinfo file for reboot reason
>
> Writing ppc kernel core file
> [0x91a01989e] Kernel core dump start...
> fill phdr vaddr=0xf462000, offset=0x43a83d4, size=0xb9e000
>
> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
> Core dump success. Total_size 83125204
> [0x935bc1d88] Successfully dumped Kernel core
> [0x93628c375] Record PCDS information
>
> Writing PCDS done
> Dump Directory
> KD: RSP1.130530-021913.tsec, start = 1000, size = 4000, crc = 0
> KD: RSP1.130530-021913.crashinfo.by.pfm_node_rp, start = 5000, size = ae8b,
> crc = 0
> KD: RSP1.130530-021913.kernel_core.by.pfm_node_rp.Z, start = 10000, size =
> ca99f4, crc = 9b470080
> KD: RSP1.130530-021913.pcds, start = cba000, size = ff000, crc = 18ff3d46
>
> Writing kernel core file done!
> rebooting
>
> Selecting ROMMON Image... B
> DDR in Interleaved mode
> POST 1 : PASSED : code 0 : DDR2 Memory Quick Test
>
> CPU Reset Reason = 0x000d
> POST 2 : PASSED : code 0 : FPGA Flash Image CRC Checks
>
> Loading Field Programmable Devices:
> FPGA 0-B PROGRAMMED : image: 0xff500028 - 0xff576cca, et: 117ms
> FPGA 1-B PROGRAMMED : image: 0xff400028 - 0xff4d1034, et: 206ms
> FPGA 2-B PROGRAMMED : image: 0xff100028 - 0xff276358, et: 369ms
> FPGA 3-B PROGRAMMED : image: 0xff000028 - 0xff0454a8, et: 69ms
>
> System Bootstrap, Version 1.04(20100216:021454) [ASR9K ROMMON],
> Copyright (c) 1994-2010 by Cisco Systems, Inc.
> Compiled Mon 15-Feb-10 18:14 by saurabja
>
> CPUCtrl: 1.17 [00000001/00000011]
> ClkCtrl: 1.18 [00000001/00000012]
> IntCtrl: 1.15 [00000001/0000000f]
> Punt: 1.5 [00000001/00000005]
> CBC: 1.2
> BID: 0x0006
>
>
> PPC 8641D (partnum 0x0003), Revision 0.2, (Core Version 2.20080)
> M8641 CLKIN: 66 Mhz
> Core Clock: 1333 Mhz
> MPX Clock: 533 Mhz
> LBC Clock: 33 Mhz
>
> POST 3 : PASSED : code 0 : Slot ID/Board Type Validity
> PCI-E1: Ready as Root Complex
> PCI-E2: Ready as Root Complex
>
>
> set_chassis_type: chassis_type=0xef02fe found=TRUE
> ASR9K (8641D PPC) platform with 4096 Mb of main memory
>
> program load complete, entry point: 0x100000, size: 0x2ac20
> program load complete, entry point: 0x100000, size: 0x2ac20
> MBI Candidate = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
>
> CARD_SLOT_NUMBER: 5
> CPU_INSTANCE: 1
> MBI Validation starts ...
> Missing or illegal ip address for variable IP_ADDRESS
>
> Mgt LAN 0 interface is selected
> tsec_init_hw: configuring FE (port 2) for: Auto Speed, Auto Duplex
>
> tsec_init_interface: hardware initialization completed
> Interface link changed state to UP.
> Interface link state up.
>
> MBI validation sending request.
> HIT CTRL-C to abort
> ..........
> No MBI confirmation received from dSC
>
> AUTOBOOT: Boot string = disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm,1;
> AUTOBOOT: autobootstate=0, autobootcount=0, cmd=boot
> disk0:asr9k-os-mbi-4.2.0/0x100000/mbiasr9k-rp.vm
> program load complete, entry point: 0x100000, size: 0x2ac20
>
> MBI size from header = 21578140,Bootflash resident MBI filesize = 21578140
>
> ...................................................................................
> program load complete, entry point: 0x201d9c, size: 0x149329c
> Attempting to start second CPU
> Config = SMP, Running = SMP
> Board type: 0x00100302
> Card Capability = 0xffffffff
>
> ###################################################################################################################
> BSP: Board type : RO-RSP2
> tracelogger: starting tracing in background ring mode
> tracelogger running with args: -startring -F 1 -F 2
> Restricted Rights Legend
>
> Use, duplication, or disclosure by the Government is
> subject to restrictions as set forth in subparagraph
> (c) of the Commercial Computer Software - Restricted
> Rights clause at FAR sec. 52.227-19 and subparagraph
> (c) (1) (ii) of the Rights in Technical Data and Computer
> Software clause at DFARS sec. 252.227-7013.
>
> cisco Systems, Inc.
> 170 West Tasman Drive
> San Jose, California 95134-1706
>
>
>
> Cisco IOS XR Software for the Cisco XR ASR9K, Version 4.2.0
>
More information about the cisco-nsp
mailing list