[c-nsp] Problem with 7507 ATM Crashing

Rodney Dunn rodunn at cisco.com
Tue Jul 11 13:52:34 EDT 2006


Skeeve,

What does 'sh diag' say for that PA. I did a couple of quick bug
searches and a couple of them pointed back to this hardware
field notice:

http://www.cisco.com/en/US/products/hw/routers/ps341/products_field_notice09186a0080178b6b.shtml

I have no clue what the upgrade might have triggered it if that's
relevant.

Rodney

On Tue, Jul 11, 2006 at 12:06:47PM +1000, Skeeve Stevens wrote:
> 
> Hey everyone,
> 
> 	I have a 7507 - details below.  I did an upgrade of the IOS from:
> rsp-ik9sv-mz.122-26.bin to rsp-ik9sv-mz.122-37.bin - which in all rights
> would seem like a simple thing. no feature change or much but some
> revisions.
> 
> 	The reason we were upgrading is the ATM card was locking up, which
> was fixable by a shut, no shut.  No idea what was actually causing it though
> - no logs or anything.
> 
> 	So I upgraded last night to 12.2(37) and then it all went to hell.
> The logs showed:
> 
> 0:00:50: %SYS-5-CONFIG_I: Configured from memory by console
> 00:02:31: %SCHED-3-THRASHING: Process thrashing on watched simple timer
> (0x42914540).
> -Process= "ATM PA Helper", ipl= 6, pid= 47
> -Traceback= 403BFA3C 403BFDFC 404BC11C
> 00:02:31: %VIP2 R5K-3-MSG: slot0 VIP-3-LQR_IPC_FAIL:  IPC open failed
> ...
> 00:02:31: %VIP2 R5K-3-MSG: slot1 VIP-3-LQR_IPC_FAIL:  IPC open failed
> 00:02:31: %SYS-5-RESTART: System restarted --
> Cisco Internetwork Operating System Software 
> IOS (tm) RSP Software (RSP-IK9SV-M), Version 12.2(37), RELEASE SOFTWARE
> (fc1)
> Copyright (c) 1986-2006 by cisco Systems, Inc.
> Compiled Thu 15-Jun-06 19:32 by pwade
> 00:02:31: %SNMP-5-COLDSTART: SNMP agent on host router is undergoing a cold
> start
> 00:02:31: %SYS-6-BOOTTIME: Time taken to reboot after reload =  322 seconds
> 00:02:31: %SSH-5-ENABLED: SSH 1.5 has been enabled
> 00:02:32: %LINEPROTO-5-UPDOWN: Line protocol on Interface ATM0/1/0, changed
> state to down
> 00:04:13: %SCHED-3-THRASHING: Process thrashing on watched simple timer
> (0x42914540).
> -Process= "ATM PA Helper", ipl= 6, pid= 47
> -Traceback= 403BFA3C 403BFDFC 404BC11C
> 00:06:43: %SCHED-3-THRASHING: Process thrashing on watched simple timer
> (0x42914540).
> -Process= "ATM PA Helper", ipl= 6, pid= 47
> -Traceback= 403BFA3C 403BFDFC 404BC11C
> 00:09:13: %SCHED-3-THRASHING: Process thrashing on watched simple timer
> (0x42914540).
> -Process= "ATM PA Helper", ipl= 6, pid= 47
> -Traceback= 403BFA3C 403BFDFC 404BC11C
> 00:11:43: %SCHED-3-THRASHING: Process thrashing on watched simple timer
> (0x42914540).
> -Process= "ATM PA Helper", ipl= 6, pid= 47
> -Traceback= 403BFA3C 403BFDFC 404BC11C
> 
> 
> I downgraded back to 12.2(26) and it came back to its current working state.
> 
> Anyone know why such a trivial upgrade could cause such drastic problems?
> 
> .Skeeve
> 
> 
> 
> ---
> 
> cisco RSP4 (R5000) processor with 262144K/2072K bytes of memory.
> R5000 CPU at 200Mhz, Implementation 35, Rev 2.1, 512KB L2 Cache
> Last reset from power-on
> G.703/E1 software, Version 1.0.
> G.703/JT2 software, Version 1.0.
> X.25 software, Version 3.0.0.
> Bridging software.
> Chassis Interface.
> 2 VIP2 R5K controllers (3 FastEthernet)(1 ATM).
> 3 FastEthernet/IEEE 802.3 interface(s)
> 1 ATM network interface(s)
> 123K bytes of non-volatile configuration memory.
> 
> 32768K bytes of Flash PCMCIA card at slot 0 (Sector size 128K).
> 16384K bytes of Flash PCMCIA card at slot 1 (Sector size 128K).
> 8192K bytes of Flash internal SIMM (Sector size 256K).
> 
> 
> router#show diag
> Slot 0:
>         Physical slot 0, ~physical slot 0xF, logical slot 0, CBus 0
>         Microcode Status 0x4
>         Master Enable, LED, WCS Loaded
>         Board is analyzed 
>         Pending I/O Status: None
>         EEPROM format version 1
>         VIP2 R5K controller, HW rev 2.02, board revision C0
>         Serial number: 12719540  Part number: 73-2167-05
>         Test history: 0x00        RMA number: 00-00-00
>         Flags: cisco 7000 board; 7500 compatible
> 
>         EEPROM contents (hex):
>           0x20: 01 1E 02 02 00 C2 15 B4 49 08 77 05 00 00 00 00
>           0x30: 60 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
> 
>         Slot database information:
>         Flags: 0x4      Insertion time: 0x2C3C (00:06:14 ago)
> 
>         Controller Memory Size: 128 MBytes DRAM, 8192 KBytes SRAM
> 
>         PA Bay 0 Information:
>                 Fast-Ethernet PA, 1 ports, 100BaseTX-ISL
>                 EEPROM format version 1
>                 HW rev 1.04, Board revision B0
>                 Serial number: 15805065  Part number: 73-1688-05 
> 
>         PA Bay 1 Information:
>                 ENHANCED ATM OC3 MM PA, 1 ports
>                 EEPROM format version 1
>                 HW rev 1.00, Board revision A0
>                 Serial number: 09590776  Part number: 73-2430-03 
> 
>         --Boot log begin--
> 
> Cisco Internetwork Operating System Software 
> IOS (tm) VIP Software (SVIP-CDW-M), Version 12.2(37), RELEASE SOFTWARE (fc1)
> Copyright (c) 1986-2006 by cisco Systems, Inc.
> Compiled Thu 15-Jun-06 19:35 by pwade
> Image text-base: 0x60010930, data-base: 0x60460000
> 
> 
>         --Boot log end--
> 
> Slot 1:
>         Physical slot 1, ~physical slot 0xE, logical slot 1, CBus 0
>         Microcode Status 0x4
>         Master Enable, LED, WCS Loaded
>         Board is analyzed 
>         Pending I/O Status: None
>         EEPROM format version 1
>         VIP2 R5K controller, HW rev 2.02, board revision D0
>         Serial number: 17953867  Part number: 73-2167-05
>         Test history: 0x00        RMA number: 00-00-00
>         Flags: cisco 7000 board; 7500 compatible
>           
>         EEPROM contents (hex):
>           0x20: 01 1E 02 02 01 11 F4 4B 49 08 77 05 00 00 00 00
>           0x30: 68 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
>           
>         Slot database information:
>         Flags: 0x4      Insertion time: 0x2C3C (00:06:35 ago)
>           
>         Controller Memory Size: 128 MBytes DRAM, 4096 KBytes SRAM
>           
>         PA Bay 0 Information:
>                 Fast-Ethernet PA, 2 ports, 100BaseTX-ISL
>                 EEPROM format version 1
>                 HW rev 1.01, Board revision A0
>                 Serial number: 14076764  Part number: 73-2618-02 
>           
>         --Boot log begin--
>           
> Cisco Internetwork Operating System Software 
> IOS (tm) VIP Software (SVIP-CDW-M), Version 12.2(37), RELEASE SOFTWARE (fc1)
> Copyright (c) 1986-2006 by cisco Systems, Inc.
> Compiled Thu 15-Jun-06 19:35 by pwade
> Image text-base: 0x60010930, data-base: 0x60460000
>           
>           
>         --Boot log end--
>           
> Slot 3:   
>         EEPROM format version 1
>         Route/Switch Processor 4, HW rev 1.01, board revision D0
>         Serial number: 11123251  Part number: 73-1689-03
>         Test history: 0x00        RMA number: 00-00-00
>         Flags: cisco 7000 board; 7500 compatible
>           
>         EEPROM contents (hex):
>           0x20: 01 1A 01 01 00 A9 BA 33 49 06 99 03 00 00 00 00
>           0x30: 68 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
> Slot 31 (virtual):
>         EEPROM format version 1
>         Chassis Interface, HW rev 1.01, board revision C0
>         Serial number: 12266020  Part number: 73-1306-02
>         Test history: 0x00        RMA number: 00-00-00
>         Flags: cisco 7000 board
>           
>         EEPROM contents (hex):
>           0x20: 01 10 01 01 00 BB 2A 24 49 05 1A 02 00 00 00 00
>           0x30: 60 00 00 00 FF 00 00 00 00 00 00 00 00 00 00 00
> 
> 
> _______________________________________________________
> Skeeve Stevens, RHCE     Email: skeeve at skeeve.org
> Website: www.skeeve.org  - Telephone: (0414) 753 383
> Address: P.O Box 1035, Epping, NSW, 1710, Australia
> 
> eIntellego - skeeve at eintellego.net - www.eintellego.net
> _______________________________________________________
> I'm a groove licked love child king of the verse
> Si vis pacem, para bellum
> 
> 
> 
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/


More information about the cisco-nsp mailing list