[c-nsp] Cisco 6509 reboots on its own... again...

krunal shah krun.shah at gmail.com
Tue Jul 6 08:23:53 EDT 2010


There must be two crashinfo files for SP and RP and show tech-support. You
need to collect it when you contact tech support.

TAC usually has decoders from their developer to decode hex values in
traceback.

-Traceback= 41183348 41180F04 40DADF40 40FFA1CC 40FFA4D8 40752F58 40752F44


Krunal


On Mon, Jul 5, 2010 at 5:36 AM, Youssef Bengelloun-Zahr <youssef at 720.fr>wrote:

> Hello,
>
> I have a c6509 with redundant SUP720-3BXL
> (s72033-advipservicesk9_wan-mz.122-33.SXH2a.bin) that's rebooted on its own
> this morning. FYI, the same router reboot 3 weeks ago unexpectedly !
>
> Here is a trunkated output of the crashfile info :
>
> Jun 11 06:48:29.377: %PFREDUN-SP-6-ACTIVE: Standby initializing for SSO
> mode
> Jun 11 06:48:29.377: %SYS-SP-3-LOGGER_FLUSHING: System pausing to ensure
> console debugging output.
> Jun 11 06:48:29.377: %PFREDUN-SP-6-ACTIVE: Standby initializing for SSO
> mode
> Jun 11 06:48:29.569: %SYS-SP-3-LOGGER_FLUSHED: System was paused for
> 00:00:00 to ensure console debugging output.
> Jun 11 06:48:41.952: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 11 06:49:39.434: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off
> for
> the fabric in slot 5.
> Jun 11 06:49:39.530: %FABRIC-SP-5-FABRIC_MODULE_BACKUP: The Switch Fabric
> Module in slot 5 became standby
> Jun 11 06:49:42.850: %DIAG-SP-6-RUN_COMPLETE: Module 5: Running Complete
> Diagnostics...
> Jun 11 06:49:44.819: %DIAG-SP-6-DIAG_OK: Module 5: Passed Online
> Diagnostics
> Jun 11 06:49:48.673: %OIR-SP-6-INSCARD: Card inserted in slot 5, interfaces
> are now online
> Jun 11 09:53:37.178: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 11 13:02:59.715: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 11 13:04:16.254: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 14 09:00:28.800: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 14 09:05:08.864: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 17 08:35:59.058: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 17 08:39:58.941: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> CMD: 'sh mls cef summary ' 11:31:24 UTC Thu Jun 17 2010
> CMD: 'exit' 11:31:25 UTC Thu Jun 17 2010
> CMD: 'sh mls cef statistics ' 11:32:01 UTC Thu Jun 17 2010
> CMD: 'sh mls cef maximum-routes ' 11:32:21 UTC Thu Jun 17 2010
> CMD: 'sh mls cef rpf ' 11:33:07 UTC Thu Jun 17 2010
> CMD: 'show mls acl inconsistency' 12:18:44 UTC Thu Jun 17 2010
> Jun 21 08:14:58.161: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 22 08:15:53.784: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 22 11:56:07.044: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 22 11:58:40.637: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 23 11:01:20.484: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> Jun 23 12:31:21.556: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
> CMD: 'sh mls cef ' 21:30:10 UTC Sun Jun 27 2010
> CMD: 'sh mls cef tcam hit ' 21:31:52 UTC Sun Jun 27 2010
> Jun 29 11:51:04.876: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
> configuration to the standby Router.
>
> %Software-forced reload
>
>
>  06:23:49 UTC Mon Jul 5 2010: Breakpoint exception, CPU signal 23, PC =
> 0x41183348
>
>
>
> --------------------------------------------------------------------
>   Possible software fault. Upon reccurence, please collect
>   crashinfo, "show tech" and contact Cisco Technical Support.
> --------------------------------------------------------------------
>
>
> -Traceback= 41183348 41180F04 40DADF40 40FFA1CC 40FFA4D8 40752F58 40752F44
> $0 : 00000000, AT : 1E020000, v0 : 43720000, v1 : 00000043
> a0 : 447135B0, a1 : 00000043, a2 : 00000009, a3 : 00000000
> t0 : 44C7494C, t1 : 44C74948, t2 : 44C74944, t3 : 44C74940
> t4 : 44C7493C, t5 : 44C74938, t6 : 44C74934, t7 : 44C74930
> s0 : 00000000, s1 : 41DF0000, s2 : 08FA84B0, s3 : 44C74AC0
> s4 : 44C74AB8, s5 : 00000000, s6 : 00000000, s7 : 00000000
> t8 : 44C7499C, t9 : 00000000, k0 : 470E1200, k1 : 40798CE0
> gp : 41E591E0, sp : 44C74A20, s8 : 00000000, ra : 41180F04
> EPC  : 41183348, ErrorEPC : 40947F88, SREG     : 3400FF03
> MDLO : 333333E8, MDHI     : 000002D3, BadVaddr : 00000000
> DATA_START : 0x41C420A0
> Cause 00000024 (Code 0x9): Breakpoint exception
>
>
> ========= Start of Crashinfo Collection (06:23:49 UTC Mon Jul 5 2010)
> ==========
> For image:
> Cisco IOS Software, s72033_sp Software (s72033_sp-ADVIPSERVICESK9_WAN-M),
> Version 12.2(33)SXH2a, RELEASE SOFTWARE (fc2)
> Technical Support: http://www.cisco.com/techsupport
> Copyright (c) 1986-2008 by Cisco Systems, Inc.
> Compiled Fri 25-Apr-08 08:20 by prod_rel_team
>
>
> ========= Show Alignment
> =======================================================
>
>
> No alignment data has been recorded.
>
> No spurious memory references have been recorded.
>
>
> ========= Additional Subsystem Crashinfo
> =======================================
>
> --------- show redundancy --------
>
> Switchovers this system has experienced          : 1
> Last switchover reason                           : Active crashed.
> Uptime since this supervisor switched to active  : 3 weeks, 2 days, 23
> hours, 39 minutes
> Total system uptime from reload                  : 13 weeks, 2 days, 19
> hours, 5 minutes
>
> Standby is ready to take over
>
>
> ========= Data Inconsistency Errors =========
>
> No data inconsistency errors have been recorded.
>
>
> --------- show eobc --------
>
> Interface information:
>    Interface EOBC0/0 (idb = 0x44A888B8)
>    Hardware is Mistral EOBC (revision 5)
>    Address is 0000.0600.0000 (bia 0000.0600.0000)
>    Encap size         = 14         hardware status  = 0x210840
>    IDB type           = 18         IDB state        = 4
>    Encap type         = 0x1        Span encap size  = 0
>    Error threshold    = 5000       Error count      = 0
>
> Counters:
>    rxring             = 0x921DD00  rx ring entries       = 512
>    rx_head            = 139        rx_tail               = 0
>    inputs             = 150953935  rx_cumbytes           = 14190294763
>    hw inputs          = 0          hw rx_cumbytes        = 0
>    rx rate (bits/sec) = 41000      rx rate (packets/sec) = 53
>    rx_buf_unavail     = 0          *rx input drops        = 4397*
>    input broadcast    = 150        input resource        = 6815119
>    input error        = 0          input giants          = 0
>    *input crc          = 4397*       rx illegal length     = 0
>    rxr eobc shadow    = 0x50C438F0 txr eobc shadow       = 0x44B94BCC
>
>    txring             = 0x921FD40  tx ring entries       = 0x200
>    tx_head            = 297        tx_tail               = 297
>    outputs            = 156727081  tx_cumbytes           = 26897233358
>    hw outputs         = 0          hw tx_cumbytes        = 0
>    tx rate (bits/sec) = 84000      tx rate (packets/sec) = 56
>    tx_retry_error     = 2          tx_retry_count        = 276218
>    tx_process_stopped = 0          tx total drops        = 0
>
> Mistral Registers
>    soft_reset_cfg     = 0x000000   dma_buffer_size_reg   = 0x000000
>    int_mask_hi        = 0x000076   int_mask_lo           = 0x7001AD8
>    rxdscp_cnt         = 425        txdscp_cnt            = 0
>    rxwork_dscp        = 0xEB20     txwork_dscp           = 0x688
>    mistral_eobc_ds    = 0x44A897C4 mistral_dma_register  = 0x30000000
>    mistral_glbl_reg   = 0x10020000
>
> Misc. Global Registers:
>    global_cfg         = 0x20       mis_init_sts          = 0xF
>    dimm_parm_cfg_hi   = 0x000003F6 dimm_parm_cfg_lo      = 0x42040F5A
>    tm_init_size_cfg   = 0x8000
>
>
> Here is the output of a show version :
>
> Cisco IOS Software, s72033_rp Software (s72033_rp-ADVIPSERVICESK9_WAN-M),
> Version 12.2(33)SXH2a, RELEASE SOFTWARE (fc2)
> Technical Support: http://www.cisco.com/techsupport
> Copyright (c) 1986-2008 by Cisco Systems, Inc.
> Compiled Fri 25-Apr-08 08:07 by prod_rel_team
>
> ROM: System Bootstrap, Version 12.2(17r)SX5, RELEASE SOFTWARE (fc1)
>
>  BB1.IX1 uptime is 13 weeks, 2 days, 22 hours, 1 minute
> Uptime for this control processor is 3 weeks, 3 days, 2 hours, 26 minutes
> Time since BB1.IX1 switched to active is 2 hours, 49 minutes
> *System returned to ROM by Stateful Switchover at 07:42:25 UTC Wed May 20
> 2009 (SP by reload)
> *System restarted at 06:48:20 UTC Fri Jun 11 2010
> System image file is
> "bootdisk:s72033-advipservicesk9_wan-mz.122-33.SXH2a.bin"
>
>
> This product contains cryptographic features and is subject to United
> States and local country laws governing import, export, transfer and
> use. Delivery of Cisco cryptographic products does not imply
> third-party authority to import, export, distribute or use encryption.
> Importers, exporters, distributors and users are responsible for
> compliance with U.S. and local country laws. By using this product you
> agree to comply with applicable laws and regulations. If you are unable
> to comply with U.S. and local laws, return this product immediately.
>
> A summary of U.S. laws governing Cisco cryptographic products may be found
> at:
> http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
>
> If you require further assistance please contact us by sending email to
> export at cisco.com.
>
> cisco WS-C6509 (R7000) processor (revision 2.0) with 983008K/65536K bytes
> of
> memory.
> Processor board ID SCA043001KB
> SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
> Last reset from s/w reset
> 29 Virtual Ethernet interfaces
> 96 FastEthernet interfaces
> 124 Gigabit Ethernet interfaces
> 1917K bytes of non-volatile configuration memory.
> 8192K bytes of packet buffer memory.
>
> 65536K bytes of Flash internal SIMM (Sector size 512K).
> Configuration register is 0x2102
>
>
> I have noticed some EOBC input drops due to CRC. Would this be due to a
> chassis default ? It's been running fine for more than two years now.
>
> I am running the IOS bug toolkit looking for a possible match with my case.
>
> Thanks.
>
> Cheers.
>
> Y.
>
> --
> Youssef BENGELLOUN-ZAHR ………………………………………………
> Ingénieur Réseaux et Télécoms
>
>
> Technopole de l'Aube  en Champagne - BP 601 - 10901 TROYES  Cedex 9
> Agence Paris : 6, rue Charles Floquet - 92120 MONTROUGE
> Tel                 +33 (0) 825 000 720
> Tel. direct      +33 (0) 1 77 35 59 14
> Tel. portable  +33 (0) 6 22 42 63 80
> Email            ybz at 720.fr
> ……………………………………………………………………………….....www.720.fr
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>


More information about the cisco-nsp mailing list