[c-nsp] strange vip crash

[Rodney Dunn]

It's memory corruption on the VIP.

Almost impossible to troubleshoot without
the crashinfo from the VIP and you said
it didn't generate one. :(

Try and determine if there was any changes
around that VIP about the time it reloaded.

Rodney

On Wed, Jan 26, 2005 at 11:35:18PM -0500, Jon Lewis wrote:
> I just had an odd vip crash where no crashinfo was written.  All that was
> logged is
> 
> Jan 27 03:50:49: %VIP2 R5K-3-MSG: slot9 VIP-3-SVIP_RELOAD: SVIP Reload is
> called.
> Jan 27 03:50:49: %VIP2 R5K-3-MSG: slot9 VIP-3-SYSTEM_EXCEPTION: VIP System
> Exception occurred sig=20, code=0xA40D9C81, context=0x60BA1E84
> Jan 27 03:50:50: %DBUS-3-DBUSINTERRSWSET: Slot 9, Internal Error due to
> VIP crash
> Jan 27 03:51:22: %SYS-3-CPUHOG: Task is running for (2000)msecs, more than
> (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 40446C54 404DE738
> Jan 27 03:51:24: %SYS-3-CPUHOG: Task is running for (4000)msecs, more than
> (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 40446C5C 404DE738
> Jan 27 03:51:26: %SYS-3-CPUHOG: Task is running for (6000)msecs, more than
> (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 404DE738 404DF754 404F6D68 404E82AC
> Jan 27 03:51:28: %SYS-3-CPUHOG: Task is running for (8000)msecs, more than
> (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 40446CC4 404DE738 404DF754 404F6D68 404E82AC
> Jan 27 03:51:30: %SYS-3-CPUHOG: Task is running for (10000)msecs, more
> than (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 40446C9C 404DE738 404DF754 404F6D68 404E82AC
> Jan 27 03:51:32: %SYS-3-CPUHOG: Task is running for (12000)msecs, more
> than (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 404DE730 404DF754 404F6D68 404E82AC
> Jan 27 03:51:34: %SYS-3-CPUHOG: Task is running for (14000)msecs, more
> than (2000)msecs (5/4),process = OIR Handler.
> Jan 27 03:51:34: %SYS-3-CPUHOG: Task is running for (14000)msecs, more
> than (2000)msecs (5/4),process = OIR Handler.
> -Traceback= 4046BE50 4046BCA8 404F6DEC 404E82AC
> Jan 27 03:51:35: %HA-5-SYNC_NOTICE: OIR sync started.
> Jan 27 03:51:35: %HA-5-SYNC_NOTICE: OIR sync completed.
> 
> in show diag 9, I have:
>         1 crash since restart.
>         Last crash context (Jan 26 2005 22:50:49):
>         $0 : 00000000, AT : 60AF0000, v0 : 00000001, v1 : AB1234CD
>         a0 : 65355420, a1 : 62790DD0, a2 : 60B461A8, a3 : 00000001
>         t0 : 60C2FE40, t1 : 3400FF01, t2 : 3400C100, t3 : FFFF00FF
>         t4 : 60125DF0, t5 : 000000F8, t6 : 00000000, t7 : 447580D8
>         s0 : 638D9C90, s1 : EF430000, s2 : 00000032, s3 : 638E7380
>         s4 : 65355540, s5 : 65355420, s6 : 15A3C78B, s7 : 60510000
>         t8 : 61830EF8, t9 : 601163E0, k0 : 60DFED00, k1 : 00000200
>         gp : 60AF6120, sp : 60C60180, s8 : 60BA0000, ra : 601039FC
>         EPC  : 60103AB4, ErrorEPC : 601039DC, SREG     : 3400FF05
>         MDLO : 33054EC6, MDHI     : EFAB77C1, BadVaddr : DFFFFFFF
>         Cause 00000000 (Code 0x0): Interrupt exception
>         Traceback= 0x60103AB4 0x60103C2C 0x6010E5E0
> 
> Any ideas where to look for the cause?  i.e. bad DRAM, bad SRAM, bad IOS,
> sunspots?  Typically, when I've seen a vip crash, I get a crashinfo, and
> it points fingers at particular parts of the system.
> 
> IOS is rsp-k91pv-mz.122-18.S6.bin.
> 
> ----------------------------------------------------------------------
>  Jon Lewis                   |  I route
>  Senior Network Engineer     |  therefore you are
>  Atlantic Net                |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/