[c-nsp] Spanning-Tree Loop (12.2.18SXF7)

Antonio Soares amsoares at netcabo.pt
Mon Nov 8 06:36:37 EST 2010 

Case sent to TAC. I will update the list if something interesting comes
along.


Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
amsoares at netcabo.pt

-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Antonio Soares
Sent: sexta-feira, 5 de Novembro de 2010 22:38
To: 'Tony Varriale'; 'cisco-nsp'
Subject: Re: [c-nsp] Spanning-Tree Loop (12.2.18SXF7)

Everything is resolved. As I mentioned, the 6500 recovered from this by
itself. The debuginfo file that was created as the same format as the
crashinfo files with lots of stack dumps, register dumps and so on. I only
have the "show proc" there but I can't figure out what is the buggy process.
I have these columns:

PID, Qty, PC, Runtime (ms), Invoked, uSecs, Stacks, TTY, Process

I don't see anything evident. What should I look for ?


Thanks

Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
amsoares at netcabo.pt

-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Tony Varriale
Sent: sexta-feira, 5 de Novembro de 2010 20:40
To: 'cisco-nsp'
Subject: Re: [c-nsp] Spanning-Tree Loop (12.2.18SXF7)

Could be.

What's the rest of the file and sh proc cpu say?  I'd find out what's eating

all that memory first.

Also, your original message stated something about collisions.  Resolved? 
Related?

tv
----- Original Message ----- 
From: "Antonio Soares" <amsoares at netcabo.pt>
To: "'Jared Mauch'" <jared at puck.nether.net>
Cc: "'cisco-nsp'" <cisco-nsp at puck.nether.net>
Sent: Friday, November 05, 2010 1:33 PM
Subject: Re: [c-nsp] Spanning-Tree Loop (12.2.18SXF7)


> Interesting, the switch did not crash but a debugfile was generated. And
> something I can read from that file:
>
> ++++++++++++++++++++++++++++++++++++++++
> Processor Memory: total 388120864, free 276632816, used 111488048
> IO Memory: total 67108864, free 22640, used 67086224
> (..)
> CPU utilization for five seconds: 100%/28%; one minute: 99%; five minutes:
> 99%
> ++++++++++++++++++++++++++++++++++++++++
>
> Memory leak ? It's the first time I see an equipment recovering from a
> situation like this without human intervention. It would be nice if the
> impact was not an STP loop...
>
>
>
> Regards,
>
> Antonio Soares, CCIE #18473 (R&S/SP)
> amsoares at netcabo.pt
>
>
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Antonio Soares
> Sent: sexta-feira, 5 de Novembro de 2010 17:15
> To: 'Jared Mauch'
> Cc: 'cisco-nsp'
> Subject: Re: [c-nsp] Spanning-Tree Loop (12.2.18SXF7)
>
> The 6500's uptime is 1 year and 16 weeks. The funny thing is that the 6500
> recovered by itself, no reload or sup switchover occurred. And I don't see
> anything wrong with the memory stats. Really strange. Sometimes we prefer 
> a
> crash :) This STP loop was active a few hours...
>
> I can't upgrade without knowing what is the bug. I will keep searching.
>
> Thanks.
>
> Regards,
>
> Antonio Soares, CCIE #18473 (R&S/SP)
> amsoares at netcabo.pt
>
>
> -----Original Message-----
> From: Jared Mauch [mailto:jared at puck.nether.net]
> Sent: sexta-feira, 5 de Novembro de 2010 17:00
> To: Antonio Soares
> Cc: 'cisco-nsp'
> Subject: Re: [c-nsp] Spanning-Tree Loop (12.2.18SXF7)
>
> There could have been any sort of a memory leak since SXF7 and now that
> caused you to see this.  How long was your device up?  I hate to say this,
> but I would go to recent software (eg: SXF15a/SXF16).
>
> - Jared
>
> On Nov 5, 2010, at 12:25 PM, Antonio Soares wrote:
>
>> Hello group,
>>
>> I'm troubleshooting a STP loop that seemed to be triggered by these
> errors:
>>
>> ++++++++++++++++++++++++++++++++
>> %PM_SCP-SP-3-LCP_FW_ABLC: Late collision message from module 1, port:029
>> %IPC-SP-5-WATERMARK: 15612 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>> %IPC-SP-5-WATERMARK: 20068 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>> %IPC-SP-5-WATERMARK: 22904 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>> %PM_SCP-SP-3-LCP_FW_ABLC: Late collision message from module 2, port:039
>> %IPC-SP-5-WATERMARK: 25710 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>> %IPC-SP-5-WATERMARK: 28510 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>> %PM_SCP-SP-3-LCP_FW_ABLC: Late collision message from module 2, port:039
>> %IPC-SP-5-WATERMARK: 30344 messages pending in xmt for the port
>> CHKPT:STANDBY SP(2080000.B) seat 2080000
>>
>> %SYS-SP-2-MALLOCFAIL: Memory allocation of 1768 bytes failed from
>> 0x4020DCC8, alignment 32
>> Pool: I/O  Free: 148112  Cause: Memory fragmentation
>> Alternate Pool: None  Free: 0  Cause: No Alternate pool
>>
>> -Process= "Spanning Tree", ipl= 2, pid= 126
>> -Traceback= 40280990 402826F8 4020DCD0 4020E064 402108DC 4020D0B8 
>> 40AAB1BC
>> 40AAB64C 404A3970 404C4F64 404C51A8 404C5460 404AAA58 404AAE40 40A9EE00
>> 40A9D71C
>> ++++++++++++++++++++++++++++++++
>>
>> These were seen on one of core's 6500. The 6500s are running 12.2.18SXF7.
>>
>> Fortunately the problem went away by itself. The last thing we want is a
> STP
>> loop in a network with several 6500s and 4500s, right ? :)
>>
>> Anyone has seen something like this ? I'm now hitting Bug Toolkit to see
> if
>> this was reported before.
>>
>>
>> Thanks.
>>
>> Regards,
>>
>> Antonio Soares, CCIE #18473 (R&S/SP)
>> amsoares at netcabo.pt
>>
>>
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/ 

_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

More information about the cisco-nsp mailing list