[c-nsp] c3570 stack flapping

[Tristan Gulyas]

Hi,

We have an installation of many (1500+) 3750s in production and have seen 
this.

Some of the causes have been
* Faulty stack ports.  We are seeing an increasing amount of these failures 
on our network
* Faulty stack cables(!)
* Humidity - sometimes we notice on humid days that we see stacks randomly 
flap!  I suspect this has to do more with the cable - but this has been a 
consistent trend.  We use the APC environmental monotoring units for keeping 
track of humidity.  Hot humid air being rapidly cooled will cause 
condensation INSIDE the stack cable!!!  Nasty, nasty stuff.

Sometimes on occasion, we will see a stack flap as a once-off every few 
days.

Our steps to troubleshoot and resolve these issues are:
* Reconnect stack cable
* Replace stack cable

Alternatively if the link doesn't come up again, reload the stack...

* Replace switch - this is the tough one because we don't always know which 
stack member has the suspect port.

Let me know how you go.

Tristan.

----- Original Message ----- 
From: "Wyatt Mattias Ishmael Jovial Gyllenvarg" <wyatt.eliasson at gmail.com>
To: <cisco-nsp at puck.nether.net>
Sent: Friday, March 28, 2008 9:21 PM
Subject: [c-nsp] c3570 stack flapping


> Hi All
>
> I have a new stack of 2x 3750-12s that was recently stacked.
>
> We just noticed packet loss there and as part of the troubleshooting
> we powered of the second switch as the below was flooding the log. (no
> connections in it yet)
>
> *Nov 21 20:57:52.834: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 1
> Switch 2 has changed to state DOWN
> *Nov 21 20:57:53.589: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state DOWN
> *Nov 21 20:57:54.587: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state UP
> *Nov 21 20:57:54.587: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state DOWN
> *Nov 21 20:57:58.622: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state UP
> *Nov 21 20:57:59.629: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state DOWN
> *Nov 21 20:58:01.642: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state UP
> *Nov 21 20:58:02.900: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 1
> Switch 2 has changed to state UP
> *Nov 21 20:58:03.655: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 2
> Switch 1 has changed to state DOWN
> *Nov 21 20:58:03.907: %STACKMGR-6-STACK_LINK_CHANGE: Stack Port 1
> Switch 2 has changed to state DOWN
>
> This stopped the packet loss.
>
> As this is in production we are not able too perform and more
> troubleshooting at the time.
>
> Has anyone seen this before?
>
> Do I have a bad stack cable or a bad stack interface?
> Or perhaps a missmatch in ios version (cant double check that they are
> the same now) or firmware or hardware?
>
> Best regards
> Mattias Gyllenvarg
> Skycom AB
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>