[c-nsp] 3750: SNMP-3-INPUT_QFULL_ERR, ssh session dies, show tech support fails, switch stack crashes on reload

[Darren O'Connor]

Never seen it myself, but googling around brings up a few things.

Did this recently start? Any other switch on the same code having the same issues or not? Generally if five different devices all start having the same issue an external issue is to blame. Maybe your SNMP server is sending a particular packet that this IOS code doesn't like?

Have you tried restarting SNMP itself on the switch?

Thanks
Darren
http://www.mellowd.co.uk/ccie



> Date: Mon, 5 May 2014 16:47:02 +0200
> From: sebastian.beutel at rus.uni-stuttgart.de
> To: cisco-nsp at puck.nether.net
> Subject: [c-nsp] 3750: SNMP-3-INPUT_QFULL_ERR, ssh session dies, show tech support fails, switch stack crashes on reload
> 
> Hi List,
> 
>     we are running a lot of (three-digit) 3750 stacks that are showing a
> strange phenomenon by and by. Meanwhile we have five affected switches that
> had an uptime of almost two years and an old IOS 12.2(44)SE in common.
>     The first indication so far is, that we see "%SNMP-3-INPUT_QFULL_ERR" in
> the syslog for no reason (the affected switch receives as much snmp request
> as any other switch on our network). If we ssh to the affected switch and do
> a "show interfaces status" it shows a couple of interfaces of the first
> switch, then the ssh session crashes. The same with "show etherchannel
> summary". If we reconnect again a "show users" lists the broken connection
> but a "clear line vty" does not reset it. If we do a "show tech-support |
> redirect tftp:..." the ssh session also crashes. The file on the tftserver
> ends with the interface at which "sho int statu" breaks.
>     At this stage the stack still seems to forward traffic but if we do a
> reload in a maintenance window things get worse: The switch that has been
> the stack master instantly crashes and does not recover. Forwarding stops
> and the management interface does not come back. Even the console is
> unusable. The only thing to remedy the situation is to unplug the mains
> cables. After that the switch comes back as though nothing had happened.
> 
> We understand that we need to update the IOS (and we did on some of them).
> The problem though is that the reload that is associated with the update may
> also cause outages. We suspect that stacks are unstable long before they
> attract attention due do the snmp error messages. It would be a great help
> to predict whether a switch will survive an update or otherwise would need
> it's mains supply interrupted.
>  
> 
> Thanks in advance,
>                             Sebastian.
> 
> -- 
> Dipl.-Ing. Sebastian Beutel                              tel: +49-711-685-64538
> Rechenzentrum Universitaet Stuttgart        http://www.rus.uni-stuttgart.de/nks
> Netze und Kommunikationssysteme              Allmandring 30A, D-70550 Stuttgart
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/