[nsp] Serious bug in 12.1(20)EA1a 3550 EMI code

[Yuval Ben-Ari]

not sure how related it is but I have open bug CSCee13768 for Tacacs
connections stuck in CLOSEWAIT state.
this is a problem I think started at 12.1(19)EA1
As far as I noticed if you are using tacacs authentication and SSH to
the box, everytime you SSH to it it leaves another connection in
CLOSEWAIT until the TCP stack is filled and then all the symtoms you
mentioned appear (packet loss, hard to connect to the box, etc .....)
you can easily see if that's the case using "show tcp brief"

Yuval

> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net 
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Mark 
> Boolootian
> Sent: Friday, April 30, 2004 23:19
> To: cisco-nsp at puck.nether.net
> Subject: [nsp] Serious bug in 12.1(20)EA1a 3550 EMI code
> 
> 
> 
> We've run into a rather ugly bug in the 12.1(20)EA1a EMI 
> image Cisco released
> for the 3550 to fix the SNMP vulnerability.  The symptoms are 
> packet loss 
> (varying from 1% to 25%+) for traffic that hits the router 
> processor.  The 
> router CPU, memory, and interface stats show no indication of 
> trouble while
> packet loss is occurring.  The command 'show controllers cpu' 
> provides the 
> only clear sign that something is amiss:
> 
> router#show controllers cpu-interface 
> 
>   stp packets : 87 retrieved, 0 dropped
>   ram access packets : 5703192 retrieved, 0 dropped
>   routing protocol packets : 504326 retrieved, 0 dropped
>   forwarding packets : 0 retrieved, 0 dropped
>   routing packets : 5306728 retrieved, 345665 dropped
> 
> Note the drops on the 'routing packets' line.  
> 
> It appears to take several days for the symptoms to appear.  The first
> two routers to fail here were the busiest 3550s on campus, 
> and that took
> about four days from boot.  Six days after boot, every 3550 we had
> was showing this problem (about 15 of them).
> 
> I had Cisco engineers in the console of a failing box on 
> Wednesday morning,
> and they looked at as much as they could.  I *think* they 
> were convinced
> there is a bug, but it's now Friday afternoon and I've yet to 
> hear anything
> back.  It is hard to imagine this problem is unique to us, 
> but this is Santa 
> Cruz, so who knows...
> 
> Anyone out there rev to 12.1(20)EA1a?  Got problems?  The 
> packet loss can
> be low and hard to notice (load dependent, I think).  The 
> easiest external
> test is a fast pinger directed at the box.  From inside, 
> 'show control cpu'.
> 
> thanks,
> mb
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>