[nsp] Serious bug in 12.1(20)EA1a 3550 EMI code

[Mark Boolootian]

We've run into a rather ugly bug in the 12.1(20)EA1a EMI image Cisco released
for the 3550 to fix the SNMP vulnerability.  The symptoms are packet loss 
(varying from 1% to 25%+) for traffic that hits the router processor.  The 
router CPU, memory, and interface stats show no indication of trouble while
packet loss is occurring.  The command 'show controllers cpu' provides the 
only clear sign that something is amiss:

router#show controllers cpu-interface 

  stp packets : 87 retrieved, 0 dropped
  ram access packets : 5703192 retrieved, 0 dropped
  routing protocol packets : 504326 retrieved, 0 dropped
  forwarding packets : 0 retrieved, 0 dropped
  routing packets : 5306728 retrieved, 345665 dropped

Note the drops on the 'routing packets' line.  

It appears to take several days for the symptoms to appear.  The first
two routers to fail here were the busiest 3550s on campus, and that took
about four days from boot.  Six days after boot, every 3550 we had
was showing this problem (about 15 of them).

I had Cisco engineers in the console of a failing box on Wednesday morning,
and they looked at as much as they could.  I *think* they were convinced
there is a bug, but it's now Friday afternoon and I've yet to hear anything
back.  It is hard to imagine this problem is unique to us, but this is Santa 
Cruz, so who knows...

Anyone out there rev to 12.1(20)EA1a?  Got problems?  The packet loss can
be low and hard to notice (load dependent, I think).  The easiest external
test is a fast pinger directed at the box.  From inside, 'show control cpu'.

thanks,
mb