[c-nsp] C720X NPE-G1 - interface errors + router freezes

[Geyer, Nick]

I have seen issues like you mention creep up on NPE-G1's that have been
in service for a while. It all starts with a few input errors here and
there and progressively gets worse.

Reseating the NPE seems to clear up all the issues and it starts
chugging along happily again. Possibly worth a try on one of your
routers to see if it makes a difference.

Nick

-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of The Hawk
Sent: Friday, 5 September 2008 11:12 AM
To: cisco-nsp at puck.nether.net
Subject: [c-nsp] C720X NPE-G1 - interface errors + router freezes



Forgot to add a subject line last time...Sorry for the double
post...some additional info has also been added inline.

Hello All,

We're noticing a large number of input errors on multiple GigE
interfaces on various C720X NPE-G1 routers.  These input errors match
exactly with the number of overruns on those interfaces. I wouldn't be
so concerned about the errors if other strange things were not happening
on the routers. (such as frequent lockups and frequent eigrp drops).  At
this point they go hand in hand however we need to pinpoint the origin
of the problems and it hasn't been easy.

These are a couple strange things that we've noticed:

1.        this problem happens on more than 1 router (similar
configuration on all routers)
2.       all routers affected began exhibiting these errors roughly
around the same time (same early morning)
3.       All routers are G1 and ran the same IOS code (12.4.12)
4.       One of the routers was rebooted and upgraded to a different
code however the problem still persists (12.4.21)
5.       we suspect it directly affects the functionality of the router
as we see random lockups of the units (which may be the cause of the
increased errors not the result of)
6.       All routers are connected to redundant switches and
cabling/switches have been ruled out.

Below is a sample output of the show interface on one of the routers:

GigabitEthernet0/1 is up, line protocol is up
  Hardware is BCM1250 Internal MAC, address is 00b0.c2ee.b81b (bia
00b0.c2ee.b81b)
  Description: XXXXXXXX
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 2/255
  Encapsulation 802.1Q Virtual LAN, Vlan ID  1., loopback not set
  Keepalive set (10 sec)
  Full-duplex, 1000Mb/s, media type is RJ45
  output flow-control is XON, input flow-control is XON
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of 'show interface' counters never
  Input queue: 1/75/499528/97 (size/max/drops/flushes); Total output
drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 8539000 bits/sec, 10996 packets/sec
  5 minute output rate 1905000 bits/sec, 807 packets/sec
     139728650 packets input, 10628478461 bytes, 92 no buffer
     Received 487174 broadcasts, 0 runts, 0 giants, 0 throttles
     171399 input errors, 0 CRC, 0 frame, 171399 overrun, 0 ignored
     0 watchdog, 1904930 multicast, 0 pause input
     0 input packets with dribble condition detected
     8125773 packets output, 2521690484 bytes, 0 underruns
     2 output errors, 0 collisions, 1 interface resets
     3842 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     2 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

Does anyone have any idea what might be going on?

I initially suspected an IOS bug (since all devices were affected)
however after the upgrade the problem still persists.

Any help is appreciated.
Adrian
_________________________________________________________________

_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/