[c-nsp] Overruns - GSR Engine2 (3GE-GBIC-SC)

[David Freedman]

Am seeing incrementing overruns corresponding to the Gigmac rfifo_full
counter incrementing on this card:

# sh int g4/0 | in overr
    5519 input errors, 0 CRC, 0 frame, 5519 overrun, 0 ignored


# sh int g4/0 | in overr
    5521 input errors, 0 CRC, 0 frame, 5521 overrun, 0 ignored

#execute-on slot 4 show controllers gig 0 | in fifo_full
========= Line Card (Slot 4) =========
   0 risl, 0 riq, 12947 rdrop, 0 rsupp, 0 rinvalid_encap, 12947 rfifo_full

#execute-on slot 4 show controllers gig 0 | in fifo_full
========= Line Card (Slot 4) =========
   0 risl, 0 riq, 12953 rdrop, 0 rsupp, 0 rinvalid_encap, 12953 rfifo_full

Linecard CPU is not high:

#execute-on slot 4 show proc cpu | exc 0.00
========= Line Card (Slot 4) =========

CPU utilization for five seconds: 38%/0%; one minute: 19%; five minutes: 17%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
  53    25776564  10964270       2350  0.55%  0.75%  0.75%   0 CEF
process
  63    15474780  42493646        364  0.15%  0.24%  0.23%   0 Queue Mgr

  80  1174512036   6109315     192253 36.55% 16.26% 15.51%   0 TAG Stats
Backgr


Traffic does not seem excessive on the port in question nor have I found
any evidence of microbursts:

#sh int g4/0
GigabitEthernet4/0 is up, line protocol is up
  Hardware is GigMac 3 Port GigabitEthernet, address is 0005.5ff8.c954
(bia 0005.5ff8.c954)
  Internet address is 10.0.0.1/30
  MTU 1600 bytes, BW 1000000 Kbit, DLY 10 usec, rely 255/255, load 58/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full Duplex, 1000Mbps, link type is force-up, media type is LX
  output flow-control is unsupported, input flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 2d01h
  Queueing strategy: fifo
  Output queue 0/40, 0 drops; input queue 0/75, 0 drops
  5 minute input rate 204130000 bits/sec, 57532 packets/sec
  5 minute output rate 230560000 bits/sec, 46316 packets/sec
     7492692380 packets input, 3630841835600 bytes, 0 no buffer
     Received 42 broadcasts, 0 runts, 8629393 giants, 0 throttles
     5584 input errors, 0 CRC, 0 frame, 5584 overrun, 0 ignored
     0 watchdog, 30 multicast, 0 pause input
     5459460767 packets output, 3540440430407 bytes, 0 underruns
     Transmitted 0 broadcasts
     0 output errors, 0 collisions, 0 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out



Other ports in this card doing similar amounts of traffic are also
getting these in proportion to the traffic level :

#sh int g4/1
GigabitEthernet4/1 is up, line protocol is up
  Hardware is GigMac 3 Port GigabitEthernet, address is 0005.5ff8.c955
(bia 0005.5ff8.c955)
  Internet address is 192.168.1.1/30
  MTU 2450 bytes, BW 1000000 Kbit, DLY 10 usec, rely 255/255, load 47/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full Duplex, 1000Mbps, link type is autonegotiation, media type is LX
  output flow-control is unsupported, input flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 9w1d
  Queueing strategy: fifo
  Output queue 0/40, 0 drops; input queue 0/75, 146013 drops
  30 second input rate 85938000 bits/sec, 24296 packets/sec
  30 second output rate 185108000 bits/sec, 50553 packets/sec
     100929774614 packets input, 44365470024613 bytes, 2 no buffer
     Received 3 broadcasts, 0 runts, 3019035119 giants, 0 throttles
     92 input errors, 0 CRC, 0 frame, 92 overrun, 0 ignored
     0 watchdog, 4214445 multicast, 0 pause input
     152348554046 packets output, 61084338748442 bytes, 0 underruns
     Transmitted 8 broadcasts
     0 output errors, 0 collisions, 6 interface resets
     0 babbles, 0 late collision, 0 deferred
     3 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out


So is this a limitation of the PHY? Checking the specs for the gigmac
PMC7160 (which I think this is?) it should be able to do around
1.94Gb/Sec @ 536KFps so what is causing the overruns?

I found a report in bug CSCse98594 of broadcast frames interspersed with
<= 64b packets causing this condition in tetra cards (E3) but would
assume the PHY is different.

Also worth to note, acl is being done in PSA and virtually no process
switching (other than CEF recv/punt) is being done.

Running 12.0(32)S8 on 12000 GRP-B, bugtool does not turn up anything
wonderful.



Has anybody seen this before?