[c-nsp] GSR with STM-4 and packet loss, weird LC cpu utilization etc.

[Primoz Jeroncic]

Hi

I have some weird problems on one of our STM-4 links. From time to time
about 5-7 packets get lost on this link. It doesn't happen every
X minutes/hours but randomly, and always with 5-7 packets in row. There's
no CRC, input or output errors on any of ports, and also our carrier
is trying to tell me, everything is fine on their link. Overall, packet
loss is about 0.2 to 0.5% during longer period (1 day or more).

So I went to check with routers. On both sides we are running GSR with
4 Port Packet Over SONET OC-12c/STM-4 SM line cards. So when I went to
check, I found out, on one side, LC processor has 50-60% utilization,
while on other it has 3-4% utilization (on both sides it's constantly
at such percentage). I still don't think this packet loss would be
connected to "high" (50-60%) utilization, since if it would be, I would
have this packet loss contantly, not just few packets every few hours.
And it's also not fully utilized port either. Average throughput is
about 400Mbps, while peaks never go over 500Mbps.

So now I have 2 questions. First... does anyone have any idea what could
be reason for such packet loss?

And second... And idea why one LC would be at 3-4%, while other would
be at 50-60%? Both LCs are same, both routers are same, and even config
on both LCs is same... see below:

Diags are same for both, except one is in slot 1, while other is in slot 6:

SLOT 1  (RP/LC 1 ): 4 Port Packet Over SONET OC-12c/STM-4 Single Mode
  FRU:  Linecard/Module: 4OC12/POS-IR-SC-B=
        Route Memory: MEM-GRP/LC-256=
        Packet Memory: MEM-LC1-PKT-256=
  L3 Engine: 2 - Backbone OC48 (2.5 Gbps)
  DRAM size: 268435456 bytes
  FrFab SDRAM size: 134217728 bytes, SDRAM pagesize: 8192 bytes
  ToFab SDRAM size: 134217728 bytes, SDRAM pagesize: 8192 bytes


Config on port:
Location 1 (utilization 50-60%):
interface POS6/0
 bandwidth 622080
 ip address x.x.x.1 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip directed-broadcast
 encapsulation ppp
 ip route-cache flow sampled
 load-interval 30
 tag-switching mtu 1546
 tag-switching ip
 crc 16
 clock source internal
 pos framing sdh
 pos flag s1s0 2

Location 2 (utilization 3-4%)
interface POS1/0
 bandwidth 622080
 ip address x.x.x.2 255.255.255.252
 no ip redirects
 no ip unreachables
 no ip directed-broadcast
 encapsulation ppp
 ip route-cache flow sampled
 load-interval 30
 tag-switching mtu 1546
 tag-switching ip
 crc 16
 clock source internal
 pos framing sdh
 pos flag s1s0 2

Only difference betwen boxes is IOS. On Location 1, we are running
service provider ios 12.0(30)S5, while on Location 2 we are running
12.0(26)S5.

And help, tips, suggestions are more then welcome :)

Thanks for help.

Have fun,
Primoz Jeroncic
Support - IP Connectivity & Routing
-------------------------------------------------------------------
Softnet d.o.o.  tel:  +386 1 562 31 40   |
Borovec 2       fax:  +386 1 562 18 55   |       1 + 1 = 3
1236 Trzin      primoz(at)softnet.si     | for larger values of 1
Slovenija       http://flea.softnet.si/
-------------------------------------------------------------------