[c-nsp] low throughput of 10GbE line

[Jirí Procházka]

hi,

argh, the module model used is WS-X6708-10GE, sorry for confusion.

sitel-edge-new#show module
Mod Ports Card Type                              Model              Serial 
No.
--- ----- -------------------------------------- ------------------ -----------
  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP 
SAL1126T1EV
  2   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX 
SAL10019PF0
  3   48  48 port 10/100/1000mb EtherModule      WS-X6148-GE-TX 
SAL09486QB1
  4    8  CEF720 8 port 10GE with DFC            WS-X6708-10GE 
SAL1221R8JP
  5    2  Route Switch Processor 720 (Active)    RSP720-3CXL-GE 
JAE1127P1GQ

Traffic coming to this interface is not much bursty, its dozen of relatively 
slow streams. Other interfaces of this module are capable to handle constant 
10gbps load.

Anyway, I've found another strange behaviour, which maybe has to do with 
this issue.. Another server is connected directly to the 7600 (to the 
WS-X6748-GE-TX) and latency to this destination is bad as well.

IP of the server is let's say at VLAN100. Gateway for server's subnet on 
VLAN100 is at 7600 (no other L2, nor L3 hop)When I try to ping IP from this 
subnet without specifying source VLAN, ping is high, with packetloss.  _BUT_ 
when I specify some other VLAN as source, ping and packetloss is absolutely 
flawless. __BUT__  it's behaves extremly random, when i specify for example 
source VLAN 10,50,120, it seems like the problem never existed. Then specify 
source VLAN for example 41, and viola, ping is huge, packetloss as well.

sitel-edge-new#show ip route x.x.x.x
Routing entry for x.x.x.x/28
  Known via "connected", distance 0, metric 0 (connected, via interface)
[..]
  * directly connected, via Vlan116

sitel-edge-new#ping x.x.x.x
Sending 5, 100-byte ICMP Echos to x.x.x.x, timeout is 2 seconds:
!!!.!
Success rate is 75 percent (4/5), round-trip min/avg/max = 96/110/150 ms

sitel-edge-new#ping  x.x.x.x source vlan 10
Sending 5, 100-byte ICMP Echos to  x.x.x.x, timeout is 2 seconds:
Packet sent with a source address of y.y.y.y
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms


Any idea, how is this possible?






----- Original Message ----- 
From: "Peter Rathlev" <peter at rathlev.dk>
To: "Jirí Procházka" <jiri.prochazka at superhosting.cz>
Cc: <cisco-nsp at puck.nether.net>
Sent: Monday, March 15, 2010 7:42 PM
Subject: Re: [c-nsp] low throughput of 10GbE line


> On Mon, 2010-03-15 at 17:26 +0100, Jirí Procházka wrote:
>> Catalyst 7606 ( IOS 12.2(33)SRD4) with WS-F6700-DFC3CXL module
>> connected with C3750-E (IOS 12.2(40)SE) over TenGigabit Xenpaks. Its
>> one of our backbone lines, transferring only VLAN-201.
>
>>From the use of XenPack I assume the module is a WS-X6704-10GE.
>
>> sitel-edge-new#show int te4/8
>> TenGigabitEthernet4/8 is up, line protocol is up (connected)
> [...]
>>   Input queue: 0/4096/96151289/0 (size/max/drops/flushes); Total
>> output drops: 0
>>   Queueing strategy: fifo
>>   Output queue: 0/4096 (size/max)
>>   30 second input rate 4942942000 bits/sec, 410813 packets/sec
>>   30 second output rate 143433000 bits/sec, 241176 packets/sec
>>      34434311308 packets input, 51018973201607 bytes, 0 no buffer
>>      Received 21472 broadcasts (17607 multicasts)
>>      0 runts, 0 giants, 0 throttles
>>      0 input errors, 0 CRC, 0 frame, 96151289 overrun, 0 ignored
>>      0 watchdog, 0 multicast, 0 pause input
>>      0 input packets with dribble condition detected
>>      19623750753 packets output, 3094225873410 bytes, 0 underruns
>>      0 output errors, 0 collisions, 0 interface resets
>>      0 babbles, 0 late collision, 0 deferred
>>      0 lost carrier, 0 no carrier, 0 pause output
>>      0 output buffer failures, 0 output buffers swapped out
>
> Input overrun would mean that the card is unable to receive the traffic
> it's served. The 6704 only has 2 MB input buffers, so there isn't much
> room.
>
> You would have better luck with a WS-X6708-10G. It has 109 MB input
> buffers. They come with a "free" DFC3C by the way.
>
> Other than increasing the input buffer size (a hardware parameter)
> there's nothing much to do about it, apart from making sure the traffic
> is extremely non-bursty.
>
> -- 
> Peter
>
>