[c-nsp] 2960S drops/packet loss

Thu Dec 22 10:02:13 EST 2011

We've run into the same/similar output drop bug (technically a "different" bug, with ID CSCtq86186, but it's really the same thing).  We're seeing it on a 3750X with 12.2(58)SE.  TAC also recommended that we upgrade to 15.2 to fix the issue, but as Andriy noted, it hasn't been released yet.  When I asked them about this, they said that it was actually fixed in 15.0.1(SE1), which was released Dec 2nd.  We haven't tested that assertion out yet; we've decided to hold off on putting the "fresh out of the oven" IOS on our gear, but if someone else has, and has a good or bad experience we'd be all ears.

And just to reiterate what Andriy said, use command "sh platform port-asic stats drop" to get the true numbers as "sh int <blah>" can't be trusted.  Ensure there are actually drops happening, so you don't end up chasing a ghost :)

-Tom

Tom Lusty
Senior Systems Engineer

[cid:image001.gif at 01CCC085.5B026A10]

Wayfair
177 Huntington Ave, Suite 6000
Boston, MA 02115
P: 617.502.7026
tlusty at wayfair.com<mailto:tlusty at wayfair.com>

-----Original Message-----
Date: Thu, 22 Dec 2011 13:09:24 +0100

From: Andriy Bilous <andriy.bilous at gmail.com>

To: John Elliot <johnelliot67 at hotmail.com>

Cc: cisco-nsp <cisco-nsp at puck.nether.net>

Subject: Re: [c-nsp] 2960S drops/packet loss

Message-ID:

                <CAOnSrb9LO8YffjjwHOPW09SKK+gpKqjWWBK-UFZVtxAivphAdA at mail.gmail.com>

Content-Type: text/plain; charset=ISO-8859-1

There is a bug in most of SEs for all stackable models, which makes drop statistics unusable (see CSCso81660 for example - there are lots of BugIDs with same diagnostics and keep in mind "Fixed-in is a lie).

Reported numbers aren't realistic and often go both ways - increase and decrease in what appears to be a random manner. sh platform port-asic stats drop should report right value though. So, before you start tweaking, you may want to look further into it and just wait for the sane IOS release. Our SE reported it's fixed in 15.2, which isn't available for download yet.

On Thu, Dec 22, 2011 at 4:41 AM, John Elliot <johnelliot67 at hotmail.com<mailto:johnelliot67 at hotmail.com>> wrote:

>

> Hi Guys,

>

> Have a pair of 2960's in a stack, one port(trunk) connects to another DC and we are seeing ~5% packet-lossand large output drops to this DC.

>

>

> #sh interfaces gigabitEthernet 1/0/17 counters errors

>

> Port ? ? ? ?Align-Err ? ? FCS-Err ? ?Xmit-Err ? ? Rcv-Err ?UnderSize

> ?OutDiscards

> Gi1/0/17 ? ? ? ? ? ?0 ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? ? ? ? 0 ? ? ? ? ?0 ?

> ? ? 182867

>

> GigabitEthernet1/0/17 is up, line protocol is up (connected) ?Hardware

> is Gigabit Ethernet, address is a0cf.5b87.ec11 (bia a0cf.5b87.ec11)

> ?Description: QinQ_to_DC2 ?MTU 1998 bytes, BW 100000 Kbit, DLY 100

> usec, ? ? reliability 255/255, txload 41/255, rxload 23/255

> ?Encapsulation ARPA, loopback not set ?Keepalive set (10 sec)

> ?Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX ?input

> flow-control is off, output flow-control is unsupported ?ARP type:

> ARPA, ARP Timeout 04:00:00 ?Last input 6d13h, output 00:00:00, output

> hang never ?Last clearing of "show interface" counters 04:02:15 ?Input

> queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 183592

> ?Queueing strategy: fifo ?Output queue: 0/40 (size/max) ?30 second

> input rate 9047000 bits/sec, 2075 packets/sec ?30 second output rate

> 16324000 bits/sec, 2309 packets/sec

>

>

> As you can see, 30sec rate isnt excessive, but as the drops are outdiscards it would appear we are getting hit by the small buffers/microburst issue.

>

> Done a bit of reasearch, and as we have mls qos configured(Need to as we have to trust dscp markings), we needto look at "tweaking" the buffer allocations on the switch to hopefully mitigate these drops.

>

> There appears to be a range of recommendations when it comes to these tweaks - Hoping someone has some suggestions on what to set with "mls qos queue-set output" to alleviate the drops?(start conservative, then apply more aggressive if needed)....and also, does adjusting the buffers require an outage window?

>

> Our traffic is primarily backup(replication which is very bursty), and

> Internet

>

> Thanks in advance.

> _______________________________________________

> cisco-nsp mailing list ?cisco-nsp at puck.nether.net<mailto:?cisco-nsp at puck.nether.net>

> https://puck.nether.net/mailman/listinfo/cisco-nsp

> archive at http://puck.nether.net/pipermail/cisco-nsp/

------------------------------

_______________________________________________

cisco-nsp mailing list

cisco-nsp at puck.nether.net<mailto:cisco-nsp at puck.nether.net>

https://puck.nether.net/mailman/listinfo/cisco-nsp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-nsp/attachments/20111222/f6c70ab2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 3960 bytes
Desc: image001.gif
URL: <https://puck.nether.net/pipermail/cisco-nsp/attachments/20111222/f6c70ab2/attachment-0001.gif>