[f-nsp] Fwd: egress queue drops
George B.
georgeb at gmail.com
Thu Jan 13 13:23:19 EST 2011
What is that switch connected to? Maybe *IT* is using flow control to
back off the port noted below. I am wondering of getting xoff from
the neighbor might cause a packet drop to cause the associated flow to
back off.
Ok, then you might have a case of "microbursts". Most people measure
utilization at 5 minute or maybe even 1 minute samples. So while the
average utilization might be only 50%, there might be periods in there
where the demand exceeds capacity. Imagine 3 machines with GigE
interfaces attempting a transaction to other machines accessed through
a GigE uplink. For a couple of seconds, you might have 3 Gig/sec of
traffic attempting to hit that GigE uplink. Congestion. You want to
drop a packet so one of them backs off. Random is better than tail
drop.
But I'm not going to beat the topic into the ground. Just pointing
out that this is an example of what is actually causing degraded
performance in a lot of networks. "Preserving" packets by increasing
buffers isn't better than allowing packets to be dropped and seeing
dropped packets isn't a "bad" thing. People should accustom
themselves to the notion that dropped packets are a perfectly normal
aspect of TCP and TCP is designed to use dropped packets to regulate
traffic flows. Somehow we have this notion that packet loss is a bad
thing, then we increase buffers and the result in many cases is that
the number of dropped packets goes up OR machines end up wildly
whipsawing in their congestion avoidance algorithms. If anything we
should be reducing buffers and allowing more packets to drop or
configure your networking in a non-blocking fashion (a 48 port GigE
switch having enough uplink, 20Gig maybe more) so that you never get
into the buffers to begin with. If you are getting into buffer, then
you should be dropping packets because you are only buffering when you
are congested.
On Thu, Jan 13, 2011 at 9:39 AM, Frederic Jaeckel <jchome at jc-ix.net> wrote:
> I do agree with the theory that packets should be dropped instead of
> being buffered. But at 1 GE linerate fiber with <50% utilization and
> around 100k packets/s this shouldn't happen with this kind of switch.
>
> I tried the buffer-sharing-full command of Jared, but I'm still seeing
> packets dropped within the first 10 minutes.
>
> Cheers,
> Frederic
>
> On Thu, 13 Jan 2011 09:28:28 -0800
> "George B." <georgeb at gmail.com> wrote:
>
>> Actually meant for the below to go to the list:
>>h
>> ---------- Forwarded message ----------
>> From: George B. <georgeb at gmail.com>
>> Date: Thu, Jan 13, 2011 at 9:27 AM
>> Subject: Re: [f-nsp] egress queue drops
>> To: Jared Valentine <hidden at xmission.com>
>>
>>
>> Actually, if anyone has been following the "bufferbloat" conversation
>> on gettys blog, it is probably better to drop packets than increase
>> buffers. If you are backing up into buffers, there is congestion and
>> you WANT to drop packets so the flows back off. He has noticed up to
>> TENS of seconds of traffic buffering end to end along some paths
>> (that is cumulative buffering in all devices along the path). This
>> means there is no way that TCP congestion avoidance and tuning
>> mechanisms can work. You might not find out a packet was dropped for
>> several seconds after the fact (because of traffic buffered up ahead
>> of the dropped packet along the path) and at that point all TCP does
>> is flail.
>>
>> Here's the blog:
>>
>> https://gettys.wordpress.com/
>>
>> Go back to the beginning of the conversation and read the comments,
>> too.
>>
>> The attitude of "every packet is precious" is causing a huge
>> performance degradation of end user applications and things would work
>> a lot more smoothly if buffers were smaller and more packets got
>> dropped sooner. That would allow congestion avoidance mechanisms to
>> actually function.
>>
>> George
>>
>>
>> On Thu, Jan 13, 2011 at 9:18 AM, Jared Valentine
>> <hidden at xmission.com> wrote:
>> > If you're not leveraging any QoS (and from the looks of your "show
>> > int" it doesn't look like it), then you could add
>> > "buffer-sharing-full" in the global config. That will release the
>> > buffers that are reserved for queues 1-7 and make them available to
>> > queue0, which is where all of your traffic currently is today.
>> >
>> > If you are leveraging QoS, or plan on doing so in the near-term,
>> > then you'll probably need to be a little more selective about
>> > reallocating your buffers using specific "qd" commands instead.
>> >
>> > Jared Valentine
>> > hidden at xmission.com
>> >
>> > -----Original Message-----
>> > From: foundry-nsp-bounces at puck.nether.net
>> > [mailto:foundry-nsp-bounces at puck.nether.net] On Behalf Of Frederic
>> > Jaeckel Sent: Thursday, January 13, 2011 9:18 AM
>> > To: foundry-nsp
>> > Subject: [f-nsp] egress queue drops
>> >
>> > Hej,
>> >
>> > I'm having a few FCX switches which display egress drops in the
>> > interface statistics.
>> >
>> > I'm curious how I can fix that and if I can fix that. The mentioned
>> > thing happens on every port of the switch and at most on the uplink
>> > ports. I set the symmetric flow control like that:
>> > symmetric-flow-control set 1 buffers 320
>> >
>> > and the error still occurs.
>> >
>> > Maybe someone has a hunch or knows the problem? Would be nice if I
>> > could fix that.
>> >
>> >
>> >> #show int e 1/1/1
>> >> GigabitEthernet1/1/1 is up, line protocol is up
>> >> Hardware is GigabitEthernet, address is 0024.3877.9c40 (bia
>> > 0024.3877.9c40)
>> >> Configured speed auto, actual 1Gbit, configured duplex fdx,
>> >> actual fdx Configured mdi mode AUTO, actual none
>> >> Member of 5 L2 VLANs, port is tagged, port state is FORWARDING
>> >> BPDU guard is Disabled, ROOT protect is Disabled
>> >> Link Error Dampening is Disabled
>> >> STP configured to ON, priority is level0, mac-learning is enabled
>> >> Flow Control is config enabled, oper enabled, negotiation
>> >> disabled mirror disabled, monitor disabled
>> >> Not member of any active trunks
>> >> Not member of any configured trunks
>> >> No port name
>> >> Inter-Packet Gap (IPG) is 96 bit times
>> >> IP MTU 1500 bytes
>> >> 300 second input rate: 136526024 bits/sec, 41720 packets/sec,
>> >> 14.31%
>> > utilization
>> >> 300 second output rate: 355122840 bits/sec, 55045 packets/sec,
>> >> 36.38%
>> > utilization
>> >> 7038712 packets input, 2873484976 bytes, 0 no buffer
>> >> Received 8 broadcasts, 1101 multicasts, 7037603 unicasts
>> >> 0 input errors, 0 CRC, 0 frame, 0 ignored
>> >> 0 runts, 0 giants
>> >> 9327790 packets output, 7548747686 bytes, 0 underruns
>> >> Transmitted 16 broadcasts, 7630 multicasts, 9320144 unicasts
>> >> 0 output errors, 0 collisions
>> >> Relay Agent Information option: Disabled
>> >>
>> >> Egress queues:
>> >> Queue counters Queued packets Dropped Packets
>> >> 0 9327946 1356
>> >> 1 0 0
>> >> 2 0 0
>> >> 3 0 0
>> >> 4 0 0
>> >> 5 0 0
>> >> 6 0 0
>> >> 7 0 0
>> >
>> > best regards,
>> > Frederic Jaeckel
>> >
>> > _______________________________________________
>> > foundry-nsp mailing list
>> > foundry-nsp at puck.nether.net
>> > http://puck.nether.net/mailman/listinfo/foundry-nsp
>> >
>>
>> _______________________________________________
>> foundry-nsp mailing list
>> foundry-nsp at puck.nether.net
>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
>
> _______________________________________________
> foundry-nsp mailing list
> foundry-nsp at puck.nether.net
> http://puck.nether.net/mailman/listinfo/foundry-nsp
>
More information about the foundry-nsp
mailing list