[j-nsp] Tail drop on EX3400

[Philippe Girard]

Thanks everyone for your input, very interesting.

Reality is, we have ~300Mbps coming in the 10G port and ~50-100Mbps per
customer port at peaks, really, not that much.

Also, although tweaking TCP is nice, I can hardly go to each customer of
mine telling them to augment their TCP window settings because they're
triggering monitoring on my side.

I'm still very much unsatisfied with what JTAC told me, even with all my
very precise questions, I get few detailed answers and get told to increase
buffer allocation to 100% in COS. I'll give that the shot after hours today.

One thing that bugs me though is looking at default schedulers, still only
75% of buffer space is allocated to best effort traffic, so I'd still get
only 3/4 of the actual 100% allocation, and since I'm not getting the
answers I want, I don't know what the default allocation is VS that 100%
they want me to configure, so I have no idea of the actual increase this
will generate.

My guess is that I'll have to create a custom scheduler and apply to
interfaces to be able to have all that buffer space available for basic
Internet...

-----
Philippe Girard


On Thu, May 30, 2019 at 9:27 AM Jason Healy <jhealy at logn.net> wrote:

> On May 30, 2019, at 2:23 AM, Saku Ytti <saku at ytti.fi> wrote:
> >
> > 12MB / 1Gbps == 96ms. That would be massive buffer.
>
> Not if you're Arista... ;-)
>
> You're correct that it's 96ms for the 1Gbps side, but if packets are
> arriving at 10Gbps then that's only 9.6ms (ish) before you run out of
> buffer.  It's the mismatch in speed more than the actual buffer itself
> (assuming we're talking about megabytes of buffer, not gigabytes).
>
> For steady state at a rate less than 1Gbps, the switch has enough buffer
> to handle the packets in flight. However, if packets arrive in microbursts
> then you can exceed the buffer briefly even though the amount of traffic is
> low on a larger timescale.  15MB of traffic evenly spread out over one
> second is not an issue, but 15MB of traffic arriving at 10Gbps at the start
> of a second, even with the rest of the second unused, is enough to overflow
> a buffer.  Both rates are "15MB/s", but the arrival rate makes a huge
> difference.
>
> I've certainly seen tail drops on interfaces in bursts like this where it
> quiets down very quickly, but is enough to trip monitoring alarms.  We've
> maxed out the buffer configs on specific ports and haven't been able to
> eliminate the issue (not sure if it's reduced, as it's relatively
> infrequent).
>
> Jason
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>