[j-nsp] Tail drop on EX3400

[Jason Healy]

On May 28, 2019, at 10:17 PM, Philippe Girard <philippe at skyhook.ca> wrote:
> 
> I've asked all of those questions but I can't seem to get a clear answer.

One additional question: what is upstream from the 1g interface that's showing drops?  Is it 10g (or larger)?

We have several small buildings that we're feeding 1g to from an EX4200-24F.  However, the uplink to our core is 10g, so there's a speed mismatch in terms of how fast packets can arrive in the distribution switch versus how fast they can drain.  We'll frequently see tail drops on egress when a burst of packets come in until TCP does its thing and the bandwidth levels out.

If you look at the bandwidth graphs, we're rarely over a hundred Mb/s, so it looks like the interface isn't maxing out.  However, the packets arrive much more quickly than they can leave, so there's a bottleneck there.

There really isn't any clever way around it; I think those switches have 12MB of buffer (or is that the QFX?).  Anyway, if you do the math you quickly find out that works out to like 10ms of traffic, so the switch simply can't buffer even short amounts of mismatched speed traffic no matter what you do with the buffers.  And at 10ms, most monitoring software simply doesn't have the resolution to catch those bursts.

As you noted, the end user often doesn't notice.  However, it might help explain how you're seeing loss even at low rates, yet that don't appear to adversely affect traffic.

Jason