[j-nsp] Tail drop on EX3400

[Jason Healy]

On May 30, 2019, at 2:23 AM, Saku Ytti <saku at ytti.fi> wrote:
> 
> 12MB / 1Gbps == 96ms. That would be massive buffer.

Not if you're Arista... ;-)

You're correct that it's 96ms for the 1Gbps side, but if packets are arriving at 10Gbps then that's only 9.6ms (ish) before you run out of buffer.  It's the mismatch in speed more than the actual buffer itself (assuming we're talking about megabytes of buffer, not gigabytes).

For steady state at a rate less than 1Gbps, the switch has enough buffer to handle the packets in flight. However, if packets arrive in microbursts then you can exceed the buffer briefly even though the amount of traffic is low on a larger timescale.  15MB of traffic evenly spread out over one second is not an issue, but 15MB of traffic arriving at 10Gbps at the start of a second, even with the rest of the second unused, is enough to overflow a buffer.  Both rates are "15MB/s", but the arrival rate makes a huge difference.

I've certainly seen tail drops on interfaces in bursts like this where it quiets down very quickly, but is enough to trip monitoring alarms.  We've maxed out the buffer configs on specific ports and haven't been able to eliminate the issue (not sure if it's reduced, as it's relatively infrequent).

Jason