[j-nsp] Tail drop on EX3400

[Saku Ytti]

On Thu, 30 May 2019 at 04:06, Jason Healy <jhealy at logn.net> wrote:

> There really isn't any clever way around it; I think those switches have 12MB of buffer (or is that the QFX?).  Anyway, if you do the math you quickly find out that works out to like 10ms of traffic, so the switch simply can't buffer even short amounts of mismatched speed traffic no matter what you do with the buffers.  And at 10ms, most monitoring software simply doesn't have the resolution to catch those bursts.

12MB / 1Gbps == 96ms. That would be massive buffer.

But EX4200 actually has 2.5MB so 2.5MB / 1Gbps == 20ms, when all
buffer used exclusively by single 1GE port.

Meaning your RTT between SRC-DST needs to be <40ms or so to be able to
grow TCP window exponentially to reach from 500Mbps to 1Gbps single
stream size, when no other traffic is flowing through the switch.
After the tcp window has grown, the buffer stress is gone, as typical
sender TCP implementation floods packet as fast as they can when tcp
window grows, but in steady state paces to RX rate, so during steady
state almost no buffer stress would exist. With TCP algo like BRR
packets would always be paced, so even during window growth no
significant buffer stress would occur, it is also possible to
configure linux tc in such manner that regardless of TCP algo packet
pacing to RX rate exists, which would also alleviate buffering in
transit.


However like @op said, I don't think by default all this buffer is
actually available to single port, so situation is even worse. And
definitely right config will help the situation.



-- 
  ++ytti