[j-nsp] Cut through and buffer questions

[Thomas Bellman]

On 2021-11-19 10:07, Saku Ytti via juniper-nsp wrote:

> Cut-through does nothing, because your egress is congested, you can
> only use cut-through if egress is not congested.

Cut-through actually *can* help a little bit.  The buffer space in
the Trident and Tomahawk chips is mostly shared between all ports;
only a small portion of it is dedicated per port[1].  If you have
lots of traffic on some ports, with little or no congestion,
enabling cut-through will leave more buffer space available for
the congested ports, as the packets will leave the switch/router
quicker.

One should note though that these chips will fall back to store-
and-forward if the ingress port and egress port run at different
speeds.  (In theory, it should be possible to do cut-through as long
as the egress port is not faster than the ingress port, but as far
as I know, any speed mismatch causes store-and-forward to be used).
Also, if you have rate limiting or shaping enabled on the ingress
or egress port, the chips will fall back to store-and-forward.

Whether this helps *enough*, is another question. :-)  I believe
in general, it will only make a pretty small difference in buffer
usage.  I enabled cut-through forwarding on our QFX5xxx:es and
EX4600:s a few years ago, and any change in packet drop rates or
TCP performance (both local and long-distance) was lost way down
in the noise.  But I have seen reports from others that saw a
meaningful, if not exactly huge, difference; that was several
years ago, though, and I didn't save any reference to the report,
so you might want to classify that as hearsay...  (I have kept
cut-through enabled on our devices, since I don't know of any
practical disadvantages, and it *might* help a tiny little bit
in some cases.)


[1] Of the 12 Mbyte buffer space in Trident 2, which is used in
    QFX5100 and EX4600, 3 Mbyte is used for per-port dedicated
    buffers, and 9 Mbyte is shared between all ports.  I believe
    on later chips an even larger percentage is shared.


-- 
Thomas Bellman,  National Supercomputer Centre,  Linköping Univ., Sweden
"We don't understand the software, and sometimes we don't understand
 the hardware, but we can *see* the blinking lights!"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <https://puck.nether.net/pipermail/juniper-nsp/attachments/20211119/392db36c/attachment.sig>