[c-nsp] MTU and PMTUD

[Saku Ytti]

Hey Marcin,

> XR without enabled PMTUD (default setting) means ~1240 bytes available
> for TCP payload.
>
> That seems to be a bit small, did you perform any kind of performance
> testing to see the difference between defaults and let's say 9000 for BGP?

I am embarrassed to say, but I _just_ found out, like literally weeks
ago, that Junos BGP TCP window is 16kB, I did also find hidden command
(https://github.com/ytti/seeker-junos) to bump it up to 64kB. I am
working with JNPR to have public documentation for the hidden command
to improve supportability and optics. I am separately working on hopes
of getting TCP window scaling.
I know that we are limited by TCP window, as the BGP transfer speed is
within 0.5% of theoretical max, and increasing 16kB to 64kB increases
BGP transfer speed exactly 4 times, being still capped by TCP window.
I think Cisco can go to 130k, but won't by default.
Maybe after that issue is remedied I will review packet size.

> I'm thinking about RRs in particular, higher MTU (9000 vs 1200) should
> result in some performance benefit, at least from CPU point of view. I
> haven't tested this though.

I've seen Cisco presentations in the 90s and early 00s showing
significant benefit from it. I have no idea how accurate it is
today,nor why it would have made a difference in the past, like was
the CPU interrupt rate constrained?

> Agree. Thing is, PMTUD on XR is a global setting, so it does affect all
> TCP based protocols.

You can do 'tcp mss X' under neighbor stanza.

-- 
  ++ytti