[j-nsp] SSH/Telnet session hanging

[Jeff Wheeler]

On Wed, Jun 1, 2011 at 8:49 PM, Richard A Steenbergen <ras at e-gerbil.net> wrote:
> Well first off I hope you actually meant "something LOWER than 1500
> bytes", since tcp-mss doesn't include the headers that go into making up
> the 1500 byte IP packet. At a minimum you're looking at 20 bytes of IP +
> 20 bytes of TCP, so an mss of 1460, but don't forget to leave room for
> things like TCP MD5.

I used to be confused about this, because RFC 879 and RFC 2385 (and a
host of others) do not agree.

Once I spent a week fighting with an IOS bug whereby the router sends
packets/frames which are 20 octets too big when TCP MD5 is enabled
(going so far as to actually send them out interfaces and cause the
Giant counter on the remote side to increment a few times every time
the BGP session flapped) I decided to take this conservative approach.

I believe that IOS now does what RFC 2385 suggests (but older IOS is
just badly broken) and I think JUNOS follows RFC 879, and deals with
TCP Options by stuffing less payload into segments, interpreting the
MSS as including Options and payload, but not the earlier things in
the TCP header.  At least, this is what it did when I spent the better
part of a week on this problem.

> But more importantly, the maximum packet size for BGP is limited to 4096
> anyways, so the 9000 vs 9192 path mtu really doesn't make any difference
> at all. :) I suppose I could also take this opportunity to gripe about

It is important not to confuse BGP Message size as being directly
related to MSS or MTU in some way.  It isn't.  I have written some
posts on the IETF IDR list recently on this subject.

A larger MSS than the BGP Message size certainly can be taken
advantage of by a BGP speaker.  To observe this, all you must do is
disconnect an iBGP neighbor from an IOS box that is simply sending
routine keep-alives, and then cause it to send an update.  The
keep-alive will go out, fail to be ACK'd, and perhaps the same thing
will happen to the update.  Either way, when it is time to retransmit
the keep-alive, the IOS BGP Router will not just re-send the same
segment with only the keep-alive Message -- it will add the extra
payload of the subsequent update Messages and retransmit as a larger
segment.

Also, a normal TCP stack has a transmit buffer in the kernel, plus it
has a TCP Window to deal with as well.  It can accept more data into
its transmit buffer than it is allowed to have in-flight over the wire
before receiving ACKs.  This will cause some Messages to be buffered
in the kernel and be transmitted together as bigger segments once the
TCP Window advances a bit.

> an ongoing bug where Juniper's TCP stack occasionally thinks that the
> mss is ~64k, resulting in blackholing of the tcp packets and endlessly
> flapping sessions, but if I get started bitching about new junos bugs
> that are making my life hell right now I might not be able to stop. :(

This probably wouldn't happen if the perfectly-good TCP stack in the
kernel was left alone to do its job.  :-/

-- 
Jeff S Wheeler <jsw at inconcepts.biz>
Sr Network Operator  /  Innovative Network Concepts