[c-nsp] TCP ZeroWindow

[Troy Davis]

On Tue, Aug 30, 2005 at 01:58:36AM -0500, "Cheung, Rick" <Rick.Cheung at nextelpartners.com> wrote:

> 	I have an isolated network, with a Redhat 2.4 box on one end,
> connected to a 3640 (12.3.15) with an x-over, bridging its FE0/0 with
> the PPP Multilink (bundle of 8 PTP T1s), connected to another 3640 with
> a unix type box, also with an x-over.
>
> 	The response times from Mu1 to Mu1 is around 50ms. However, file

Is that reasonable latency for its physical path?  Packet serialization
time is inconsequential here, even on a single T1, so the rest is latency.
Depending on cable route, 50 ms RTT would be 1000+ miles as the crow
flies.

> 	A packet capture reveals that every so often, the Redhat box
> sends ~20 packets with a packet length of 1514, but the window size of
> 0. This happens every 100 someodd packets. The window size increases to
> ~1888 after these 20 packets, then decreases steadily to 0, and the
> process starts over again. Googling around, it seems a problem with
> Redhat/Fedora.

Indeed, sounds like the app server (Apache, ProFTPD, whatever) isn't
reading data from the OS, so the OS receive buffer fills up.

Make sure the Red Hat machine's available window is actually preventing
transmission: is the RH box actually receiving any data other than ACKs?
In an average HTTP download, after the server receives the request, the
server pumps full-frame packets to the client, and the client sends empty
ACKs.  The server's advertised window is basically immaterial.

So if the "unix type box" is doing an HTTP download from the RH box, the
Unix box's available window is what matters.

Assuming the Unix box is trying to transmit in your test, when the RH box
sends a non-zero window announcement, does the Unix box immediately tranmit?

It's worth using iperf to test UDP performance at various thruputs, and
make sure the server has enough RAM and OS recv buffer space ("sysctl
net.ipv4.tcp_rmem" - max and per-connection).

I've seen ATM or FR policing manifest itself as very deep queuing - more
than I thought practical - before anything was drop.  I don't know how it
would manifest as a full window, though, unless a ton of packets got
queued then dumped on the receiver quickly.  Long shot, just something to
consider if it's traversing an ATM/FR VC.

All of this assumes neither of the routers (and no "transparent" proxy) is
proxying/terminating TCP sessions.

Good luck,

Troy