[c-nsp] 3560 buffering

[Jeff Bacon]

> There's a fundamental clash between desktop switch design and TCP operation on
> recent operating systems. Switches like 3560G by default buffer 100 MTU-sized
> packets, i.e. something like 150 kB of data. 3560Es are even worse, they only
> buffer 64 MTU-sized packets by default (~100 kB of data).
> 
> But in recent Linux kernels (as well as in Windows Vista) TCP buffer autotuning
> rises TCP window to megabyte(!) ranges. Hence a *single* TCP connection has no
> problem to overrun the buffer space and cause large amount of drops.

<random>
Though in theory, assuming that the switch is capable of receiving and transmitting at wire speed, the maximum amount of buffer it would need to deal with a 1Gb/s flow between A and B would be <switch latency>*1Gbit (plus a shade) since from the switch's point of view, data-in == data-out, irrespective of the window size of the hosts. Any additional buffer space is essentially just making up for the notion that more than one sender may be sending to the same dest port, in which case you're hanging onto packets from host C hoping that the burst of packets from host A will subside enough that you can fit in some packets from host C. At some point you've got to drop packets to tell the senders that _someone_ has to back off because the receiver can only handle 1Gb/sec.

So how much _should_ a switch buffer on behalf of the host? Not sure it has a ton to do with the window sizes of the hosts involved.

(Rhetorical question, really. I'd like to buffer more, but not a ton more or then my switches become a significant source of latency when really the hosts need fatter pipes. Some might feel it better to buffer less and tell the hosts to back off. Depends on the workload involved, I imagine.) 
</random>

Here at home, it does appear there was a distinct deficiency in 12.2(44)SE; it appears behavior is significantly improved with 12.2(52)SE.