[c-nsp] Dynamic output buffer allocation on Cisco 4948

[John Neiberger]

I've been helping to troubleshoot an interesting problem with variable
latency through a 4948. I haven't run into this before. I usually have seen
really low latency through 4948s, but this particular application requires
consistent low latency and they've been noticing that latency goes up on
average as load goes up. It didn't seem to be a problem on their servers,
but communication through busy interfaces seemed to dramatically increase
the latency. They were used to <1ms latency and it was bouncing up to 20+
ms at times. I'm starting to think this is due to the shared output buffer
in the 4948 causing the output buffer on the uplink to dynamically get
bigger.

I've been trying to find more details on how the 4948 handles its shared
output queue space, but I haven't been able to find anything. Do any of you
know more about how this works and what commands I could use to
troubleshoot? I can't find anything that might show how much buffer space a
particular interface is using at any given time, or if it even makes sense
to think of it that way. If I knew the size of the buffer at any given
moment, I could calculate the expected latency and prove whether or not
that was the problem.

Thanks!
John