[j-nsp] QFX5100 buffer allocation

[Brian Rak]

On 5/17/2018 2:58 AM, Thomas Bellman wrote:
> On 2018-05-17 02:41, Brian Rak wrote:
>
>> We're not even doing 10gbit of traffic, so the buffers should last at
>> least a little bit.
> And you're not hitting 10 Gbit/s even under very short bursts of a few
> milliseconds?  Microbursts like that don't show up in "normal" usage
> graphs where you only poll your switches/routers every minute or so.
It's possible that we have high traffic bursts, I don't really have the 
data to say one way or another (we only graph traffic at 5 minute intervals)
>> Thanks for the tip about cut-through, we didn't have that enabled.
>> Do you happen to know if it works from a 10g port to a broken out
>> 4x10g port?
> Should do.  From the perspective of the Trident II chip, they are
> not any different from normal 10G ports.  Cut-through doesn't work
> between ports of different speed, and the ports involved must not
> have any rate limiting or shaping, but other than that I don't know
> of any limitations.  (And if you receive broken packets, those will
> be forwarded instead of thrown away; that is the only disadvantage
> of cut-through mode that I have heard of.)
>
>> It's annoying to be dropping packets with a bunch of unused buffer
>> space.
> Just make sure you don't fill your buffers so much that you get a long
> (measured in time), standing queue, since that will just turn into a
> long delay for the packets without helping anything (search for "buffer-
> bloat" for mor information).  Not a big problem on Trident II-based
> hardware, but if you have equipment that boasts about gigabytes of buffer
> space, you may need to watch out.
>
> Oh, and I believe both changing buffer allocation and enabling/disabling
> cut-through mode resets the Trident chip, causing a short period (less
> than one second, I belive) where traffic is lost.
>
>
> 	/Bellman

We've seen significant reductions in dropped traffic after changing the 
buffer allocations.  We're going to continue to investigate why it's 
happening, but things seem to be much happier now.

I'm still not sure why the defaults are the way they are, I can't 
imagine FCOE traffic is common enough to warrant inclusion in the 
default configs.