[j-nsp] QFX5100 buffer allocation

[Brian Rak]

We've been trying to track down why our 5100's are dropping traffic due 
to lack of buffer space, even with very low link utilization.

It seems like they're classifying all our traffic as best-effort:
 > show interfaces xe-0/0/49:0 extensive
....
     Carrier transitions: 1, Errors: 0, Drops: 276796488, Collisions: 0, 
Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, 
Resource errors: 0, Bucket drops: 0
   Egress queues: 12 supported, 5 in use
   Queue counters:       Queued packets  Transmitted packets Dropped packets
     0                                0 1876090180637            276796488
     3                                0 0                    0
     4                                0 0                    0
     7                                0 663877                    0
     8                                0 0                    0

Then, if we look at those queues:

 > show class-of-service forwarding-class
Forwarding class                       ID      Queue  Policing priority  
No-Loss
   best-effort                          0         0 normal        Disabled
   fcoe                                 1         3 normal        Enabled
   no-loss                              2         4 normal        Enabled
   network-control                      3         7 normal        Disabled
   mcast                                8         8 normal        Disabled

So whatever, we've got queues configured for traffic we never see. 
That's not really a huge deal.  But then:

 > show class-of-service shared-buffer egress
Egress:
   Total Buffer     :  12480.00 KB
   Dedicated Buffer :  3744.00 KB
   Shared Buffer    :  8736.00 KB
     Lossless          :  4368.00 KB
     Multicast         :  1659.84 KB
     Lossy             :  2708.16 KB

To me, this appears that the 5100's by default reserve about 70% of the 
available shared port buffers for lossless+multicast traffic. The 
documentation seems to back me up here:

https://www.juniper.net/documentation/en_US/junos/topics/concept/cos-qfx-series-buffer-configuration-understanding.html#jd0e1441
https://www.juniper.net/documentation/en_US/junos/topics/example/cos-shared-buffer-allocation-lossy-ucast-qfx-series-configuring.html

Has anyone else encountered this?  We're going to be adjusting the 
buffers per the documentation, but I'd be really interested to hear if 
anyone has hit this before.