[c-nsp] iSCSI, port buffers, and small switches

[John Gill]

Peter,
Not too long ago there were some QoS knobs which gave us the ability to 
increase the shared pool to an even greater extent than the default "no 
mls qos".  Queue-set thresholds used to be  configurable in the range of 
1-100%, then 400%, and now up to 3200% of the reserved pool.  This 
effectively makes the buffer more like a fully shared pool, instead of a 
mix of reserved and shared.

So, I meant to say that the default vs just turning on "mls qos" without 
any configuration, and while using only one class of traffic, will 
result in less buffers available.

Chuck,
The decision on what switch to buy would depend on proper configuration 
(some older configurations and questions to the archives may not have 
been using the code sufficient to configure the queue-set in this 
manner), what classes you need to support, and the traffic patterns you 
expect to be able to deal with in terms of how much congestion/buffering 
you want to be able to absorb.

If we are just talking pure buffer space, the 4948 will be best for a 
scenario where fully shared buffering is preferable and you aren't 
worried about starvation for individual interfaces out of the box. 
Scale matters too; a test of 2 ports to 1 may perform well, but note a 
test where 2 instances of this pattern (2 to 1 and a separate 2 to 1 is 
also present) things start to look different.  If you have just one busy 
destination, then it is fairly simple - just get as much buffer as you 
can.  But speeding up your destination in that scenario would be money 
wisely spent.

Regards,
John Gill
cisco


On 7/26/11 3:38 PM, Peter Rathlev wrote:
> On Tue, 2011-07-26 at 14:47 -0400, John Gill wrote:
>> The customers I have come across with buffering problems on 3750 were
>> either not effectively configuring QoS - sometimes the best config is
>> no QoS actually.  The buffers get divided up when you enable "mls qos"
>> and if you don't use all the queues as they are set up, they will not
>> be optimal, especially for the default class.
>
> Hmm... this specific part has me a little confused. As the archives
> mention in detail the "apparently too small buffers" on the 3560/3750
> family of switches is not an uncommon problem. We have ourselves worked
> hard to come up with a solution, since replacing several hundreds of
> switches would be prohibitively expensive.
>
> The most successful of the solutions has been to actually configure QoS
> and re-partition the buffers. This makes me think that a switch in this
> family with "no mls qos" does not actually use the available buffers
> fully.
>
> I've talked with several SEs and some marketing boss (following up on a
> survey) about this, and none have been able to give any answers without
> having me sign NDAs. I'd rather not do that, since we have an acceptable
> solution until these switches are naturally replaced.
>
> We haven't fully tested the newest software (12.2(55)SE and later where
> the switch does a lot of ASIC reprogramming on startup) to see if the
> problem persists.
>
>> The big scenarios you need buffering are speed difference (regardless
>> of average rate!) or many-to-one host at the same time, otherwise you
>> don't need significant buffers at all.
>
> This is an important point IMO. If you don't need any kind of traffic
> differentiation and have ample bandwidth there should be no problems
> with small buffers.
>