[c-nsp] iSCSI, port buffers, and small switches

[Geert Nijs]

I have also played with the QOS buffers and repartitioning on C3750E
switches. It helped, but i couldn't get the drops to go away completely.
We do have 10GE uplinks with 1GE server ports, so might need buffering.
On tip i have is also the following: although the C3750E switch is a 'shared
bufffer switch', each ASIC does have memory on-board. Normally 4 GE ports
are using 1 ASIC.
(you can see the mapping with "sh platform pm port-asic" or something like
that).
If you really have a critical server, connect it to its own ASIC and don't
use the other three ports. Then the server has all the buffer space for
himself. Also , don't set the QOS buffer division
to small and let each queue burst up to maximum size (full asic memory).
This was not the default setting in some IOS versions, hence the problems.
If you set the default queue too small or don't let it burst up to full
memory, i have noticed drops early on , even starting from 3Mbps from
3000-4000 pkts/sec...

regards,
Geert



2011/7/26 Peter Rathlev <peter at rathlev.dk>

> On Tue, 2011-07-26 at 14:47 -0400, John Gill wrote:
> > The customers I have come across with buffering problems on 3750 were
> > either not effectively configuring QoS - sometimes the best config is
> > no QoS actually.  The buffers get divided up when you enable "mls qos"
> > and if you don't use all the queues as they are set up, they will not
> > be optimal, especially for the default class.
>
> Hmm... this specific part has me a little confused. As the archives
> mention in detail the "apparently too small buffers" on the 3560/3750
> family of switches is not an uncommon problem. We have ourselves worked
> hard to come up with a solution, since replacing several hundreds of
> switches would be prohibitively expensive.
>
> The most successful of the solutions has been to actually configure QoS
> and re-partition the buffers. This makes me think that a switch in this
> family with "no mls qos" does not actually use the available buffers
> fully.
>
> I've talked with several SEs and some marketing boss (following up on a
> survey) about this, and none have been able to give any answers without
> having me sign NDAs. I'd rather not do that, since we have an acceptable
> solution until these switches are naturally replaced.
>
> We haven't fully tested the newest software (12.2(55)SE and later where
> the switch does a lot of ASIC reprogramming on startup) to see if the
> problem persists.
>
> > The big scenarios you need buffering are speed difference (regardless
> > of average rate!) or many-to-one host at the same time, otherwise you
> > don't need significant buffers at all.
>
> This is an important point IMO. If you don't need any kind of traffic
> differentiation and have ample bandwidth there should be no problems
> with small buffers.
>
> --
> Peter
>
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>