[c-nsp] Choosing Switches to Support VM Clusters

[Peter Rathlev]

On Thu, 2012-12-13 at 13:15 +0000, Gordon Bryan wrote:
> I have seen and read many a post on this list regarding microbursts
> and have always tried to err on the side of caution when choosing
> switches and usually go for models with meaty buffers and was
> considering for this deployment the 4948E. I was also considering the
> 3750X but not sure how the buffers stack up against the 4948 as the
> old 3750 models were poor in this regard.

The 3750X buffers are AFAICT similar to what the 3750 has. But you won't
necassarily need deep buffers in your deployment.

The shallow buffers are only a problem when converting down from a 1
Gbps flow to a 100 Mbps port or from a 10 Gbps to a 1 Gbps port. If you
use only 1 Gbps flows (i.e. all hosts have a 1 Gbps connection) and have
limited or no overlapping flows to the same host then you're fine.

The problem primarily manifests itself when your server has a 1 Gbps
connection and it tries to send data to a client on a 100 Mbps port. A
server with a 10 Gbps connection toward a 1 Gbps client would face a
similar problem, though I have little operational experience with that.

FWIW we use 3560X with 10G uplinks for VMware servers with N x 1 Gbps
trunks (typicall 6) and have no problems with buffers. This is probably
a combination of these being servers and thus on the sending end of most
traffic and almost all backend systems having just 1 Gbps connections.

> However, I'm looking for any general advice for choosing switches to
> support a virtualised environment. Are there any particular features I
> should be looking for or are there any gotchas that I should be aware
> of? My exposure to VMs is very limited so I'm hoping that the
> expertise available on this group can guide me

We're happy with the 3560X so far. If you want/need 10 Gbps directly to
the VMware servers you should look at something else, probably Nexus 5k
or 3k.

We use port configurations like this on the 3560X:

interface GigabitEthernet0/3
 description (Unused VMware port)
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 100-299,800-899
 switchport mode trunk
 switchport nonegotiate
 link state group 1 downstream
 storm-control broadcast level pps 250
 spanning-tree portfast trunk
 spanning-tree bpduguard enable
!

I've left out the QoS bits here.

The 3750X gives you stacking which, with the recently introduced rolling
upgrades, might not be the worst idea. They're a bit pricey compared to
the almost identical 3560Xs though.

-- 
Peter