[c-nsp] Cisco Nexus and Port-VLAN limits

[Tóth András]

Hi Adam,

The reason for having a limit is that the switch needs to program the
hardware to allow or deny packets on each port for each vlan which is
allowed on the port. In other words, based on whether a port is up/down,
whether in one particular vlan it can receive traffic or not (STP BLK), the
forwarding hardware needs to be programmed in order to process the packets
at wire-rate and this programming needs time and processing power, hence
the limit.

As the document mentions for Logical interfaces: "This parameter reflects
the load of handling port programming...". This needs to be done a per-vlan
basis, regardless of the STP mode.

Sometimes you can go above these limits a little, however be very cautious
about it, because:
1) it can result in unexpected behavior, mostly instability of STP and
other errors,
2) not supported by TAC, so if you go above the limit and some error
messages start to appear or STP goes crazy, TAC will ask you to reduce the
number of vlans on your ports so that you're below the documented limit.

Best regards,
Andras



On Sat, Jan 12, 2013 at 3:45 PM, Adam Chappell <adam.chappell at gmail.com>wrote:

> Hello all. Does anyone have extensive experience and familiarity with Cisco
> Nexus and specifically the causes and effects of the Port-VLAN product
> limit described here (mentioned as "Logical interfaces"):
>
>
> http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration_limits/limits_521/nexus_5000_config_limits_521.html#wp327738
>
> The number appears to have varied in different software releases, but in
> 5.2.1, it's claimed as 16,000 verified, and 32,000 theoretical maximum. I
> am interpreting this to mean that I can run 320 layer-2 ports if I run them
> with 100 VLANs, but if I decide to run them with 1000 VLANs, I am limited
> to 32 ports etc. etc (depending on the limit I trust).
>
> I've got a topology probably not uncommon for cloud service provider
> environments: multi-VLAN, MAC-aware switching for a number of 10G trunk
> edge ports, which are hosting virtualisation servers with per-customer
> VLAN-based logical separation.
>
> The Nexus gives me fabulous port density, especially with the fabric
> extenders, but really I want the flexibility for the edge virtualisation
> servers to be able to manage the VM load and "tap into" VLANs
> however/whenever they wish, without manipulating switchport trunk allow
> lists, or active VLAN lists.
>
> It isn't the case that any single trunk would have anything like 1000 VLANs
> on it at any one time - but I simply want the option for the end-host to be
> able to use VLANs in that space without needing switch reconfiguration.
>
> The active topology for all VLANs would be the same, and MST is proposed
> with all VLANs mapped to single instance, so there is no per-VLAN STP tax
> that would cause a burden. I note though that the Cisco docs explicitly
> note that these limits exist independent of STP choice, so it's something
> causing the limit.
>
> It seems though that running this sort of promiscuous trunk port
> configuration seriously hampers the number of physical ports I could run.
> The number isn't disastrous  but it's not ideal either, and I am very
> curious to understand the causes of the limit, and possible actions I might
> take to mitigate its effects.  Creative suggestions I've had so far:
>
> - run the edge trunks as 802.1Q tunnel, so as to only really consume a
> single VLAN on the Nexus.
>
> - switch off or otherwise disable VLAN-awareness on the Nexus, or change
> the EtherType recognised for VLANs.
>
> These options are interesting, but they would both collapse the MAC
> switching into a single VLAN-space as well, which I'm concerned wouldn't be
> ideal for any protocols with well-known MACs (router upstreams with
> VRRP/HSRP for example?).
>
> Any experiences or suggestions welcomed and appreciated.
>
> -- Adam.
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>