[c-nsp] Brief CPU spikes on 6500 Sup 720

[Aaron Riemer]

Thanks Lincoln.

The server team must be using the "Route based on IP hash" method then.

"All adapters in the NIC team must be attached to the same physical switch
or an appropriate set of stacked physical switches."

Aaron.

-----Original Message-----
From: Lincoln Dale [mailto:ltd at cisco.com] 
Sent: Saturday, 17 July 2010 2:46 PM
To: Aaron Riemer
Cc: 'Lee'; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] Brief CPU spikes on 6500 Sup 720

On 17/07/2010, at 9:58 AM, Aaron Riemer wrote:
> Enabled SNMP traps and MAC-notifications and this brought another issue to
> my attention. There is a huge amount of mac-flapping going on (not for
this
> host) but our ESX hosts that have vmnics trunking to both our cores.
> 
> The VM guys are sending traffic for each VM host out both connected
vmnic's
> causing the MAC to be learnt on the vmnic port and the trunk port between
> the core switches hence the flapping.

your VM guys have seemingly configured their setup for link aggregation
(port channel) without LACP (i.e. "mode on") but failed to tell you network
guys.

suggest you configure the interface(s) facing these VMW servers to be
port-channels.

if its 'across chassis' and these are not VSS'd C6Ks then you need them to
change the loadbalancing policy.

see the NIC teaming section of
<http://www.vmware.com/files/pdf/virtual_networking_concepts.pdf>


> 
> Could this be contributing to the problem and possibly explain why MAC
> addresses are removed from the CAM table? No TCN's are noted so this
surely
> isn't the reason. MAC age is set to 4 hours same as default ARP timeout..

it could be contributing to it, yes.

be thankful that the C6K learns mac addresses in hardware.  if it did not,
likely you wouldn't have an operational network.

there may be other issues going on that this has been masking.  but
addressing this is a good start to then allow you to see if there anything
else awry going on.

> 
> I would prefer the VM team dedicated a NIC for their VM's to eliminate
this
> kind of behaviour.

the default is in fact that way.  ("Route based on the originating virtual
switch port ID").


cheers,

lincoln.=