[c-nsp] All multicast punting to CPU on 6500

[Jens S Andersen]

Hi

We use the static ARP/mac address trick for our NLB load sharing:

arp 172.25.14.30 03bf.ac19.0e1e ARPA
arp 172.25.14.35 03bf.ac19.0e23 ARPA
arp 172.25.14.69 03bf.ac19.0e45 ARPA

mac address-table static 03bf.ac19.0e1e vlan 846 interface TenGigabitEthernet2/3
mac address-table static 03bf.ac19.0e23 vlan 846 interface TenGigabitEthernet2/3
mac address-table static 03bf.ac19.0e45 vlan 846 interface TenGigabitEthernet2/3

http://www.cisco.com/en/US/products/hw/switches/ps708/products_configuration_example09186a0080a07203.shtml

-Jens

>On 12/16/2012 11:59 AM, Robert Williams wrote:
>> Hi, I'll try to go into some additional detail on the traffic and
>> other router config elements now.
>>
>> The traffic is basically made up of a randomly generated packet which
>> is almost identical to the below.

>Yuck.

>Essentially, this is a malformed packet and I'm not surprised it's
>hitting the CPU ("it's multicast" / "it's not multicast") and causing
>problems.

>Basically "don't do that".

>> As such there is no multicast configuration. In fact, if anything it
>> would be ideal if the box dropped all multicast traffic apart from
>> the HSRP/VRRP to be honest.

>You might be able to write a MAC ACL which drops all multicast frames.

>>
>> The reason I think this may be causing issues is because it is
>> destined to a non-multicast IP, but with a multicast MAC....?

>Very likely.

>> The situation arose in the real world when a Windows NLB cluster went

>Wait, so this isn't the "real" problem traffic? You're testing?

>> offline and there was a load of traffic heading to its shared IPv4
>> address ('not' a multicast IP, but 'is' a multicast MAC) - so the
>> switch flooded to all ports, including the 6500 upstream, triggering
>> high CPU.

>Well known problem. Microsoft NLB is broken by design. Don't use it.

>Not the most helpful answer I know, but honestly, any work you do
>"fixing" this is better spent on supplanting MS NLB (e.g. install a
>couple of Linux boxes using haproxy as SLB-on-the-cheap).

>IIRC people who *do* use MS NLB often have to install static ARP / FDB
>entries to "stick" the output frames and get around just these sorts of
>problems.

>That said - I'm sympathetic to the general problem of "device collapsed
>under external load and I want to secure it from happening again". But
>bear in mind the age of the sup720/Earl7 architecture and its
>limitations (e.g. CoPP and multicast/ipv6). You might find a sup2t can
>be made more resistant to this kind of thing - but then again, you might
>not. State of the art in control-plane resilience to unusual traffic is
>not great IME :o(

>Cheers,
>Phil
>_______________________________________________
>cisco-nsp mailing list  cisco-nsp at puck.nether.net
>https://puck.nether.net/mailman/listinfo/cisco-nsp
>archive at http://puck.nether.net/pipermail/cisco-nsp/


Jens S Andersen                 Email:  jsa at adm.aau.dk
Aalborg University              Telf:   9940 9464
Niels Jernes Vej 14, C3, r 324  Fax:    9940 7593
9220 Aalborg
Denmark