[c-nsp] Nexus vPC loop avoidance details?

[Lincoln Dale]

>> whether a device sends to the 'right' or 'wrong' N7K depends on which
>> physical link it chooses to use in a LAG bundle.  as the neighboring
>> device has no idea its a point-to-multipoint bundle, its not really in a
>> position to choose the 'right' or 'wrong' link.
> 
> This makes complete sense.  It's just weird that when sourced from transit
> interfaces on the directly adjacent 6500s, traffic to the "wrong" N7K is
> actually dropped when the egress would be to another vPC, but when sourced
> from something beyond the 6500s, regardless of the physical link within
> the LAG that's chosen, all traffic appears to work.

the example i was talking about was OSPF which uses link-local multicast and TTL==1 packets - which means that if they arrive at the 'wrong' device, they cannot easily just be sent to the 'right' device.

i think what you're describing here is aspects of the vPC loop avoidance mechanism.

> 
>> as to how vPC does loop avoidance, its sort of beside the point as to how
>> it actually does it - just that it _does_ do it.
>> i don't think its a secret per-se as to how we do it, but what you've
>> observed with routing protocols is somewhat orthogonal to that.
> 
> Mostly curious as to why some scenarios appear to work even though the
> traffic is traversing links which are to the "wrong" N7K.  vPC loop
> avoidance should be dropping these packets as well, so my assumption was
> that it is more involved than simply setting a bit when the packet
> traverses the peer-link and then filtering.

the vPC loop avoidance system functions prevents loops by dropping packets/frames that are destined to go out a vPC member port _if all of:
 (a) that frame/packet arrived on the vPC peer-link
 (b) the vPC 'peer' switch could have sent the packet out an operational vPC member port itself

e.g. lets say that a packet/frame arrived on Nexus-A Portchannel100 (vPC peer-link) from Nexux-B.
if on making a forwarding decision Nexus-A determines that the packet/frame is to be forwarded to vPC PortChannel150 _and_ it knows that Nexus-B has operationally-up interfaces also in vPC PortChannel150, _then_ it will drop the frame/packet to prevent a loop, as it knows that Nexus-B could have sent it down that path itself.


if you're seeing drops in various scenarios, suggest you diagnose/debug what the IP-address and MAC address is on where you are directing frames.
it may well be that whatever you are testing is already in violation of RFC-791/RFC-826/RFC-5227 and are not associating a mac address with an IP address correctly.

over time, we have discovered numerous devices from a variety of vendors that violate some very basic RFC behavior which has caused issues.  introducing things like vPC peer-switch addresses some of these misbehaving devices, but are really addressing the symptom not the root cause.


> 
> In any case, it's not a big deal.  Unsupported, we won't do it, we'll
> leave it at that. :)

if you absolutely must run a routing protocol over vPC there are ways you CAN do it.  e.g. run it as router-on-a-stick that has L2 hops via vPC.  you can achieve this on a single physical box on N7K by making use of VDCs.

there's other ways too, e.g. run something like BGP with set-next-hop to the FHRP address.


cheers,

lincoln.