[c-nsp] Nexus vPC loop avoidance details?

Tue Apr 26 07:15:20 EDT 2011

I don't know if this is your case, but we've had some similar problem on
that. The problem was that some hosts weren't using the HSRP IP as the
gateway, they were using the physical IP of one of the Nexus 7000 box.
Changing the gateway to the HSRP IP solved the problem.

The TAC wasn't helpfull on this case.

2011/4/23 Tóth András <diosbejgli at gmail.com>

> Hi,
>
> There's a good reason why it's not supported, mainly because it's not
> expected to work. It might work but there's no guarantee.
>
> You might consider opening a TAC case and do some further ELAM
> captures on the 6500 and N7k switches to see where the packets are
> seen, how and with which source/destination. This could explain the
> strange behavior you are seeing. Albeit, you might get the same
> information from TAC about the unsupported configuration.
>
> Best regards,
> Andras
>
>
> On Sat, Apr 23, 2011 at 6:46 PM, Adrian Chung <adrian at enfusion-group.com>
> wrote:
> > Even with peer-gateway this is still unsupported, but I'm trying to
> understand why some traffic flows that appear to take the same layer 2 and
> layer 3 paths work while others don't.
> >
> >
> > --
> > Adrian Chung (adrian at enfusion-group dot com)
> > http://www.enfusion-group.com/~adrian/
> > GPG Fingerprint: C620 C8EA 86BA 79CC 384C E7BE A10C 353B 919D 1A17
> >
> > ----- Original Message -----
> > From: Tóth András [mailto:diosbejgli at gmail.com]
> > Sent: Saturday, April 23, 2011 12:43 PM
> > To: Adrian Chung
> > Cc: cisco-nsp at puck.nether.net <cisco-nsp at puck.nether.net>
> > Subject: Re: [c-nsp] Nexus vPC loop avoidance details?
> >
> > Hi,
> >
> > Have you tried with peer-gateway enabled? Is it working with that?
> >
> > Please keep in mind the following limitation:
> > When you attach a Layer 3 device to a vPC domain, the peering of
> > routing protocols using a VLAN also carried on the vPC peer-link is
> > not supported. If routing protocol adjacencies are needed between vPC
> > peer devices and a generic Layer 3 device, you must use physical
> > routed interfaces for the interconnection. Use of the vPC peer-gateway
> > feature does not change this requirement.
> >
> > Refer to the following link:
> >
> http://www.cisco.com/en/US/docs/switches/datacenter/sw/5_x/nx-os/interfaces/configuration/guide/if_vPC.html
> >
> > Best regards,
> > Andras
> >
> >
> > On Sat, Apr 23, 2011 at 3:08 AM, Adrian Chung <adrian at enfusion-group.com>
> wrote:
> >> We've got 2 6500s (.145 and .146) connected to 2 Nexus 7Ks (.148, .149)
> >> running 5.1.3.
> >>
> >> The 6500s each have two ten gigE interfaces in a port-channel connected
> up
> >> to vPCs on the 7K side.  On top of this, each 6500 is forming an OSPF
> >> adjacency with each 7K.  The adjacencies form without a problem, and
> we're
> >> not using peer-gateway.
> >>
> >> I know this isn't a supported configuration, and we're looking at
> changing
> >> it, but in testing I've seen some weird scenarios that I can partly
> >> explain based on vPC loop avoidance, but other scenarios which I think
> >> should be blocked but don't seem to be.
> >>
> >> Here are a few scenarios, all based on two destination IPs .1 (an SVI on
> >> the 7K) and .10 (a server) in a subnet that is learned via OSPF and
> >> preferred via .149 (7K-2).  The destination subnet is routed from the
> 7Ks
> >> over a vPC to a service chassis on a transit subnet which goes to an
> FWSM.
> >>  The inside interface of the FWSM is on another transit subnet that
> comes
> >> back up the same vPC to a different VRF on the 7Ks which the destination
> >> subnet resides in.
> >>
> >> 1) from 6500 #1 (.145) attempting to ping an address ending in .10.
>  This
> >> works, with no packet loss.  If I calculate the port-channel hash on
> >> 6500-1 it uses the port-channel member that goes to 7K-2 (which is also
> >> where the L3 nexthop is.  7K-2 from what I can gather should send the
> >> traffic out its local vPC member port facing the service chassis.  On
> the
> >> service chassis, return packets hash to the port-channel member which
> goes
> >> to 7K-2.  No cross vPC peer-link traffic.
> >>
> >> 2) from 6500-1 (.145) attempting to ping the .1 IP which is an HSRP IP
> on
> >> an SVI on the 7Ks.  This doesn't work at all.  Port-channel hash on
> 6500-1
> >> sends to port-channel member that goes to 7K-1.  Since the nexthop is
> .149
> >> and is across the vPC peer-link, my understanding is it gets the loop
> >> avoidance bit set and sent across the peer-link, but since the egress
> port
> >> to the service chassis is on a vPC, and the vPC member port on 7K-1 is
> >> active, the packet gets dropped.
> >>
> >> That makes sense so far.  From 6500-2 (.146) the opposite scenarios
> work,
> >> can ping .1, but not .10.  For both, behaviour is consistent in that
> >> anything that crosses the peer-link gets filtered/dropped...
> >>
> >> But when testing with an IP that is upstream from the 6500s (204.x.x.x)
> I
> >> can ping both the .1 and .10 IPs with absolutely no problems, even when
> >> the traffic appears to cross the peer-link and takes the same path as 1)
> >> and 2) above.
> >>
> >> I'm trying to figure out what I'm missing or misunderstanding, it seems
> >> like all scenarios should fail equally as long as the vPC peer-link is
> >> involved in the data path, but this doesn't seem to be case.
> >>
> >>
> >> _______________________________________________
> >> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>
> >
> >
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>

-- 
[]'s

Lívio Zanol Puppim