[j-nsp] ex4200 routing weirdness

[Ross Vandegrift]

On Wed, Oct 14, 2009 at 05:58:50PM -0700, Cord MacLeod wrote:
> I have 2 switches in a 2 member virtual chassis and one of them my  
> siteops knocked over.  The use ISIS for the point to point links.  BGP  
> to carry the networks.  They take a default BGP route from both routers 
> and reflect a default to the top of rack 6 member virtual chassis switch.
>
> The 2 member switch receives 2 BGP routes from the to of rack virtual  
> chassis and reflects them to the routers.
>
> When switch in the 2 member virtual chassis died, I experienced some  
> strangeness in routing:

Do you have graceful-switchover enabled?  Are you using single,
physical point-to-point links or aggregated ethernet across memeber?
If you are using graceful-switchover, and you have point-to-point
links that terminate in only the failed member of the VC, then you can
run into that situation as follows:

When the member fails, graceful-switchover keeps the forwrding plane
intact until restart is complete.  However, for whatever reason (bug
or design, not sure), the top of rack switch fails to invalidate the
path over the link to the restarting peer that is now down.  This
means the top of rack switch blackholes all return traffic until the
failed member comes back, or the restart timer expires and routing
reconverges about the other member.

GRES with 4200 VC is weird, since you know you'll lose a large number
of ports when you lose the master.  So even if you want forwarding
paths maintained (there's plenty that will be unaffected by a
mastership change - best not to mess with those), there are other
paths that you definitely *want* invalidated, since the links have
gone away.

I'm avoiding this by making all of the L3 point-to-point links between
VCs aggregated ethernet devices that cross stack members.  This gives
you the most possible isolation of forwarding path failure from
control plane failure - LACP will notice that the individual link is
down, and disable that particular member without losing the layer 3
forwarding path.

So in your case, I'd build a single point-to-point link from the top
of rack switch to the aggregation VC, at least one element of the AE
on each member of the aggregation VC.

If you're not using graceful-switchover, then I'd expect your config
to work fine, modulo some forwarding impact while routing reconverged.

Ross

-- 
Ross Vandegrift
ross at kallisti.us

"If the fight gets hot, the songs get hotter.  If the going gets tough,
the songs get tougher."
	--Woody Guthrie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <https://puck.nether.net/pipermail/juniper-nsp/attachments/20091015/f799ae1a/attachment.bin>