[c-nsp] Best practice for redundancy

[Phil Mayers]

Ed Butler wrote:
> I am wondering on what is considered best practice for redundancy.
> 
> We have two 6500-Sup720s in physically different sites, and we have two
> diverse pairs of fibre between them. We are linking them together with
> OSPF and BGP.
> 
> My question is what is considered best practice for that set up? We can
> have a single 20G port channel, one run a single OSPF and BGP link, or
> else leave the links unbundled and have 2 OSPF and 2 BGP links.
> Immediate disadvantage to this is tieing up more CPU cycles, but it does
> mitigate the effects of something affecting the port channel globally.

As far as I could see when I made that choice, the only disadvantages of 
the port channel approach are:

  1. Unless you use dynamic trunking protocols (which hurts convergence 
times), mis-patching one of the links causes absolute disaster - don't 
do that

  2. Error rates on one of the underlying links can cause havoc. An 
upper-layer protocol is needed to detect and disable noisy links. I am 
hoping that UDLD aggressive will serve as such, since it's what we've 
deployed.

For a separate pair of layer3 links 1. is obviously not a problem, and 
there is a larger range of "detection" for 2. (UDLD for the layer1.5, 
BFD, the OSPF packets and the BGP TCP packets)

If anyone has experience of UDLD succeeding or failing to detect errors 
on links, I'd like to hear about them. The other option is an EEM applet 
monitoring each link, both the tx and rx error rate, and shutting it 
down preemptively if the rates climb. Risky though... proper errdisable 
support would be preferred.

Having said all that, we use static port channels for this - it has by 
far the lowest maintenance penalty of the two options IMHO.

It also has the advantage that if you find yourself needing to put a 
layer2 networks over the link later on (e.g. migration) then spanning 
tree is not needed.