[nsp] Redundancy options

[Joshua Coombs]

I operate a small Co-location lan as part of my network.  This lan is
isolated as much as possible from my primary network to eliminate the
chance that a pesky customer box can take down my primary business.  To
that end, I have a 4500M+ setup as both a 'fuse-able link' and light link
throttle.  I have a block of ip's statically routed to it, it then
presents subnets of that main block to the individual colos using vlan
tagging to an HP 4000 switch.  Theoretically if something were to happen
to cause that router to go berserk, it's not participating in ospf, bgp,
etc, and as such can't impact the rest of my network.

This setup so far has worked great, simple, easy to maintain, but its a
flaming single point of failure.  I had this pointed out to me last
night when for no reason that I can currently ascertain, it stopped
passing traffic on it's public ethernet interface.  There were no error
counters incrementing, no log entries, etc.  Systems on the private
(colo) side could reach the router and pass packets amongst the various
subnets, so routing was up and functional.  After much head scratching,
I saved the conf, reloaded, observed correct behavior, and gave the
router an ok bill of health when the stored conf matched my remote
archive copy.

While I try to figure out what happened, I also need to implement some
sort of fail over system, preferably automatic to keep this from
happening in the future.  HSRP/VRRP should be easy enough to implement
on the public interface, which will work if one router completely dies.
My concern is how well would hsrp work in a situation like last night.
On the public side the functioning router will pickup the ip and respond
to the outside world, on the private/colo side, the 'dead' router will
still be responding, so it'll hold the ips for the colo interfaces,
which will result in 0 traffic traversing the routers.  Is there a more
elegant solution than hsrp for this setup?

My current contingency is going to be manual fall-over for now, a second
identically configured router, plugged in, but the corresponding switch
ports on each end disabled.  In the event of a failure I'll manually
remote activate those ports to let it take over.

Joshua Coombs