[c-nsp] Dual Layer3 CPE-PE links

[Jee Kay]

I have a customer to whom I have 2 separate layer{1,2,3} links. These
are with different carriers, terminating on different sets of
equipment in diverse parts of the country with the aim of redundancy
in case of most failures. One link is 'primary' and should always be
used if it is available except after a failure - see below.

The customer unfortunately has stateful firewalling on their side, and
the firewalls at the two customer sites cannot synchronise state. As
such, if there is a failover all the customer's users are disconnected
but can then reconnect again pretty quickly.

This means that a flapping (or a simple down for a while/up
transition) primary causes an unacceptable number of disconnections.
More than one in a single day isn't really tenable for us. As such,
what we'd _really_ like to do is some way whereby we can say 'if the
primary link goes down, keep it down until manual intervention brings
it back up', and use the backup link in the mean time. Note that as
the links go across 'metro ethernet' (ie cheap) circuits, the link
going down doesn't necessarily result in a L1/L2 down on either CPE/PE
routers - most likely the only thing we'll see is a failure of
whatever L3 routing protocol we're running across the links or any
other shiny monitoring we set up.

(As a final caveat, the customer is pretty much unable to change any
config or behaviour on their routers due to internal procedures).

In more detail, we have eBGP peerings across either link (but do not
run iBGP in any shape between the routers internally) and the IGP is
OSPF.

Has anyone run into a similar problem, or can see a reasonable way to
sort this out? We're currently hacking something together with lots of
HSRP addresses and iBGP peerings and other pretty-but-nasty things
that sort of get the job done but are a pain in the backside to
maintain/understand.

Thanks in advance,
Ras