[c-nsp] Annoying BGP+MPLS Tunnel problem

[Code Monkey]

Hi,

Short version: I set up an MPLS tunnel between two routers, using
global addresses, and the VRFs between the two routers stop working.
Why?

Long version:

I have an MPLS network composed of three sites (A, B, C) connected in
a triangle by very unequal links. Only sites A and C have Internet
transits, and they exchange the Internet routes by iBGP over the very
high bandwidth link A-C. The routers at B do not have the horsepower
for full Internet routes, and I don't really care about B choosing the
best exit point, so I have an IGP (OSPF) default route towards the
nearest eBGP speaker that has an active session; so far, so good.

The problem is that the high-bandwidth link A-C can go down, and has
done so several times. That leaves me with a dumb router at B that
does not carry Internet routes sitting between two eBGP speakers at A
and C that do, which is a problem when the eBGP speakers agree that
the best route is through one of them, but the intermediate dumb
router flips its coin and decides to to send it to the other router.
This is exactly the situation described at
    http://www.cisco.com/en/US/tech/tk365/technologies_tech_note09186a00800c95bb.shtml#sec5

The case study in question solves the problem by (cough)
redistributing BGP into OSPF (choke), and I Don't Want To Go There.

I had believed that MPLS tags would automagically solve the problem,
but I don't think they did or I wouldn't have had problems. I may be
wrong about that, but nothing is *forcing* my MPLS eBGP speakers to
tag the packets.

So I thought that I have an MPLS network after all, what about using
those cool TE tunnels I was drooling over, that should force the
tagging of the packets.

I first wanted to create an explicit-path fallback tunnel over the
backdoor, but it got used while the primary link was up, no good. I
set up bandwidth costs so it wouldn't be used, but if I set less or
equal to the existent bandwidth, I think the path would not be used
because the non-tunnel alternative would be better, and if I set more
bandwidth than there was available, it didn't set up the tunnel.

So why define an explicit path after all, it shouldn't matter if the
tunnel is always used (should it?). I set the tunnel type to dynamic,
and everything seemed to work wonderfully, until I realized that
packets going between A and C inside a VRF... didn't. ip cef said they
were sent over the tunnel, with one tag. On the other side of the
tunnel, the tag existed and corresponded exactly to the desired
destination inside the VRF. I didn't have time to check the return
lap, I shut down the tunnels, and everything worked again.

With tunnel:

sh ip cef vrf $MYVRF $IP
$IP/32, version 871, epoch 0
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
    fast tag rewrite with Tu0, point2point, tags imposed: {339}
  via $GLOBAL_LOOPBACK_OF_A, 0 dependencies, recursive
    next hop $GLOBAL_LOOPBACK_OF_A, Tunnel0 via $GLOBAL_LOOPBACK_OF_A/32
    valid adjacency
    tag rewrite with Tu0, point2point, tags imposed: {339}

After removing tunnel:

sh ip cef vrf $MYVRF $IP
$IP/32, version 871, epoch 0, cached adjacency $CONNECTED_INTERFACE_OF_A
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
    fast tag rewrite with Fa1/0, $CONNECTED_INTERFACE_OF_A, tags imposed: {339}
  via $GLOBAL_LOOPBACK_OF_A, 0 dependencies, recursive
    next hop $CONNECTED_INTERFACE_OF_A, FastEthernet1/0 via
$GLOBAL_LOOPBACK_OF_A/32
    valid cached adjacency
    tag rewrite with Fa1/0, $CONNECTED_INTERFACE_OF_A, tags imposed: {339}

So my questions: 1) am I attacking my problem in a reasonable way? 2)
anyone have an idea why it doesn't work?

Thanks a lot for any insights!