[c-nsp] Annoying BGP+MPLS Tunnel problem

[Code Monkey]

On 4/18/06, Marko Milivojevic <markom at pangalactic.net> wrote:
>
> It would help to see some configuration (and possibly some ASCII art) in
> your network. It's difficult to pinpoint the problem when you don't see
what
> you're dealing with.

Agreed, but there isn't much more to say :-)

I said an A B C triangle, OK it's not quite that simple but I don't think it
changes a lot.

I typed = for reliable and easily changed links like 1 yard long ethernet
cables, and - for WAN links. Hope the ASCII art is OK:

transit--\                              /--transit
          A1==\                     /==C1
transit--/     \                   /    \--transit
              switchA---WAN---switchC
               /                   \    /--transit
          A2==/                     \==C2
           |                            |
           |                            |
           \---WAN---B2===B1---WAN-----/

All links described except the transits are running MPLS with MTU >=1526.
All six routers are also connected to non-MPLS switches that have the
servers. All six routers are fully meshed using MPBGP / iBGP, but only the
ones with transits exchange the full Internet routes (using different
loopbacks for MPBGP and iBGP, if that makes a difference). The B1 and B2
routers would keel over and die in seconds if I gave them a full BGP feed.

There are also other MPLS-aware routers connected to switchA and switchC;
those are connected to further non-MPLS switches with servers.

I'm in the process of adding another A-C link (use the new one to link A1
and C1, use the old one to link A2 and C2 instead of switchA and switchC).
That will avoid problems if the existing A-C link goes down, avoid the
unhealthy reliance on switchA and switchC (those are very simply configured
switches, just pass MTU 1526 on all ports, with cold spares in the rack, but
still), and get rid of the disagreeable feeling I get when I look at an
ureliable link that would partition my backbone network (switchA and switchC
are the same layer2 ethernet network, no VLANS no nothing, I've never
managed to run MPLS on the same physical interface as VLANs).

All comments on network design heartily welcome!

> Did you create tunnels automatically or did you make them manually?

I created the tunnel interfaces manually (is there another way?) but I let
the path be chosen dynamically.

> P.S. I don't think that you're on the right track to solve your
> original problem wqith this, but I could be misunderstanding
> the whole problem.

OK :-( The original problem is that if the A-C link goes down,  I get
problems. I *THINK* the problem is that packets that should exit A1 loop
between B1 and C2, because C2 knows the packet should go out through A1, but
B1 is on the way from C2 to A1 and default-routes it back to C2. You'll
understand I'm reluctant to cut the link just to check that it doesn't work
:-) I'd prefer to believe it should work first, and test afterwards...

Thanks for the pointers, I'll try with TDP on the tunnel, but again, isn't
there a way to say "this is only for Internet traffic"? I tried setting the
tunnel destination IP to  the other side of the transit link, so that the
tunnel would be used only for packets that needed it in order to tunnel
through B, but that doesn't work because the IP is not in the MPLS topology.
:-)