[nsp] more confused routing

[jlewis at lewis.org]

How can this be happening?

jcvmfl-br-1 is connected to the rest of our network via several T1s to 
jcvlflcl-br-1.  The involved part of the network looks like:
[hopefully you can view with a fixed width font]

jaxvm--jaxcl--orl--andc
        |    /
        |   /
        |  /
        | /
        mia--mmaero


jcvmfl-br-1#
Tracing the route to gw.mmaero.com (208.152.224.1)
 
  1 jcvlflcl-br-1-s2-0-8.atlantic.net (209.208.112.153) [MPLS: Label 109 Exp 0] 8 msec
  2 orldflma-br-2-s2-1.atlantic.net (69.28.66.17) [MPLS: Label 208 Exp 0] 24 msec
  3 andc-br-2.atlantic.net (69.28.68.3) 88 msec
  4 orldflma-br-2.atlantic.net (69.28.68.8) [MPLS: Label 1235 Exp 0] 24 msec
  5 miamflgr-br-1-s1-1.atlantic.net (209.208.12.186) 24 msec
  6 gw.mmaero.com (208.152.224.1) 36 msec

jcvlflcl-br-1#traceroute 208.152.224.1
 
Type escape sequence to abort.
Tracing the route to gw.mmaero.com (208.152.224.1)
 
  1 miamflgr-br-1.atlantic.net (69.28.66.22) 8 msec 8 msec 4 msec
  2 gw.mmaero.com (208.152.224.1) [AS 21921] 8 msec 8 msec 8 msec

The two things I don't understand here are 

1) when I trace from jcvmfl-br-1, even though jcvlflcl-br-1 knows a 
shorter path to 208.152.224.1, while jcvmfl-br-1 is using its default 
route to pass traffic for 208.152.224.1 to jcvlflcl-br-1, traffic gets 
routed the wrong way several hops into our network before turning around 
and taking an alternate (longer) path to the destination.  The same trace 
done from jcvlflcl-br-1 will take the direct route above.

2) how can orldflma-br-2 route traffic the wrong direction the first time 
it sees it (hop 2) and the right way the second time (hop 4)?

I know I can fix this by cleaning up my IGP, but I don't understand how
routing table changes on an edge router (jcvmfl-br-1) can effect routing
decisions on core routers that "know better".

If I turn off synchronization on jcvmfl-br-1 and it accepts the bgp route
208.152.224.0/24 into its routing table, the trace takes the appropriate
shorter path.

  1 jcvlflcl-br-1-s2-0-6.atlantic.net (209.208.112.145) [MPLS: Label 141 Exp 0] 4 msec
  2 miamflgr-br-1.atlantic.net (69.28.66.22) 8 msec
  3 gw.mmaero.com (208.152.224.1) [AS 21921] 16 msec
 
----------------------------------------------------------------------
 Jon Lewis *jlewis at lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |  
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________