[nsp] strange (dcef?) routing

[jlewis at lewis.org]

I just turned up a peering connection between Atlantic.Net and a 
small network that's multihomed to Sprint, C&W, and Broadwing.  I'm very 
confused by routing decisions being made on a 7513 that has 2 of our own 
transit connections and which handles traffic for a bunch of our core 
network.

208.152.224.0/24 is one of the routes received from the new peer.

On the 7513:
#sh ip bg 208.152.224.0
BGP routing table entry for 208.152.224.0/24, version 18173843
Paths: (3 available, best #2, table Default-IP-Routing-Table)
  Advertised to non peer-group peers:
  209.208.6.225 209.208.6.242 209.208.6.250
  6461 3561 21921
    209.249.254.146 from 209.249.254.146 (209.249.254.146)
      Origin IGP, metric 0, localpref 100, valid, external
      Community: 6461:5997
  21921
    69.28.66.46 (metric 20) from 209.208.6.243 (209.208.6.243)
      Origin IGP, localpref 120, valid, internal, best
      Originator: 209.208.6.235, Cluster list: 209.208.6.243
  3356 1239 21921
    64.156.210.13 from 64.156.210.13 (209.247.3.131)
      Origin IGP, metric 0, localpref 100, valid, external
      Community: 3356:3 3356:86 3356:575 3356:666 3356:2006

#sh ip ro 208.152.224.0
Routing entry for 208.152.224.0/24
  Known via "bgp 6364", distance 200, metric 0
  Tag 21921, type internal
  Last update from 69.28.66.46 00:58:13 ago
  Routing Descriptor Blocks:
  * 69.28.66.46, from 209.208.6.243, 00:58:13 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1

This shows the peering path is the best path.  However, when I traceroute,
depending on where on our network I do the trace from and which of the
peer's networks I trace too (all are localpref 120, best via the peering
connection), the 7513 may send the packets out to Level3, MFN, or use the
internal path several hops across our network to the peering connection.

traceroute to 208.152.224.1 (208.152.224.1), 30 hops max, 38 byte packets
 1  border1-fast0-0.Gainesville.atlantic.net (209.208.0.1)  2.095 ms
 2  gsvlflma-br-1-s0-0 (209.208.6.125)  1.449 ms
 3  progress-br-1-s2-0 (209.208.112.133)  4.688 ms
 4  andc-br-2 (69.28.68.3)  6.160 ms
 5  gigabitethernet5-1-111.hsipaccess1.Orlando1.Level3.net (64.156.210.13)  6.792 ms
 6  ge-6-2-0.mp1.Orlando1.Level3.net (209.247.11.37)  7.018 ms00
 ...

traceroute to 208.152.224.1 (208.152.224.1), 30 hops max, 38 byte packets
 1  209.208.127.13 (209.208.127.13)  0.484 ms
 2  andc-vl-1-gw (69.28.65.1)  0.663 ms
 3  orldflma-br-2 (69.28.68.8)  2.103 ms
 4  miamflbr-br-1-s1-1 (209.208.12.186)  11.475 ms
 5  gw.mmaero.com (208.152.224.1)  14.656 ms

andc-br-2 and andc-vl-1-gw are different names/IPs/interfaces for the same
7513.  Other than CEF bugs, why would the router send packets anywhere
other than the best BGP path?  The only policy routing currently setup on
this router is on our transit (MFN and Level3) interfaces blocking
incoming nachi pings.  The 7513 is running 12.2(14)S1 with dcef enabled.

----------------------------------------------------------------------
 Jon Lewis *jlewis at lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |  
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________