[c-nsp] traceroutes via mpls network - works for on-net but not for off-net (def rt)

[Aaron]

123.123.144.1 is one of my l3vpn customer subnets and I can trace to it and see all my mpls p hops along the way... (mpls p boxes are 172.20.x.x) (not sure why the penultimate hop times out.... but perhaps that's another topic)

C:\>tracert -d -w 1 123.123.144.1

Tracing route to 123.123.144.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.150.65
  2     1 ms     1 ms     1 ms  172.17.0.9
  3     2 ms     2 ms     2 ms  123.123.191.49
  4     2 ms     2 ms     1 ms  123.123.191.17
  5     5 ms     5 ms     5 ms  172.20.1.34
  6     5 ms     4 ms     4 ms  172.20.1.2
  7     5 ms     4 ms     5 ms  172.20.1.6
  8     5 ms     5 ms     4 ms  172.20.1.62
  9     4 ms     4 ms     5 ms  172.20.45.2
 10     5 ms     5 ms     5 ms  172.20.45.22
 11     *        *        *     Request timed out.
 12     4 ms     5 ms     5 ms  123.123.144.1

Trace complete.

C:\>

When tracing to Cisco.com I follow the default route... the way I'm getting into this path is behind an ASA firewall (first couple hops, I've allowed ttl expired in transit icmp via the asa, and then I flow through the C/CE side of the L3VPN for one hop.... 123.123.x.x) 

C:\>tracert -d -w 1 www.cisco.com

Tracing route to e144.dscb.akamaiedge.net [172.226.176.54]
over a maximum of 30 hops:

  1     3 ms    <1 ms    <1 ms  192.168.150.65	- private inside of asa firewall
  2     1 ms     1 ms     1 ms  172.17.0.1		- private inside of asa firewall
  3     2 ms     2 ms     1 ms  123.123.191.49		- CE router of my L3VPN
  4     2 ms     2 ms     2 ms  123.123.191.17		- PE interface, facing CE router
  5     *        *        *     Request timed out.		- P router
  6     *        *        *     Request timed out.		- P router
  7     *        *        *     Request timed out.		- P router
  8     *        *        *     Request timed out.		- PE router facing internet provider
  9    12 ms     2 ms     2 ms  124.173.255.221	- CE router of my L3VPN
 10     3 ms     3 ms     3 ms  124.73.242.160	- on and on it goes out the internet......
 11     8 ms     8 ms     8 ms  97.77.2.200
 12     9 ms     9 ms     9 ms  124.175.33.58
 13    10 ms     9 ms     8 ms  124.175.32.144
 14     9 ms    10 ms    10 ms  124.175.32.156
 15    11 ms    11 ms    10 ms  107.14.19.94
 16    14 ms    14 ms    14 ms  66.109.6.39
 17    15 ms    15 ms    13 ms  66.109.9.105
 18    12 ms    12 ms    12 ms  172.226.176.54

Trace complete.


-----Original Message-----
From: Christian Meutes [mailto:christian at errxtx.net] 
Sent: Friday, September 05, 2014 4:51 AM
To: Aaron
Cc: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] traceroutes via mpls network - works for on-net but not for off-net (def rt)

On 2014-09-04 19:16, Aaron wrote:
> In my network traceroute works fine for on-net (known) subnets. I can 
> see the mpls lsr P hops.
>
>
>
> But when I traceroute to internet destinations off-net (unknown) 
> subnets and
> my packets follow default routing, I do not see my mpls lsr P hops.
>
>
>
> What is the deal with traceroute being broken when following the 
> default
> route ?

Just a guess:

Remember that the ICMP ttl-exceeded packet gets switched to the LSPs 
tail-
end LER/PE where IP processing can happen, but for a learned 
default-route
will most probably not occur and instead packets get MPLS-switched to 
the
default-routes l2adjacency on your ISP-facing LER/PE directly (without
consulting the VRF). Hence my guess is that your ISPs router doesn't 
want
to route the ttl-exceeded packets back to you (maybe URPF ingress -> 
you
have private linknetworks sourcing the ICMP-ttl's?).

Cheers
Chris