[c-nsp] Troubleshooting a GRE tunnel terminated in a MPLS VPN

Justin Shore justin at justinshore.com
Sat Nov 22 11:08:02 EST 2008


I'm testing a different method of terminating VPN tunnels in our data 
center.  We're going to switch from IPSec L2L tunnels to GRE tunnels 
with IPSec protection.  The big benefit is that it cuts our admin 
overhead significantly.  It's also a hell of a lot easier to deploy. 
Instead of having to have 6 lines of config for each remote customer 
prefix I can simply set up an IGP in their VRF and let the customer 
drive.  Much easier.

So I have one of my 7600s with a Sup720-3BXL running SRB1 with a 2G 
IPSec SPA and 6700 series line cards.  On the 7600 I've configured my 
VRF, GRE tunnel, OSPF vrf instance and set up iBGP to carry the VRF 
routes upstream to the data center.  I'm holding off on the IPSec config 
for now.  I want to get GRE working first.  Upstream at the DC I've 
configured that router with the VRF, sub-int in the VRF facing the DC 
switch, and iBGP for the VRF's routes.  LDP was configured with MPLS 
between the core and the DC and has been working for over a year so I 
doubt if that's a problem.  Downstream across the ISP I've set up a test 
router to simulate the customer's CPE.  I've configured it with the GRE 
tunnel, OSPF and the back-side network for a test laptop to test 
connectivity from the 192.168.0/24 subnet.

Laptop---CPE router---(ISP)---7600----DC Router
          |--------------GRE Tunnel------------|
                               |------VRF------|

Here's my config:

!!! 7600
ip vrf dc-gre-test
  description DC - GRE Test
  rd 100:2999
  route-target export 100:2999
  route-target import 100:2999
!
interface Tunnel2999
  ip vrf forwarding dc-gre-test
  ip address 10.125.124.1 255.255.255.252
  tunnel source aa.bb.cc.1
  tunnel destination cc.dd.aa.2
!
router ospf 2999 vrf dc-gre-test
  ignore lsa mospf
  ispf
  log-adjacency-changes
  redistribute bgp 65001 subnets
  passive-interface default
  no passive-interface Tunnel2999
  network 10.125.124.0 0.0.0.3 area 0
  network 10.125.125.0 0.0.0.255 area 0
!
router bgp 65001
  <snip>
  address-family ipv4 vrf dc-gre-test
  no synchronization
  redistribute connected
  redistribute static
  redistribute ospf 2999 vrf dc-gre-test
  exit-address-family


!!! CPE Router
interface Tunnel2999
  ip address 10.125.124.2 255.255.255.252
  tunnel source FastEthernet0/0
  tunnel destination aa.bb.cc.1
!
router ospf 1
  ignore lsa mospf
  ispf
  log-adjacency-changes
  passive-interface default
  no passive-interface Tunnel2999
  network 10.125.124.0 0.0.0.3 area 0
  network 192.168.0.0 0.0.0.255 area 0


!!! DC router
ip vrf dc-gre-test
  description DC - GRE Test
  rd 100:2999
  route-target export 100:2999
  route-target import 100:2999
!
interface GigabitEthernet0/1.2999
  description DC - GRE Test
  encapsulation dot1Q 2999
  ip vrf forwarding dc-gre-test
  ip address 10.125.125.2 255.255.255.0
  no ip redirects
  no ip unreachables
  no ip proxy-arp
  standby version 2
  standby 2999 ip 10.125.125.1
  standby 2999 timers msec 500 3
  standby 2999 priority 255
  standby 2999 preempt
!
router bgp 65001
  <snip>
  address-family ipv4 vrf dc-gre-test
  redistribute connected
  no synchronization
  exit-address-family


OSPF is up on the CPE and 7600 and I'm getting the DC route on the CPE 
and the CPE route on the DC router (and everything on the 7600 in the 
middle).  Currently I can ping within the VRF from the DC router to the 
tunnel interface on the 7600.  From the CPE router I can ping across the 
tunnel to the tunnel interface on the 7600.  For some reason though I 
can't ping from the DC to the CPE.  Traceroutes from both sides get to 
the tunnel interface on the 7600 and then die.  I can't figure out where 
the packets are going.  The CEF table for that VRF looks ok.  All 
adjacencies are pointing out the right interfaces.

7613-1.clr#sh ip cef vrf dc-gre-test detail
IPv4 CEF is enabled for distributed and running
VRF dc-gre-test
  12 prefixes (12/0 fwd/non-fwd)
  Table id 0x8
  Database epoch:        3 (12 entries at this epoch)

0.0.0.0/0, epoch 3, flags default route handler
   no route
0.0.0.0/32, epoch 3, flags receive
   Special source: receive
   receive
10.125.124.0/30, epoch 3, flags attached, connected, cover dependents, 
need deagg
   Covered dependent prefixes: 2
     need deagg: 2
   attached to Tunnel2999
10.125.124.0/32, epoch 3, flags receive
   Dependent covered prefix type cover need deagg cover 10.125.124.0/30
   Interface source: Tunnel2999
   receive for Tunnel2999
10.125.124.1/32, epoch 3, flags receive
   Interface source: Tunnel2999
   receive for Tunnel2999
10.125.124.3/32, epoch 3, flags receive
   Dependent covered prefix type cover need deagg cover 10.125.124.0/30
   Interface source: Tunnel2999
   receive for Tunnel2999
10.125.125.0/24, epoch 3
   recursive via 10.64.0.33 label 31
     nexthop 10.64.0.176 GigabitEthernet9/1
192.168.0.0/24, epoch 3
   local label info: other/138
   nexthop 10.125.124.2 Tunnel2999
224.0.0.0/4, epoch 3
   Special source: drop
   drop
224.0.0.0/24, epoch 3, flags receive
   Special source: receive
   receive
240.0.0.0/4, epoch 3
   Special source: drop
   drop
255.255.255.255/32, epoch 3, flags receive
   Special source: receive
   receive

The LFIB for that VRF on the 7600:

7613-1.clr#sh mpls forwarding-table vrf dc-gre-test detail
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
56     Pop Label     IPv4 VRF[V]       1140 
aggregate/dc-gre-test
         MAC/Encaps=0/0, MRU=0, Label Stack{}
         VPN route: dc-gre-test
         No output feature configured
138    No Label      192.168.0.0/24[V] 96080         Tu2999     point2point
         MAC/Encaps=24/24, MRU=1480, Label Stack{}
         4500000000000000FF2F5D194ADDC00143D5100200000800
         VPN route: dc-gre-test
         No output feature configured


The RIB for that VRF on the 7600:

Gateway of last resort is not set
       10.0.0.0/8 is variably subnetted, 3 subnets, 3 masks
C        10.125.124.0/30 is directly connected, Tunnel2999
L        10.125.124.1/32 is directly connected, Tunnel2999
B        10.125.125.0/24 [200/0] via 10.64.0.33, 7w0d
O     192.168.0.0/24 [110/11112] via 10.125.124.2, 7w0d, Tunnel2999

On the DC router:

Gateway of last resort is not set
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       10.125.125.0/24 is directly connected, GigabitEthernet0/1.2999
B       10.125.124.0/30 [200/0] via 10.64.0.10, 00:56:22
B    192.168.0.0/24 [200/11112] via 10.64.0.10, 00:56:07

On the CPE router:

Gateway of last resort is cc.dd.aa.1 to network 0.0.0.0
      cc.0.0.0/30 is subnetted, 1 subnets
C       cc.dd.aa.0 is directly connected, FastEthernet0/0
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
O E2    10.125.125.0/24 [110/1] via 10.125.124.1, 12:05:31, Tunnel2999
C       10.125.124.0/30 is directly connected, Tunnel2999
C    192.168.0.0/24 is directly connected, FastEthernet0/1
S*   0.0.0.0/0 [1/0] via 67.213.16.1


I don't have remote access to the laptop at this time so I can't sniff 
on the wire to see what's coming in there.  I'd have to place a device 
at the DC in that VRF to have something to ping out there other than the 
sub-int.  I can do that next week.

Any thoughts as to what might be going on?  Nothing is jumping out at me.

As a separate issue, my tunnel source on the 7600 is a SVI on a VLAN 
that spans 2 7600s.  HSRP is configured between them already though I 
haven't set up HA for VPN yet.  The 7600 we're working with is forced 
active with the priority setting.  I tried to use define tunnel source 
as that SVI, vl192.  It wouldn't work though.  I had to explicitly 
define the IP.  I used the HSRP floater and it worked.  I ran into a 
problem recently with our IPSec L2Ls where I couldn't use the floater. 
I had to use the interface IP specifically.  The IPSec packets 
originated from the interface IP and not the HSRP floater even though I 
had my crypto map local-address defined at that interface.  Any ideas 
why that is?  I need to do more research on IPSec and GRE HA.

Thanks
  Justin


More information about the cisco-nsp mailing list