[c-nsp] MPLS VPN with PE over GRE tunnels
Ross Halliday
ross.halliday at wtccommunications.ca
Mon Sep 19 19:18:19 EDT 2011
Hi list,
This one just popped up recently. I'm wondering if I'm missing some sort of platform caveat as I can't find a routing issue.
We have a customer with multi-site IP VPN. The whole thing previously was done in VRF-lite-style with VRFs, VLANs, and separate OSPF. We've since start cutting over to MPLS land. Since then there has been a problem with one remote site that connects over GRE tunnels.
Currently our network has one switch that is at the hub of our transition to MPLS as we cut various devices over and wait for maintenance windows. It has:
- Sites connected via MPLS
- Sites connected directly
- Sites connected via OSPF to an un-MPLSed box
- Sites connected via GRE tunnels
I've tried every combination I can think of, but any MPLS-transported site to the GRE-tunneled sites don't pass traffic in one direction. I started out with OSPF running over GRE tunnels from the PE router to the CE routers and switched to BGP with no dice. Route tables on every device are correct.
After trying to pass pings between loopback interfaces I have found:
Packets from MPLS site to GRE site work
Packets from GRE site to MPLS site do NOT work
Here is a brief sanitized config:
>From the P/PE box, a Cisco 6509 with SUP7203Bs on 12.2(33)SXI4a:
interface Loopback2210
description Customer VPN
ip vrf forwarding vpn-customer
ip address 10.17.10.17 255.255.255.255
!
interface Tunnel23
description Data VLAN 573
ip vrf forwarding vpn-customer
ip address 10.17.10.64 255.255.255.254
ip tcp adjust-mss 1400
tunnel source 10.10.3.5
tunnel destination 10.10.3.6
tunnel path-mtu-discovery
tunnel vrf vpn-customer-dsl
service-policy output Data-Tunnel-3meg
!
interface Tunnel24
description Data VLAN 574
ip vrf forwarding vpn-customer
ip address 10.17.10.66 255.255.255.254
ip tcp adjust-mss 1400
tunnel source 10.10.4.5
tunnel destination 10.10.4.6
tunnel path-mtu-discovery
tunnel vrf vpn-customer-dsl
service-policy output Data-Tunnel-3meg
!
router ospf 2210 vrf vpn-customer
router-id 10.17.10.17
log-adjacency-changes
area 2210 authentication message-digest
redistribute connected subnets
redistribute static subnets
redistribute bgp 64723 metric 20 subnets
network 10.17.10.0 0.0.0.15 area 2210
network 10.17.10.64 0.0.0.1 area 2210
network 10.17.10.66 0.0.0.1 area 2210
network 10.17.10.68 0.0.0.1 area 2210
network 10.17.10.70 0.0.0.1 area 2210
network 10.17.10.72 0.0.0.1 area 2210
!
!
router bgp 64723
!
address-family ipv4 vrf vpn-customer
redistribute connected
redistribute ospf 2210 vrf vpn-customer metric 20 match internal external 1 external 2
neighbor 10.17.10.65 remote-as 65171
neighbor 10.17.10.65 activate
neighbor 10.17.10.65 as-override
neighbor 10.17.10.67 remote-as 65171
neighbor 10.17.10.67 activate
neighbor 10.17.10.67 as-override
maximum-paths 4
default-information originate
no synchronization
exit-address-family
core-kgtn-c6509#sh ip route vrf vpn-customer
Routing Table: vpn-customer
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 10.17.10.1 to network 0.0.0.0
172.19.0.0/27 is subnetted, 2 subnets
O E2 172.19.10.32 [110/1] via 10.17.10.1, 00:05:15, Vlan2210
O E2 172.19.10.0 [110/1] via 10.17.10.1, 00:05:15, Vlan2210
10.0.0.0/8 is variably subnetted, 19 subnets, 5 masks
C 10.17.10.17/32 is directly connected, Loopback2210
O E2 10.17.10.16/32 [110/20] via 10.17.10.1, 00:05:15, Vlan2210
O E2 10.17.10.19/32 [110/20] via 10.17.10.4, 00:05:15, Vlan2210
B 10.17.10.18/32 [200/0] via 172.18.3.240, 01:32:11 <-------- MPLS-connected site
B 10.17.10.21/32 [20/0] via 10.17.10.67, 00:06:47 <-------- GRE site
[20/0] via 10.17.10.65, 00:05:20 <-------- GRE site
O E2 10.17.10.24/32 [110/20] via 10.17.10.73, 00:05:15, Tunnel27 <-------- GRE site
[110/20] via 10.17.10.71, 00:05:15, Tunnel26 <-------- GRE site
[110/20] via 10.17.10.69, 00:05:15, Tunnel25 <-------- GRE site
C 10.17.10.0/28 is directly connected, Vlan2210
O E2 10.17.10.48/30 [110/20] via 10.17.10.73, 00:05:15, Tunnel27 <-------- GRE site
[110/20] via 10.17.10.71, 00:05:15, Tunnel26 <-------- GRE site
[110/20] via 10.17.10.69, 00:05:15, Tunnel25 <-------- GRE site
B 10.17.10.52/30 [20/0] via 10.17.10.67, 00:06:47 <-------- GRE site
[20/0] via 10.17.10.65, 00:05:20 <-------- GRE site
O E2 10.17.10.56/29 [110/20] via 10.17.10.1, 00:05:15, Vlan2210
O E2 10.17.10.32/30 [110/20] via 10.17.10.4, 00:05:15, Vlan2210
B 10.17.10.36/30 [200/0] via 172.18.3.240, 01:32:11 <-------- MPLS-connected site
C 10.17.10.40/30 is directly connected, Vlan2213
C 10.17.10.44/30 is directly connected, Vlan2214
C 10.17.10.68/31 is directly connected, Tunnel25 <-------- GRE site
C 10.17.10.70/31 is directly connected, Tunnel26 <-------- GRE site
C 10.17.10.64/31 is directly connected, Tunnel23 <-------- GRE site
C 10.17.10.66/31 is directly connected, Tunnel24 <-------- GRE site
C 10.17.10.72/31 is directly connected, Tunnel27 <-------- GRE site
O*E2 0.0.0.0/0 [110/1] via 10.17.10.1, 00:05:15, Vlan2210
>From the CE device, a Cisco 1811 running 12.4(15)T13:
interface Loopback1
ip vrf forwarding vpn-uscu
ip address 10.17.10.21 255.255.255.255
!
interface Tunnel23
description Data VLAN 573
ip vrf forwarding vpn-customer
ip address 10.17.10.65 255.255.255.254
ip virtual-reassembly
ip tcp adjust-mss 1400
tunnel source 10.10.3.6
tunnel destination 10.10.3.5
tunnel path-mtu-discovery
tunnel vrf DoNotRoute
service-policy output Data-Tunnel
!
interface Tunnel24
description Data VLAN 574
ip vrf forwarding vpn-customer
ip address 10.17.10.67 255.255.255.254
ip virtual-reassembly
ip tcp adjust-mss 1400
tunnel source 10.10.4.6
tunnel destination 10.10.4.5
tunnel path-mtu-discovery
tunnel vrf DoNotRoute
service-policy output Data-Tunnel
!
router bgp 65171
no synchronization
bgp router-id 10.17.10.21
bgp log-neighbor-changes
no auto-summary
!
address-family ipv4 vrf vpn-uscu
redistribute connected
neighbor 10.17.10.64 remote-as 64723
neighbor 10.17.10.64 activate
neighbor 10.17.10.66 remote-as 64723
neighbor 10.17.10.66 activate
maximum-paths 2
no synchronization
exit-address-family
!
ce-CUSTOMER-vd-c1811#sh ip route vrf vpn-customer
Routing Table: vpn-customer
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 10.17.10.66 to network 0.0.0.0
172.19.0.0/27 is subnetted, 2 subnets
B 172.19.10.32 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 172.19.10.0 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
10.0.0.0/8 is variably subnetted, 19 subnets, 5 masks
B 10.17.10.17/32 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B 10.17.10.16/32 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 10.17.10.19/32 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 10.17.10.18/32 [20/0] via 10.17.10.66, 00:00:01 <-------- MPLS-connected site
[20/0] via 10.17.10.64, 00:00:44 <-------- MPLS-connected site
C 10.17.10.21/32 is directly connected, Loopback1
B 10.17.10.24/32 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 10.17.10.0/28 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B 10.17.10.48/30 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
C 10.17.10.52/30 is directly connected, Vlan2216
B 10.17.10.56/29 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 10.17.10.32/30 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
B 10.17.10.36/30 [20/0] via 10.17.10.66, 00:00:01 <-------- MPLS-connected site
[20/0] via 10.17.10.64, 00:00:44 <-------- MPLS-connected site
B 10.17.10.40/30 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B 10.17.10.44/30 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
C 10.17.10.64/31 is directly connected, Tunnel23
C 10.17.10.66/31 is directly connected, Tunnel24
B 10.17.10.68/31 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B 10.17.10.70/31 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B 10.17.10.72/31 [20/0] via 10.17.10.66, 00:00:01
[20/0] via 10.17.10.64, 00:00:44
B* 0.0.0.0/0 [20/20] via 10.17.10.66, 00:00:01
[20/20] via 10.17.10.64, 00:00:44
I've tried IPIP tunnels but they don't pass any traffic at all. Tomorrow I'll be ripping out the GRE config from the 6509 and pushing them to an outboard 1811 as it's the GRE tunnels that seem to stick out in my mind.
Any help or suggestions would be very appreciated!
Thanks
Ross Halliday
More information about the cisco-nsp
mailing list