[j-nsp] SRX/MX, GRE, BGP, VRF and indirect prefixes

[Mike Williams]

Hey all,

Am hoping I can get some pointers here, as this has me stumped.

I'm trying to provision an IPv6 prefix to a remote SRX.
So far I've got a GRE tunnel between the devices, with an IPv6 prefix on.
The tunnel is between the loopback of the MX (main table) and the interface facing interface of the SRX.
The MX end is in a virtual-router routing instance (routing-instances blah interface gr-1/0/0.4702), the SRX end in the appropriate security zone.
BGP between them works. Prefixes can be sent between them, and they appear in the relevant RIBs and FIBs.

However we can't actually route any traffic between them.
Nor can the prefix announced from the SRX be further announced from the MX to other MXes.
Which I think is related to the fact the prefixes are all indirect.


SRX;

         BGP    Preference: 170/-101
                Next hop type: Indirect, Next hop index: 0
                Address: 0x19ade9c
                Next-hop reference count: 1
                Source: 2001:db8:85a3:4702::c000:220
                Next hop type: Router, Next hop index: 0
                Next hop: 2001:db8:85a3:4702::c000:220 via gr-0/0/0.4702, selected
                Session Id: 0x0
                Protocol next hop: 2001:db8:85a3:4702::c000:220
                Indirect next hop: 0x199c000 - INH Session ID: 0x0
                State: <Int Ext>
                Inactive reason: Route Preference
                Local AS: 65536 Peer AS: 65536
                Age: 1:30:53    Metric2: 0
                Validation State: unverified
                Task: BGP_65536.2001:db8:85a3:4702::c000:220
                AS path: I
                Accepted
                Localpref: 100
                Router ID: 192.0.2.32
                Indirect next hops: 1
                        Protocol next hop: 2001:db8:85a3:4702::c000:220
                        Indirect next hop: 0x199c000 - INH Session ID: 0x0
                        Indirect path forwarding next hops: 1
                                Next hop type: Router
                                Next hop: 2001:db8:85a3:4702::c000:220 via gr-0/0/0.4702
                                Session Id: 0x0
                        2001:db8:85a3:4702::/64 Originating RIB: inet6.0
                          Node path count: 1
                          Forwarding nexthops: 1
                                Next hop type: Interface
                                Nexthop: via gr-0/0/0.4702



MX;

mikew at mcdpinrt1> show route 2001:db8:25f:2000::/52 exact extensive table blah.inet6.0
2001:db8:25f:2000::/52 (1 entry, 1 announced)
TSI:
KRT in-kernel 2001:db8:25f:2000::/52 -> {indirect(1048602)}
        *BGP    Preference: 170/-101
                Next hop type: Indirect, Next hop index: 0
                Address: 0x572b5274
                Next-hop reference count: 2
                Source: 2001:db8:85a3:4702::4702
                Next hop type: Router, Next hop index: 1748
                Next hop: 2001:db8:85a3:4702::4702 via gr-1/0/0.4702, selected
                Session Id: 0x586
                Protocol next hop: 2001:db8:85a3:4702::4702
                Indirect next hop: 0x94c0ce00 1048602 INH Session ID: 0x587
                State: <Active Int Ext>
                Local AS: 65536 Peer AS: 65536
                Age: 1:48:35    Metric2: 0
                Validation State: unverified
                Task: BGP_65536.2001:db8:85a3:4702::4702+179
                Announcement bits (2): 1-KRT 4-Resolve tree 4
                AS path: I
                Aggregator: 65536 10.249.105.32
                Accepted
                Localpref: 100
                Router ID: 10.249.105.32
                Indirect next hops: 1
                        Protocol next hop: 2001:db8:85a3:4702::4702
                        Indirect next hop: 0x94c0ce00 1048602 INH Session ID: 0x587
                        Indirect path forwarding next hops: 1
                                Next hop type: Router
                                Next hop: 2001:db8:85a3:4702::4702 via gr-1/0/0.4702
                                Session Id: 0x586
                        2001:db8:85a3:4702::/64 Originating RIB: blah.inet6.0
                          Node path count: 1
                          Forwarding nexthops: 1
                                Next hop type: Interface
                                Nexthop: via gr-1/0/0.4702

mikew at mcdpinrt1> show route 2001:db8:85a3:4702::4702 extensive table blah

blah.inet6.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
2001:db8:85a3:4702::/64 (1 entry, 1 announced)
        *Direct Preference: 0
                Next hop type: Interface, Next hop index: 0
                Address: 0x9659f44
                Next-hop reference count: 2
                Next hop: via gr-1/0/0.4702, selected
                State: <Active Int>
                Local AS: 48447
                Age: 2:41:08
                Validation State: unverified
                Task: IF
                Announcement bits (2): 4-Resolve tree 4 5-Resolve_IGP_FRR task
                AS path: I



One of the more curious things I've found is that the subnet (2001:db8:85a3:4702::/64) on the GRE tunnel is in the FIB.
But the IP addresses at the other end aren't.


{primary:node0}
mikew at rlopinsg> show route forwarding-table destination 2001:db8:85a3:4702::c000:220/64 table default
Routing table: default.inet6
Internet6:
Destination        Type RtRef Next hop           Type Index    NhRef Netif
2001:db8:85a3:4702::/64
                   intf     1                    ucst     1592     2 gr-0/0/0.4702

{primary:node0}
mikew at rlopinsg> show route forwarding-table destination 2001:db8:85a3:4702::c000:220 table default
Routing table: default.inet6
Internet6:

{primary:node0}
mikew at rlopinsg>
Broadly the same at the MX end.



The virtual-router on the MX is connected to virtual-routers on other MXs via VPLS and logical tunnels, those can communicate, share prefixes, and route, with each other just fine.
They have /128 entries in the FIB, type dest and type ucst.

It seems like it's just this GRE that's misbehaving. I figure I'm just missing something small, something to cause next hops to not resolve over GRE, but haven't yet found it.

The tunnels are as easy as you can get;
    tunnel {
        source a.b.c.d;
        destination w.x.y.z;
    }
    family inet6 {
        address 2001:db8:85a3:4702::xxx/64;
    }

`host-inbound-traffic {system-services,protocols} all` makes no difference.

The prefix sent from SRX to MX is an aggregate, and MX to SRX a generate.

Changing from a virtual-router to a vrf, along with setting a route-distinguisher and vrf-target, made no difference.



Any help would be appreciated.

Thanks
Mike