[j-nsp] filter-based forwarding... struggling

[ryanL]

hi. this should be dead simple, but it isn't and my google-fu is sucking.

all i want to do on my ex4500 is punt traffic to a next hop. simple
policy-based routing in cisco-speak. apparently you need a routing-instance
to do so. fine. we'll try it.

so here we go. i'm basically saying if the destination isn't other fleet
machines in 10/8, or the source isn't any of my public ip, throw it at my
proxy/nat box that lives at 10.1.0.51, which is learned via bgp (exabgp).
for now, i'm testing this only on one machine - 10.1.12.2, as referenced in
the firewall filter.

// config //

routing-instances {
    nat-vrf {
        instance-type forwarding;
        routing-options {
            static {
                route 0.0.0.0/0 next-hop 10.1.0.51;
            }
        }
    }
}
routing-options {
    interface-routes {
        rib-group inet fbf-group;
    }
    rib-groups {
        fbf-group {
            import-rib [ inet.0 nat-vrf.inet.0 ];
        }
    }
}
protocols {
    bgp {
        group NAT-VIP {
            family inet {
                unicast {
                    rib-group fbf-group;
                }
            }
        }
    }
}

interfaces {
    vlan {
        unit 112 {
            family inet {
                filter {
                    input FLEET-NAT;
                }
            }
        }
    }
}

firewall {
        filter FLEET-NAT {
            term pass-1 {
                from {
                    source-address {
                        <snip>;
                    }
                }
                then accept;
            }
            term pass-2 {
                from {
                    destination-address {
                        10.0.0.0/8;
                    }
                }
                then accept;
            }
            term else-nat {
                from {
                    source-address {
                        10.1.12.2/32;
                    }
                }
                then {
                    log;
                    routing-instance nat-vrf;
                }
            }
        }
    }
}
// end //

the routing instance nat-vrf sees the route to 10.1.0.51:

# show route table nat-vrf 10.1.0.51

nat-vrf.inet.0: 61 destinations, 62 routes (61 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.1.0.51/32       *[BGP/170] 00:36:53, localpref 500
                      AS path: I, validation-state: unverified
                    > to 10.1.5.11 via vlan.105

and we have a recursed route to the 10.1.5.11 next hop.

# show route table nat-vrf 10.1.5.11

nat-vrf.inet.0: 61 destinations, 62 routes (61 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.1.5.0/24        *[Direct/0] 00:41:46
                    > via vlan.105

forwarding table looks ok, i think:

# show route forwarding-table table nat-vrf destination 10.1.0.51
Routing table: nat-vrf.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
10.1.0.51/32       user     0                    indr 131083     5
                              10.1.5.11          ucst  1639     4 vlan.105

# show route forwarding-table table nat-vrf destination 10.1.5.11
Routing table: nat-vrf.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
10.1.5.0/24        user     0                    rtbl     1    29

i think the thing missing here is that nat-vrf doesn't have a mac address
next-hop for 10.1.5.11/32, much like inet.0 does:

# show route forwarding-table destination 10.1.5.11
Routing table: default.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
10.1.5.11/32       dest     1 0:25:90:19:93:ca   ucst  1639     4 vlan.105

so, when tcpdumping on 10.1.5.11, i see no packets come in from a fleet
machine as i'd expect.

the firewall log shows my curl attempts to google, so i know i'm making it
into the else-nat term properly.

# show firewall log
Log :
Time      Filter    Action Interface     Protocol        Src Addr
              Dest Addr
23:55:55  pfe       A      xe-0/0/12.0   TCP             10.1.12.2
               74.125.228.230
23:55:54  pfe       A      xe-0/0/12.0   TCP             10.1.12.2
               74.125.228.230

i'm a bit stumped from this point forward. i entirely admit that i don't
necessarily understand some of the knobs to turn with this setup. i did at
least try changing the routing-instance from "forwarding" to
"virtual-router".

not quite sure how to get nat-vrf to actually do the f part. do i have to
share arp across instances somehow as well?

appreciate any pointers!

ryan