[nsp] policy routing / CEF-IP-POLICY: fib for address 10.209.254.254 is with flag 0

[Gert Doering]

Hi,

another interesting challenge...

The problem:

Policy-Routing doesn't want to policy-route if the "set ip next-hop"
address is not directly adjacent.

interface FastEthernet0/1
 ip address 10.209.3.254 255.255.252.0
 ip route-cache policy
 ip policy route-map labor-policy
!
access-list 130 permit ip 10.209.1.0 0.0.0.255 135.80.0.0 0.0.255.255
access-list 130 permit ip 10.209.1.0 0.0.0.255 192.168.254.0 0.0.0.255
!
route-map labor-policy permit 10
 match ip address 130
 set ip next-hop 10.209.254.254
!

Intention: "Packets from 10.209.1.0/24 towards these destinations should
take whichever path is the best towards 10.209.254.254".

If I do this, and have CEF enabled, "debug ip policy" prints:

CEF-IP-POLICY: fib for address 10.209.254.254 is with flag 0
IP: s=10.209.1.1 (FastEthernet0/1), d=135.80.82.138, len 84, FIB policy rejected - normal forwarding

If CEF is off, I get:

IP: s=10.209.1.1 (FastEthernet0/1), d=135.80.82.138, len 84, policy match
IP: route map labor-policy, item 10, permit
IP: s=10.209.1.1 (FastEthernet0/1), d=135.80.82.138 (Multilink1), len 84, policy rejected -- normal forwarding


Connectivity and adjacencies are there:

Cisco-AVK#sh ip cef 10.209.254.254
10.209.254.254/32, version 21, cached adjacency 10.211.63.254
0 packets, 0 bytes
  via 135.80.82.254, 0 dependencies, recursive
    next hop 10.211.63.254, FastEthernet0/0 via 135.80.82.254/32
    valid cached adjacency

If I change the policy to read:

route-map labor-policy permit 10
 match ip address 130
 set ip next-hop 10.211.63.254

(which is directly adjacent), everything works as expected.

So my interpretation of this is: "indirect next-hops do not work for
policy routing".  And of course the question is "why?  and is there a
workaround to make it work?".  IOS is 12.2(21a) on a 2621XM.


As for "why would anybody want to do this", here's a bit of motivation.

Subnets A and B need to reliably communicate.

There is a ethernet connection with much bandwidth, but with some delay
issues and with some reliability issues, between them, crossing routes
X, Y, Z.

For timing critical traffic, there is a ISDN leased line, directly
connecting A and B.


  A ---------- small/low RTT ------------- B
  |                                        |
  +-- X --- Y --- Z --- fast/high delay----+


Bulk traffic is coming from 10.209.1.0/24, and should take the route
A->X->Y->Z->B.

Timing sensitive traffic is coming from 10.209.0.0/24, and should take
the direct route A->B.  (So "classic" routing is out).

So far, this could easily be done with a "set ip next-hop X" (direct
adjacency from A).

Unfortunately, it gets worse.  X/Y/Z have reliability issues, *and* are
not under our control.  So there is a BGP session A->X->Y->Z->B which
is used to make sure there is connectivity over that path.

B is announcing a loopback address via B->Z->Y->X->A and via a second
session B->A (with higher MED).  

So the idea was "policy-route towards that loopback IP.  If X/Y/Z work,
packets will travel that way.  If it doesn't work, packets take the
ISDN leased line for backup".

Yes, this is ugly (and I told them that it is, but there's a Telco 
involved in managing X, Y, and Z, so this cannot be fixed quick enough
for their needs).

Any other suggestions to sort this out:

  10.209.0.0/24 -> via ISDN, if available, X/Y/Z otherwise
  10.209.1.0/24 -> via X/Y/Z, if available, ISDN otherwise

are welcome.

gert
-- 
USENET is *not* the non-clickable part of WWW!
                                                           //www.muc.de/~gert/
Gert Doering - Munich, Germany                             gert at greenie.muc.de
fax: +49-89-35655025                        gert at net.informatik.tu-muenchen.de