[c-nsp] policy routing not working reliably?

[Furnish, Trever G]

Hello,

I have what seems like a simple policy routing set-up, and which I
thought was working well to allow gradual migration of our internal
subnets one at a time from one firewall to a replacement firewall, but
now it seems it's just not working reliably.  I haven't opened a tac
request yet (will do tomorrow) but I've been going nuts on this problem
and thought I'd poll quickly for thoughts before continuing.  Any ideas
appreciated.

Our WAN-connected sites come in from a router with inside interface
192.168.10.1 on vlan1 of a 4507R with vlan1 address 192.168.10.2.  The
old firewall is 192.168.10.9; the new firewall (where I want certain
sources to be sent for their next hop) is 10.1.1.51.

The problem is that some of the remote subnets never seem to get
policy-routed.  Others do.  Sometimes a machine will be policy-routed
for a while and then suddenly start getting routed through the old
firewall.  The set of symptoms seems impossible. :-(

I have the following set-up for the policy routing on the 4507R switch:

interface Vlan1
 description HQ Bldg
 ip address 192.168.10.2 255.255.255.0
 no ip redirects
 ip policy route-map vlan1_policy
end

route-map vlan1_policy deny 3
 match ip address internally_known_prefixes
!
route-map vlan1_policy permit 5
 match ip address inxx4_to_the_internet
 set ip next-hop 10.1.1.250
!
route-map vlan1_policy permit 10
 match ip address vlan1_pbr
 set ip next-hop 10.1.1.51
!

insw1#show ip access-list internally_known_prefixes
Extended IP access list internally_known_prefixes
    10 permit ip any 192.168.0.0 0.0.255.255 (6864103 matches)
    20 permit ip any 10.0.0.0 0.255.255.255 (25245 matches)
    30 permit ip any 172.16.0.0 0.15.255.255 (5042 matches)
    40 permit ip any 12.154.48.8 0.0.0.7 (8741 matches)
    50 permit ip any 12.23.229.64 0.0.0.7 (5652 matches)
    60 permit ip any 12.169.192.40 0.0.0.7 (2810 matches)
    70 permit ip any 12.168.84.32 0.0.0.7
    80 permit ip any 12.152.170.96 0.0.0.7 (2802 matches)
    90 permit ip any 192.90.191.32 0.0.0.31 (3624 matches)
    100 permit ip any 192.90.178.128 0.0.0.7
    110 permit ip any 192.90.16.0 0.0.0.255
    120 permit ip any 192.90.100.0 0.0.0.255
    130 permit ip any 192.90.15.0 0.0.0.255

insw1#show ip access-list inxx4_to_the_internet
Extended IP access list inxx4_to_the_internet
    10 permit ip host 192.168.10.88 any

insw1#show ip access-list vlan1_pbr
Extended IP access list vlan1_pbr
    40 permit ip 192.168.50.0 0.0.0.255 any (14 matches)
    50 permit ip 192.168.51.0 0.0.0.255 any
    60 permit ip 192.168.52.0 0.0.0.255 any
    70 permit ip 192.168.90.0 0.0.0.255 any (14705 matches)
    80 permit ip 192.168.255.0 0.0.0.255 any
    90 permit ip 192.168.2.0 0.0.0.255 any
    100 permit ip 192.168.53.0 0.0.0.255 any
    110 permit ip 192.168.121.0 0.0.0.255 any

insw1#show version
Cisco IOS Software, Catalyst 4000 L3 Switch Software (cat4000-I5S-M),
Version 12.2(25)EWA1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2005 by Cisco Systems, Inc.
Compiled Mon 25-Apr-05 15:37 by kellmill
Image text-base: 0x10000000, data-base: 0x114D5CF4

ROM: 12.1(12r)EW
Dagobah Revision 95, Swamp Revision 28

insw1 uptime is 36 weeks, 3 days, 13 hours, 35 minutes
Uptime for this control processor is 36 weeks, 3 days, 13 hours, 43
minutes
System returned to ROM by power-on
System restarted at 09:31:45 EDT Mon Apr 2 2007
System image file is "bootflash:cat4000-i5s-mz.122-25.EWA1.bin"

cisco WS-C4507R (MPC8245) processor (revision 7) with 524288K bytes of
memory.
Processor board ID FOX071501BB
MPC8245 CPU at 333Mhz, Supervisor IV
Last reset from PowerUp
22 Virtual Ethernet interfaces
96 FastEthernet interfaces
52 Gigabit Ethernet interfaces
403K bytes of non-volatile configuration memory.

Configuration register is 0x2102

--
Trever Furnish, tgfurnish at herffjones.com
Herff Jones, Inc. Unix / Network Administrator
Phone: 317.612.3519
Any sufficiently advanced technology is indistinguishable from Unix.