[c-nsp] Incomplete netflow on 7606/RSP720/MSFC4 L3 hardware switched interface with NAT & ACLs

[Matthew Huff]

We are getting intermittent netflow from our 7606 routers (flows show up for a  few seconds, then go away). The traffic is a consistent market data feed that averages around 100MBps, but we are only seeing a fraction of that via Netflow.  We are running NAT and ACL, but the FM seems to be happy.

Anyone seen this, or got any ideas? TAC has been less than helpful (or at least slow trying to see what the problem is).

We are running 12.2(33)SRE4.

interface GigabitEthernet5/1
 description SFTI-LCN
 ip address xx.xx.xx.xx 255.255.255.252
 ip access-group acl_lcn_in in
 ip access-group acl_lcn_out out
 no ip unreachables
 ip nat outside
 ip flow ingress
 load-interval 30
 no cdp enable

rtr-mahwah1#show fm fie interface gi5/1
Interface Gi5/1: 
Feature interaction state created: Yes
 Flowmask conflict status for protocol IP : FIE_FLOWMASK_STATUS_SUCCESS
 Flowmask conflict status for protocol OTHER : FIE_FLOWMASK_STATUS_SUCCESS
Interface Gi5/1 [Ingress]:
 FIE Result for protocol IP : FIE_SUCCESS_NO_CONFLICT
 Features Configured : RACL  NAT  L3_IFNDE   - Protocol : IP
 FM Label when FIE was invoked : 27
 Current FM Label : 27
 Features in Bank1 = RACL  L3_IFNDE  
+------------------------------------------------+
                Action Merge Table
+------------------------------------------------+
   RACL         L3_IFND RSLT    R_RSLT  COL
+------------------------------------------------+
   SB           X       HB      P       0
   HB           X       HB      P       0
   L3D          X       L3D     L3D     0
   P            X       P       P       0
+------------------------------------------------+
 Features in Bank2 = NAT  L3_IFNDE  
+------------------------------------------------+
                Action Merge Table
+------------------------------------------------+
   NAT          L3_IFND RSLT    R_RSLT  COL
+------------------------------------------------+
   HB           X       HB      P       0
   SB           X       HB      P       0
   X            X       P       P       0
+------------------------------------------------+
 num# of strategies tried : 1
 Description of merging strategy used:
  Serialized Banks: FALSE
  Bank1 Only Features: RACL  
  Bank2 Only Features: NAT  
  Banks Swappable: FALSE
 Merge Algorithm: ODM
  num# of merged VMRs in bank 1 = 9 
  num# of free TCAM entries in Bank1 = Unknown
  num# of merged VMRs in bank 2 = 65 
  num# of free TCAM entries in Bank2 = Unknown
 FIE Result for protocol OTHER : FIE_SUCCESS_NO_CONFLICT
 Features Configured : OTH_DEF   - Protocol : OTHER
 FM Label when FIE was invoked : 27
 Current FM Label : 27
 Features in Bank1 = OTH_DEF  
+-------------------------------------+
        Action Merge Table
+-------------------------------------+
   OTH_DEF      RSLT    R_RSLT  COL
+-------------------------------------+
   SB           HB      P       0
   X            P       P       0
+-------------------------------------+
 num# of strategies tried : 1
 Description of merging strategy used:
  Serialized Banks: FALSE
  Bank1 Only Features: [empty]
  Bank2 Only Features: [empty]
  Banks Swappable: TRUE
 Merge Algorithm: ODM
  num# of merged VMRs in bank 1 = 1 
  num# of free TCAM entries in Bank1 = 32664
  num# of merged VMRs in bank 2 = 0 
  num# of free TCAM entries in Bank2 = 32600

rtr-mahwah1#show ip int gi5/1
GigabitEthernet5/1 is up, line protocol is up
  Internet address is xx.xx.xx.xx/30
  Broadcast address is 255.255.255.255
  Address determined by non-volatile memory
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Outgoing access list is acl_lcn_out
  Inbound  access list is acl_lcn_in
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is enabled
  ICMP redirects are always sent
  ICMP unreachables are never sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP Flow switching is disabled
  IP CEF switching is enabled
  IP CEF switching turbo vector
  IP Null turbo vector
  Associated unicast routing topologies:
        Topology "base", operation state is UP
  IP multicast fast switching is enabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF, Sampled Output Flow
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Probe proxy name replies are disabled
  Policy routing is disabled
  Network address translation is enabled, interface in domain outside
  BGP Policy Mapping is disabled
  Input features: Ingress-NetFlow, Access List, NAT Outside, MCI Check
  Output features: Post-routing NAT Outside, Post-Ingress-NetFlow, Access List, HW Shortcut Installation
  Post encapsulation features: HW Shortcut Installation
  Sampled Netflow is disabled
  IP Routed Flow creation is enabled in netflow table
  IP Bridged Flow creation is disabled in netflow table
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled

>From the config
-----------------
mls aging long 64
mls aging normal 32
mls flow ip interface-destination-source
mls nde sender version 5

ip flow-export source Loopback0
ip flow-export version 9
ip flow-export destination xx.xx.xx.xx 2055


----
Matthew Huff             | 1 Manhattanville Rd
Director of Operations   | Purchase, NY 10577
OTA Management LLC       | Phone: 914-460-4039
aim: matthewbhuff        | Fax:   914-460-4139