[c-nsp] Netflow on SUP720-3BXL

[Andy BIERLAIR]

I'm trying to run netflow on one of our Cisco core routers (SUP720-3BXL with
SXF15a), but I think I am hitting some limitations because of this:

  %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM
Utilization [99%]

The setup of netflow looks like this (globally):

  ip flow-cache entries 524288
  mls aging fast time 5 threshold 32
  mls aging long 300
  mls aging normal 60
  mls netflow usage notify 80 300
  mls flow ip full
  no mls flow ipv6
  mls nde sender version 5
  no mls verify ip checksum
  no mls acl tcam share-global

  ip flow-export source Loopback0
  ip flow-export version 5 origin-as
  ip flow-export destination <ip> <port>

Then I have this enabled on all border interfaces/vlans (peering / transit /
other core routers) that are of interest for my stats:

  ip route-cache flow

Some more details about the problem:


#sh mls netflow table-contention detailed
Earl in Module 5
Detailed Netflow CAM (TCAM and ICAM) Utilization
================================================
TCAM Utilization             :   100%
ICAM Utilization             :   13%
Netflow TCAM count           :   262033
Netflow ICAM count           :   17
Netflow Creation Failures    :   4822220
Netflow CAM aliases          :   1


#sh mls netflow table-contention aggregate
Earl in Module 5
Aggregate Netflow CAM Contention Information
=============================================
Netflow Creation Failures    :   130003616
Netflow Hash Aliases         :   4


#sh mls netflow flowmask                 
current ip   flowmask for unicast:    full   
current ipv6 flowmask for unicast:    null   



I understand that the TCAM is full, but what can I do against it? This is a
busy core router:

  Aggregated traffic: 7-8 GBIT/s
  Packets per Second: 1.0 - 1.2 Million

I have heard that more agressive aging might help, but I expect the router's
traffic and pps to increase dramatically, so I'll be hitting the roof over
and over again.

I wouldn't mind analyzing only every 10th or 100th flow (sampling), which
seems to be a common practice, but will it help?

What is the common netflow setup without additional DFCs for a busy router?

Any good piece of advice is welcome.

Thanks!


-
Andy