[c-nsp] A9K Netflow export drops

[Robert Williams]

Hi,

Doing some more digging, found this from 2014:

Netflow specific scale and Limitations are described below:
  1. Supports configurable Sampling Rate 1:1 ~ 1: 65535
  2. Supports only up to 4 Sampling Rates (or Intervals) per Ethernet LC LC; no such limit for Enhanced Ethernet LC.
  3. Up to 4k interfaces/sub-interfaces (4K system limitation) can be configured with flow monitor per system.
  4. Supports up to 8 flow exporters per flow monitor
  5. Supports up to 1 million flow entries per LC
  6. Supports up to 50k flows per second with LC CPU usage up to 50% per Ethernet LC LC
  7. Supports upto 100K flows per second with LC CPU usage up to 50% per Enhanced Ethernet LC LC
  8. Netflow scale is increased to 200Kpps on Enhanced Ethernet LC based LCs
  9. Supports exporting packet rates up to 50k flows per second (100K flows per sec on Enhanced Ethernet LC based LCs) with LC CPU usage up to 50%

"Irrespective of the rate at which the NP punts the records to CPU, exporter picks up a maximum of 2000 records at a time from the cache that are eligible for export (timers, network/TCP session events, etc). This is basically to avoid NetIO dropping the packets due to lack of b/w. When the exporter wakes up again, it repeats the same."

So, it can collect 100k flows per second, but can only export 2k each time it runs the exporter. The interval for the exporter is unclear however.

I've also found out why this is such an issue on our 9001 but not on any of our 900x larger chassis. Looks like on those the hardware punt is being limited to 25kpps per NP because we have some BVIs with Netflow on them. This causes it to distribute the rate limit for punting to ALL the NPs on the LC, even when only two ports are involved in Netflow. Thus, it's "sampled sampling" and so the rate of flow data is significantly lower than the 9001 which is allowing all 100kpps on one NP which has 4 x 10G interfaces punting into it.

mmm...



Robert Williams
Custodian Data Centre
Email: Robert at CustodianDC.com
http://www.CustodianDC.com

-----Original Message-----
From: Dale W. Carder [mailto:dwcarder at wisc.edu]
Sent: 23 May 2016 16:02
To: Robert Williams <Robert at CustodianDC.com>
Cc: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] A9K Netflow export drops

Thus spake Robert Williams (Robert at CustodianDC.com) on Sat, May 21, 2016 at 10:59:50AM +0000:
>
> I've got an issue on one of our smaller 9001 boxes which is puzzling me.
> It suffers from a high rate of netflow export drops (not cache drops) shown here:
>
> So from what I understand, it is capturing the flows OK but is unable to get the flow data out, for some reason.

I can confirm that our 9k's suffer from this also.

The last I checked you can export at the rate of 2000 flows/sec.  I have not
looked in 2 years or so to see if this limit was configurable yet.

> So - what am I missing here? Surely with a cache capability of 1M it should be ok to export flows when were are only around 30,000 of them nicely ticking over?

join the club.  :-(

Dale