[c-nsp] A9K Netflow export drops

Tue Jun 14 08:32:09 EDT 2016

Hi Chris,

Thanks for this, we’ve not considered 6.0.1 yet, mainly due to it being relatively new and I’m not aware currently of anyone running it in production on a 90xx, so slightly apprehensive :)

I wonder if there will be a patch for 5.3.3 to stop the drops?...

Cheers!

From: ckildau at unixhosts.org [mailto:ckildau at unixhosts.org] On Behalf Of Christian Kildau
Sent: 14 June 2016 13:07
To: Robert Williams <Robert at CustodianDC.com>
Cc: Jimmy <hngjimmy at gmail.com>; cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] A9K Netflow export drops

Hi Robert,

we've finally received clarification from TAC:
In our case this was a bug within IOS-XR 5.3.X.
For us, this is fixed in 6.0.1 which we wanted to upgrade to anyway due to extended netconf support.

hth,
Chris

On Wed, May 25, 2016 at 5:05 PM, Robert Williams <Robert at custodiandc.com<mailto:Robert at custodiandc.com>> wrote:
Chris - Thanks, I've not yet opened one, but would be curious to hear the outcome of yours as it may save doubling up.

Jimmy - If I take the sampling to 1:1 then yes I can achieve this event, however, we appear to be at the limit of the 'exporter' not the 'monitor/cache' so at present the cache is not being exceeded.

Cheers guys!

Robert Williams
Custodian Data Centre
Email: Robert at CustodianDC.com
http://www.CustodianDC.com

Robert Williams
Custodian Data Centre
Email: Robert at CustodianDC.com
http://www.CustodianDC.com

-----Original Message-----
From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net<mailto:cisco-nsp-bounces at puck.nether.net>] On Behalf Of Jimmy
Sent: 23 May 2016 17:16
To: cisco-nsp at puck.nether.net<mailto:cisco-nsp at puck.nether.net>
Subject: Re: [c-nsp] A9K Netflow export drops

Hi,
Just wondering,
Did you find something like this on your syslog ?
 %MGBL-NETFLOW-6-INFO_CACHE_SIZE_EXCEEDED : Cache size of 1000000 for monitor xxx has been exceeded

Regards,
Jimmy Hng.

On Tue, May 24, 2016 at 12:02 AM, Robert Williams <Robert at custodiandc.com<mailto:Robert at custodiandc.com>>
wrote:

> Hi,
>
> Doing some more digging, found this from 2014:
>
> Netflow specific scale and Limitations are described below:
>   1. Supports configurable Sampling Rate 1:1 ~ 1: 65535
>   2. Supports only up to 4 Sampling Rates (or Intervals) per Ethernet
> LC LC; no such limit for Enhanced Ethernet LC.
>   3. Up to 4k interfaces/sub-interfaces (4K system limitation) can be
> configured with flow monitor per system.
>   4. Supports up to 8 flow exporters per flow monitor
>   5. Supports up to 1 million flow entries per LC
>   6. Supports up to 50k flows per second with LC CPU usage up to 50%
> per Ethernet LC LC
>   7. Supports upto 100K flows per second with LC CPU usage up to 50%
> per Enhanced Ethernet LC LC
>   8. Netflow scale is increased to 200Kpps on Enhanced Ethernet LC
> based LCs
>   9. Supports exporting packet rates up to 50k flows per second (100K
> flows per sec on Enhanced Ethernet LC based LCs) with LC CPU usage up
> to 50%
>
> "Irrespective of the rate at which the NP punts the records to CPU,
> exporter picks up a maximum of 2000 records at a time from the cache
> that are eligible for export (timers, network/TCP session events,
> etc). This is basically to avoid NetIO dropping the packets due to
> lack of b/w. When the exporter wakes up again, it repeats the same."
>
> So, it can collect 100k flows per second, but can only export 2k each
> time it runs the exporter. The interval for the exporter is unclear however.
>
> I've also found out why this is such an issue on our 9001 but not on
> any of our 900x larger chassis. Looks like on those the hardware punt
> is being limited to 25kpps per NP because we have some BVIs with Netflow on them.
> This causes it to distribute the rate limit for punting to ALL the NPs
> on the LC, even when only two ports are involved in Netflow. Thus,
> it's "sampled sampling" and so the rate of flow data is significantly
> lower than the 9001 which is allowing all 100kpps on one NP which has
> 4 x 10G interfaces punting into it.
>
> mmm...
>
>
>
> Robert Williams
> Custodian Data Centre
> Email: Robert at CustodianDC.com
> http://www.CustodianDC.com
>
> -----Original Message-----
> From: Dale W. Carder [mailto:dwcarder at wisc.edu<mailto:dwcarder at wisc.edu>]
> Sent: 23 May 2016 16:02
> To: Robert Williams <Robert at CustodianDC.com>
> Cc: cisco-nsp at puck.nether.net<mailto:cisco-nsp at puck.nether.net>
> Subject: Re: [c-nsp] A9K Netflow export drops
>
> Thus spake Robert Williams (Robert at CustodianDC.com) on Sat, May 21,
> 2016 at 10:59:50AM +0000:
> >
> > I've got an issue on one of our smaller 9001 boxes which is puzzling me.
> > It suffers from a high rate of netflow export drops (not cache
> > drops)
> shown here:
> >
> > So from what I understand, it is capturing the flows OK but is
> > unable to
> get the flow data out, for some reason.
>
> I can confirm that our 9k's suffer from this also.
>
> The last I checked you can export at the rate of 2000 flows/sec.  I
> have not looked in 2 years or so to see if this limit was configurable
> yet.
>
> > So - what am I missing here? Surely with a cache capability of 1M it
> should be ok to export flows when were are only around 30,000 of them
> nicely ticking over?
>
> join the club.  :-(
>
> Dale
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net<mailto:cisco-nsp at puck.nether.net>
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net<mailto:cisco-nsp at puck.nether.net> https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/