[c-nsp] A9K Netflow export drops

Tue Jun 14 08:07:01 EDT 2016

Hi Robert,

we've finally received clarification from TAC:
In our case this was a bug within IOS-XR 5.3.X.
For us, this is fixed in 6.0.1 which we wanted to upgrade to anyway due to
extended netconf support.

hth,
Chris

On Wed, May 25, 2016 at 5:05 PM, Robert Williams <Robert at custodiandc.com>
wrote:

> Chris - Thanks, I've not yet opened one, but would be curious to hear the
> outcome of yours as it may save doubling up.
>
> Jimmy - If I take the sampling to 1:1 then yes I can achieve this event,
> however, we appear to be at the limit of the 'exporter' not the
> 'monitor/cache' so at present the cache is not being exceeded.
>
> Cheers guys!
>
>
>
> Robert Williams
> Custodian Data Centre
> Email: Robert at CustodianDC.com
> http://www.CustodianDC.com
>
> -----Original Message-----
> From: cisco-nsp [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of
> Jimmy
> Sent: 23 May 2016 17:16
> To: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] A9K Netflow export drops
>
> Hi,
> Just wondering,
> Did you find something like this on your syslog ?
>  %MGBL-NETFLOW-6-INFO_CACHE_SIZE_EXCEEDED : Cache size of 1000000 for
> monitor xxx has been exceeded
>
> Regards,
> Jimmy Hng.
>
>
> On Tue, May 24, 2016 at 12:02 AM, Robert Williams <Robert at custodiandc.com>
> wrote:
>
> > Hi,
> >
> > Doing some more digging, found this from 2014:
> >
> > Netflow specific scale and Limitations are described below:
> >   1. Supports configurable Sampling Rate 1:1 ~ 1: 65535
> >   2. Supports only up to 4 Sampling Rates (or Intervals) per Ethernet
> > LC LC; no such limit for Enhanced Ethernet LC.
> >   3. Up to 4k interfaces/sub-interfaces (4K system limitation) can be
> > configured with flow monitor per system.
> >   4. Supports up to 8 flow exporters per flow monitor
> >   5. Supports up to 1 million flow entries per LC
> >   6. Supports up to 50k flows per second with LC CPU usage up to 50%
> > per Ethernet LC LC
> >   7. Supports upto 100K flows per second with LC CPU usage up to 50%
> > per Enhanced Ethernet LC LC
> >   8. Netflow scale is increased to 200Kpps on Enhanced Ethernet LC
> > based LCs
> >   9. Supports exporting packet rates up to 50k flows per second (100K
> > flows per sec on Enhanced Ethernet LC based LCs) with LC CPU usage up
> > to 50%
> >
> > "Irrespective of the rate at which the NP punts the records to CPU,
> > exporter picks up a maximum of 2000 records at a time from the cache
> > that are eligible for export (timers, network/TCP session events,
> > etc). This is basically to avoid NetIO dropping the packets due to
> > lack of b/w. When the exporter wakes up again, it repeats the same."
> >
> > So, it can collect 100k flows per second, but can only export 2k each
> > time it runs the exporter. The interval for the exporter is unclear
> however.
> >
> > I've also found out why this is such an issue on our 9001 but not on
> > any of our 900x larger chassis. Looks like on those the hardware punt
> > is being limited to 25kpps per NP because we have some BVIs with Netflow
> on them.
> > This causes it to distribute the rate limit for punting to ALL the NPs
> > on the LC, even when only two ports are involved in Netflow. Thus,
> > it's "sampled sampling" and so the rate of flow data is significantly
> > lower than the 9001 which is allowing all 100kpps on one NP which has
> > 4 x 10G interfaces punting into it.
> >
> > mmm...
> >
> >
> >
> > Robert Williams
> > Custodian Data Centre
> > Email: Robert at CustodianDC.com
> > http://www.CustodianDC.com
> >
> > -----Original Message-----
> > From: Dale W. Carder [mailto:dwcarder at wisc.edu]
> > Sent: 23 May 2016 16:02
> > To: Robert Williams <Robert at CustodianDC.com>
> > Cc: cisco-nsp at puck.nether.net
> > Subject: Re: [c-nsp] A9K Netflow export drops
> >
> > Thus spake Robert Williams (Robert at CustodianDC.com) on Sat, May 21,
> > 2016 at 10:59:50AM +0000:
> > >
> > > I've got an issue on one of our smaller 9001 boxes which is puzzling
> me.
> > > It suffers from a high rate of netflow export drops (not cache
> > > drops)
> > shown here:
> > >
> > > So from what I understand, it is capturing the flows OK but is
> > > unable to
> > get the flow data out, for some reason.
> >
> > I can confirm that our 9k's suffer from this also.
> >
> > The last I checked you can export at the rate of 2000 flows/sec.  I
> > have not looked in 2 years or so to see if this limit was configurable
> > yet.
> >
> > > So - what am I missing here? Surely with a cache capability of 1M it
> > should be ok to export flows when were are only around 30,000 of them
> > nicely ticking over?
> >
> > join the club.  :-(
> >
> > Dale
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>