[c-nsp] Nexus 9300 sflow performance
Tim Stevenson (tstevens)
tstevens at cisco.com
Mon May 13 14:06:58 EDT 2019
First gen n9k does not support Netflow at all, only sflow. 2nd gen (EX/FX/FX2) support both, but there is the SPAN+SFlow limitation (we are working on fixing that for FX2, which can theoretically support these concurrently).
For recommended sampling value, we set the rate limiters at values that we feel the switch can handle. So you should select a sampling value where you do NOT see HWRL drops, as then it's truly 1:n. If you drop samples, then your sampling is 1:n up to the point where you tail drop excess packets, and that will skew the samples you actually process and export and reduce the statistical validity of the sample.
Using mgmt0 should be fine for sflow export.
Hope that helps,
Tim
-----Original Message-----
From: Lasse Birnbaum Jensen <lasse at sdu.dk>
Sent: Monday, May 13, 2019 10:58 AM
To: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] Nexus 9300 sflow performance
Im not totally sure about the N9300 architecture. But normally the mgmt interface is connected "directly" to the control plane cpu, thus having it process a lot of packets will take CPU resources and might impact control-plane protocols and jobs. Netflow is performed in the ASICs and I think it would be better to use an ASIC bounded interface if possible.
Best regards
Lasse Birnbaum Jensen
Network Architect
IT-services
T +45 65 50 28 73
M +45 60 11 28 73
lasse at sdu.dk
http://www.sdu.dk/ansat/lbje
University of Southern Denmark
Campusvej 55
DK-5230 Odense M
www.sdu.dk <http://www.sdu.dk/>
Lasse Birnbaum Jensen
D. 20/03/2019 18.27 skrev "cisco-nsp på vegne af Satish Patel" <cisco-nsp-bounces at puck.nether.net på vegne af satish.txt at gmail.com>:
Thanks Tim,
Here is the output of show hardware rate-limiter. ( i believe it's 40k)
This is my first time dealing with SFLOW, Can you share some
configuration parameter i should use for best practice would be great,
What is 1-in-N sample actually?
I am planning to use mgmt0 interface for SFLOW and its 1G so i assume
it will handle all the flow. do you seeing any concern there?
# show hardware rate-limiter
Units for Config: packets per second
Allowed, Dropped & Total: aggregated since last clear counters
Module: 1
R-L Class Config Allowed Dropped Total
+------------------+--------+---------------+---------------+-----------------+
L3 glean 100 0 0 0
L3 mcast loc-grp 3000 0 0 0
access-list-log 100 0 0 0
bfd 10000 0 0 0
exception 50 0 0 0
fex 3000 0 0 0
span 50 0 0 0
dpss 6400 0 0 0
sflow 40000 25134089890 0 25134089890
On Wed, Mar 20, 2019 at 12:07 PM Tim Stevenson (tstevens)
<tstevens at cisco.com> wrote:
>
> Yes, this is 1st gen. The SFLOW/SPAN restriction should not apply there.
>
> Re: 60Gbps/24Mpps and SFLOW, SFLOW does not do aggregation of stats for flows in the switch like netflow does - it's just 1-in-n packet sampling. As such, the value of "n" should be high enough that both the switch & the collector are not overburdened. Note that we will rate limit SFLOW copies to the CPU so that's the first 'bottleneck'. If you end up tail-dropping samples, the statistical validity of your sampled set goes out the window, so you want to ensure that 1-in-n is a number that does not hit that rate limiter.
>
> I don't have a 1st gen switch handy to see what the defaults are for that value. It should show up in 'sh hardware rate-limiter'. In 9300-EX with 9.2.2 it's 40Kpps.
>
> Beyond that, you also want to make sure the collector is able to consume everything coming from all sflow enabled switches without dropping, for the same reason mentioned above.
>
> Hope that helps,
> Tim
>
>
> -----Original Message-----
> From: Satish Patel <satish.txt at gmail.com>
> Sent: Wednesday, March 20, 2019 8:40 AM
> To: Nick Cutting <ncutting at edgetg.com>
> Cc: Tim Stevenson (tstevens) <tstevens at cisco.com>; cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] Nexus 9300 sflow performance
>
> We have cisco Nexus9000 C9396PX
>
> 60 Gbs is data traffic, and 24Mpps ( packet per second ) not sure how
> to convert it into flows. Could you please share your sflow
> configuration if you don't mind?
>
> I had nfsen in past with 8CPU / 4GB memory but it was damn slow :(
> but it could be me.. i will set up again and see if it worth it or
> not.
>
> On Wed, Mar 20, 2019 at 11:34 AM Nick Cutting <ncutting at edgetg.com> wrote:
> >
> > Good point. We waited for the second Gen
> >
> > Regarding 60 Gbs, isn’t that is the data traffic, not the flows or sampled flows levels?
> >
> > Our NFSEn box is centos
> >
> > 4 vCPU and 4 GBrams
> >
> > Collecting flows from maybe only 30 devices, about 20Gbs and 3k flows per sec.
> >
> > -----Original Message-----
> > From: Tim Stevenson (tstevens) <tstevens at cisco.com>
> > Sent: Wednesday, March 20, 2019 11:20 AM
> > To: Nick Cutting <ncutting at edgetg.com>; Satish Patel <satish.txt at gmail.com>; cisco-nsp at puck.nether.net
> > Subject: RE: [c-nsp] Nexus 9300 sflow performance
> >
> > This message originated from outside your organization.
> >
> > Make sure you distinguish between N9300 (1st generation) and N9300-EX/FX/FX2 (2nd generation). The SFLOW + SPAN limitation applies only to the latter. It's also on the latter that Netflow is supported, which can run concurrently with SPAN sessions.
> >
> > Tim
> >
> > -----Original Message-----
> > From: cisco-nsp <cisco-nsp-bounces at puck.nether.net> On Behalf Of Nick Cutting
> > Sent: Wednesday, March 20, 2019 6:19 AM
> > To: Satish Patel <satish.txt at gmail.com>; cisco-nsp at puck.nether.net
> > Subject: Re: [c-nsp] Nexus 9300 sflow performance
> >
> > We use sflow on 9300's, no performance hit - but you cannot use span sessions at the same time.
> >
> > Newer code revisions support netflow, without the SPAN session limitation, although we have not tried netflow on the 9300 yet.
> >
> > For a collector We use NFSEN - opensource, and quite a big install base, and it seems to handle a lot of flows.
> >
> > It supports sflow and netflow as we have a mix, just make sure you add the sflow option at build time as it’s a bit funky old linux to add it after.
> >
> >
> >
> > -----Original Message-----
> > From: cisco-nsp <cisco-nsp-bounces at puck.nether.net> On Behalf Of Satish Patel
> > Sent: Wednesday, March 20, 2019 8:21 AM
> > To: cisco-nsp at puck.nether.net
> > Subject: [c-nsp] Nexus 9300 sflow performance
> >
> > This message originates from outside of your organisation.
> >
> > Folks,
> >
> > I have L3 Nexus 9300 switch which is running 60Gbps traffic on ISP interface so I’m planning to run sflow on that specific interference to get flow.
> >
> > Does it going to create any performances issue on switch?
> >
> > Can I run sflow on Layer 3 LACP interface?
> >
> > Can anyone suggest free open source sflow collector?
> >
> > Sent from my iPhone
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >
> > _______________________________________________
> > cisco-nsp mailing list cisco-nsp at puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
More information about the cisco-nsp
mailing list