RE: [j-nsp] cflowd sampling

From: Robert O'Hara (rohara@juniper.net)
Date: Wed Nov 07 2001 - 22:21:37 EST


Przemek - your understanding is correct. The setting
for run-length takes effect when the trigger event is
reached, so for a sample of 1 packet in 100, with a run-length
of 3, when you encounter packet number 100, you would sample packet
#100, #101 and #102.

Version 8 cflow introduced aggregation types which now
keep aggregated counters of packets that match certain
characterisitcs, e.g. number of packets coming from a particular
source network prefix/length and forwarded to a particular destination
network/length. These aggregated counters are used for trending network use
and understanding at higher level how traffic is flowing across
your AS. Prior to version 8, the focus was on individual flows.
In my opinion, version 8 cflow seems to offer a better high level view of traffic
flow in the network.

-----Original Message-----
From: Przemyslaw Karwasiecki [mailto:karwas@ifxcorp.com]
Sent: Wednesday, November 07, 2001 5:59 PM
To: Robert O'Hara; juniper-nsp@puck.nether.net
Subject: RE: [j-nsp] cflowd sampling

Bob,

Thanks for your answer.

Please let me know if my understanding is correct:
  case 1. run-length 0 --> only 1/rate packet sampled over all population
  case 2. run-length 1 --> (1+1)/rate packet sampled.

I did some experiments, and in case 2 cflowd was reporting 2% of traffic,
and when I changed my config to case 1 cflowd is reporting 1% of traffic.

Thanks,

Przemek

PS.
I understand that I will never be able to sample all packets,
as actual sampling is done on re connected to switching fabric
via 100BaseT. Also I don't expect that any CPU would be capable
to analyze 2GB/s worth of traffic in real time.

What is confusing here is that this technology inherits its original
name: netflow, while it doesn't have anything to do with Flows any more.

Also definition of run-length (size of sample [with or without original packet])
can be confusing. Particularly after analyzing examples from JunOS documentation:

http://www.juniper.net/techpubs/software/junos50/swconfig50-interfaces/html/sampling-config3.html

-----Original Message-----
From: Robert O'Hara [mailto:rohara@juniper.net]
Sent: Wednesday, November 07, 2001 5:38 PM
To: Stephen Gill; Przemyslaw Karwasiecki; juniper-nsp@puck.nether.net
Subject: RE: [j-nsp] cflowd sampling

Hi,

The default maximum setting for sampling is 1000pps per
active switch fabric. On the M160, because it has four switch
fabs, the default is 4000pps. As Steve has noted in his earlier
email, you can configure the router to provide more than 1000pps
under the [edit forwarding-options] heirarchy, use the
'set sampling max-packets-per-second x', where x is the max number
of packets per second you define. Juniper's recommended guideline
to customers is to stick with the default of 1000pps/switch fabric.
This has been tested and is certified.

Also, while using the run-rate is useful in some circumstances,
if you are truly trying to capture a representative sample
of the flows that are running through the box, then a run-rate
greater than '1' will skew your results.

Thanks,

Bob O'Hara

Systems Engineer
Juniper Networks
Northeast Sales Region

.........................................
. email: rohara@juniper.net .
.........................................
 

-----Original Message-----
From: Stephen Gill [mailto:gillsr@yahoo.com]
Sent: Wednesday, November 07, 2001 4:39 PM
To: 'Przemyslaw Karwasiecki'; juniper-nsp@puck.nether.net
Subject: RE: [j-nsp] cflowd sampling

Unfortunately you are limited by a 100-Mbps bus b/n the RE and PFE
(fxp1). For this reason, the number of packets that you can sample is
limited, thus the need for telling the router what you really want to
classify as interesting. You can be as granular as you would like when
doing so, such as by sampling syn/fin packets, etc... Keep in mind that
there is a built in rate-limiting mechanism of 7000pps no matter how
much you may try to sample.

It will be difficult to measure a full flow (including ack packets) if
you cannot sample ALL traffic. As long as you stay within the built in
limitations of pps you can sample based on filters.

According to the docs on the 'run-length' flag: "Set the number of
samples following the initial trigger event, thus allowing you to sample
adjacent packets to those already being sampled." IE. A run-length of
0 will not sample any other packets in addition to the first one - this
is the behavior you have noticed.

You may also wish to visit the juniper-nsp archives for previous posts
on netflow here: http://puck.nether.net/lists/juniper-nsp/

Juniper has posted a relevant Whitepaper on accounting that you may find
useful here: http://www.juniper.net/techcenter/techpapers/200010.pdf

Cheers,
-- steve

> -----Original Message-----
> From: Przemyslaw Karwasiecki [mailto:karwas@ifxcorp.com]
> Sent: Wednesday, November 07, 2001 10:46 AM
> To: juniper-nsp@puck.nether.net
> Subject: [j-nsp] cflowd sampling
>
> All,
>
> I am looking for some more detail descriptions how traffic sampling
> really work.
>
> I have just setup cflowd with 'rate 100' 'and run-length 1',
> and the results given by cfdnexthops are far different from
> what I would expect. Specifically traffic reported by this utility
> is approximately 2% of traffic which is actually send over each
> of the next hops.
> Because of rate ratio and run-length, I would expect to see 1%
> of traffic to be reported.
>
> Also, cflowd is actually meant to work on flow data, and I don't
> understand how you can identify full flows, from SYN/ACK to FIN
> just by looking at every 100ths packet. Or I am missing something.
>
> Any help, pointers, suggestions, explanations will be greatly
appreciated.
>
> TIA,
>
> Przemek

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com



This archive was generated by hypermail 2b29 : Mon Aug 05 2002 - 10:42:37 EDT