[j-nsp] j-flow exports upon active-timeout expiration

[Maurizio Molina]

Hi,
I recently migrated the j-flow generation and export from Routing Processor to Multiservice PICs, to be able to use a higher sampling rate (I moved from 1/1000 to 1/100).
My routers are mainly T-series, with JUNOS 9.1R4.4
I'm nothing now the following behaviour associated with long lasting flows, i.e. flows continuously receiving traffic for a long time and whose information get exported via the "active-timeout". At each export time (e.g. for my routers it is 300s) I get a flow record with the packets and bytes received by the flow since the previous export (so, it's "delta" information),  but the "flow start time" information is kept fixed at the time the first packet of the flow was received (in other words, the flow creation time in the flow cache).

This, IMO, creates problem for whatever collector is receiving and analysing this information: regular exports of long lasting flows are useful to timely update all the necessary flow
aggregations that are used to keep control of the network (e.g. per prefix, per router, per protocol, per AS, per application, etc.). When I receive an update, I have to attribute the traffic update to a time bin in the recent past (I believe most flow-analysis applications use this time-binning concept). With the information exported as I described, I don't know exactly to which time span flow information refers to (I know the time of the last packet of the reported information, but the time of the start is NOT the time of the first packet of the reported information, it is the time of the flow creation in the cache, that can be way back in the past!).

A collector will be forced to "guess" this start time by subtracting the  active-timeout value from the time of the last export, but  this is error prone for several reasons:
- juniper themselves say that this export time is not exactly the configured active-timeout, but the upper minute boundary
(http://www.juniper.net/techpubs/software/junos/junos91/swconfig-services/configuringtimers.html#id-11344779)
-if the long lasting flow is expired and exported for other reasons than active-timeout expiration (e.g. for inactive-timeout expiration), the export can happen well before the expected time (e.g. well before the next 300s, in my case....) so I will make a mistake!
In addition to that, I need to keep my flow collector application in sync with the active-timeout definition plus any implementation-specific drift from it (e.g.. the one-minute upper boundary mentioned above).

I'd appreciate comments from people having faced this issue. I also believe that the above behaviour was NOT appearing when I was using the  Routing Processor export, so anybody planning the migration should be aware of this issue.

Regards,
Maurizio

--
______________________________________________________________________

Maurizio Molina
Network Engineer

DANTE - www.dante.net<http://www.dante.net/>

Tel: +44 (0)1223 371 300
Fax: +44 (0)1223 371 371
Email: maurizio.molina at dante.net<mailto:maurizio.molina at dante.org.uk>
PGP Key ID: 3FF58D51

City House, 126-130 Hills Road
Cambridge CB2 1PQ
UK
_____________________________________________________________________