[nsp] From the baby to the beast

[Simon Leinen]

On Tue, 10 Dec 2002 14:31:02 -0000, "Darren Smith" <data@barrysworld.com> said:
> 1. Has anyone actually got netflow accounting working on Cisco
> 6509's?  reliably?

We use NetFlow on several Catalyst 6500 SupII (actually 7600 OSRs).
After some initial problems it works great now.  The main trick is to
run 12.1(13)E(1) if you need input/output interface information (we
do).  This release also fills in the AS numbers.

With the new software, NetFlow on the PFC2 is basically equivalent to
NetFlow v5 on the "traditional" platforms (7x00).  There are a few
differences though:

1.) The PFC2 does NetFlow in hardware (in the EARL ASIC), so it can
    count lots and lots of packets.  However, it uses a hash table of
    limited size, and when that table fills up beyond a certain point,
    some packets cannot be accounted for.  The upside is that all
    packets are still forwarded in constant time, i.e. even if you
    have lots of flows, such as during fast port scans, NetFlow
    doesn't slow down the router.

    This may not be a problem for you at all, provided that you don't
    have too many flows.  The recommendation is that the output of
    "show mls ip count" should not usually exceed 32000, and if it
    does you should tune your flow timeouts.  We use these boxes as
    border routers for our national research & education network, and
    I can tune the timeouts all I want and still exceed this value by
    a lot.

    If you want to quantify how much packets aren't being accounted
    for, use the command

        show mls netflow table-contention detailed

    The interesting number are the "Page Misses".  This tells you how
    many packets haven't been accounted for in the MLS NetFlow table.
    The numbers refer to an interval between consecutive
    cleanup/export runs through the MLS NetFlow table; I think this
    interval is four seconds long.

2.) The PFC2 tends to export NetFlow (NDE) packets in bursts.  So
    where your 7401 probably sends you a NetFlow PDU a couple times a
    second, the 6509 will send you a bunch of PDUs back-to-back, then
    nothing for four seconds, then another bunch... so be sure to have
    enough buffer space between the 6509 and your collecting process.
    A fast network, preferably something like a dedicated Gigabit
    Ethernet link between the 6509 and your collector, helps.

3.) Even with 12.1(13)E1, the input/output prefix-length fields aren't
    filled in.  This may be fixed in the not-too-distant future.

4.) Another problem is that if you use NetFlow v5 format (new with
    12.1(13)E), it is hard to distinguish the NetFlow stream from the
    PFC2 from the NetFlow stream from the MSFC (this hopefully doesn't
    contain much data), because they both use engine-type=0 and
    engine-id=0.  This is confusing if you look at sequence numbers to
    detect lost NetFlow packets.  Until Cisco fixes this by setting
    engine-type/engine-id to distinct values, you can work around this
    by exporting v7 packets from the PFC2.  NetFlow v7 contains the
    same amount of information as NetFlow v5 (it just has an
    additional field that I don't quite understand and that might not
    even be useful on this architecture).

5.) Oh yes, and configuration of "NDE" is quite different from normal
    "NetFlow export", although the end result is very similar.  Please
    check the Catalyst 6500-specific documentation.
-- 
Simon.