[c-nsp] WARNING: Netflow Data Export & Hardware assisted NAT not supported on 76xx/65xx on the same interface

[Jeff Bacon]

> Nowhere in the Cisco marketing literature, Cisco
> Documentation, or even Cisco bug lists does it mention this.
> There are some caveats listed regarding NDE and NAT (flow
> mask conflicts, and fragments), but even given that, the
> caveats imply that it will work if the caveats don't apply or
> the flowmask conflicts are resolved. Also, there are no
> warnings when configuring it. The feature manager shows no
> errors or conflicts, etc...

Yeah, there aren't a lot of warnings or errors on configuring anything, and unfortunately just because FM doesn't complain about it doesn't mean it actually works. Don't try to use reflexive ACLs either, by the way, unless you don't mind the occasional dropped TCP session.

> I'm looking at using SPAN to replicate the data and send it
> to a linux box to then create netflow data exports, however,
> given the nature of the data (high bandwidth and microburst),
> I'm not sure that the Linux box will work accurately. I
> assumed the PFC would be doing the exports in hardware giving
> us the most accurate realtime look at the market data.
> Evidently I was wrong.

It's only a gig connection. I assume you are only pulling the equities mcast feeds through it and pushing order connections - there's no way OPRA or the Arca options feeds are going to fit through 1G now.

SPAN shouldn't have any issue replicating a 1G connection accurately. You can get the linux box to handle a 1G connection easily enough - you need to tweak the settings a bit, up the ring buffers and possibly pin threads to CPUs, depends on what s/w you want to use on the Linux side - but it should be fine. tcpdump on an average Nehalem will keep up with a gig link no sweat, well enough that I could measure a 600Mbit+/sec microburst to tell a developer why he had to rework his code. 

(I don't use netflow so I can't comment there.) 

<clipping in something from Gert)
>One could try some VRF tricks (NAT in one VRF, netflow in another VRF,
>hardware-loopback from one GigE/VRF-1 to another GigE/VRF-2) on the
>same box, but that's not exactly a clean design.

It's not a clean design, and I also suspect there's no way OP can configure an appropriate setup such that he could get NAT to work in a VRF in the way that he needs. Maybe. I can say this because I had a configuration that looked a lot like that (using phy hairpins and NAT-in-VRF). 

>OTOH, I can't see why the hardware couldn't properly age out NAT
>entries, and then send a NDE record when the NAT entry expires... after
>all, it will have to do NAT state table entry cleanup anyway. 

Because NAT entries are special. Note they show up as "sw-installed" - they're plugged in by the RP. I'm not actually sure who ages those. I don't think it's the hardware. Admittedly, the sw should then know to do the right thing, but a NAT entry has a different life cycle than your average netflow flow. It might be a s/w programming oversight, or maybe s/w can't add entries to the NDE output? *shrug* 


For the record, I gave up on NAT on the 6500. Not because I wanted to - trust me, it's the last thing I wanted - but because VRF-aware NAT doesn't exist and isn't going to exist, NAT on a 6500 is really only reliable in global space, and I also have my 6500s running my MPLS ring.

So I now use a Juniper NS-5200 as a physical hairpin between VRFs on the 6500 to handle NAT duties. It's an ugly hack too, but it's reliable. I could have just as easily used a 6503 or a ME6524 doing NAT in global space - the 6500 does have a lower latency than the Juniper - but the Juniper is simply better at NAT, and there's some other side benefits. Meanwhile, I'm shifting to move boxes that need low latency to have separate interfaces with pub IPs and pushing the exchange routes to them via RIP/quagga. Anything that isn't co-located at the exchange - well, given where NYSE put the exchange, an extra 40usec isn't gonna matter anyway. 

-bacon