[j-nsp] Juniper "firewall policer" inner workings

[Martin T]

Stefan,
I see. If policer counts in the IPv4 header as well, it would do
(51021*28)/(1024*1024)=1.4MB which is rather close to 71.5-69.8=1.7MB.
Could you please explain this "larger buffer to smooth things out"
argument? As I understand, in simple terms, larger buffer is able to
hold larger amount of received data in memory so in case of a burst,
more data is held in the memory buffer and processed bit later. If the
buffer is very small, buffer memory would be filled fast and even a
short burst would be noticed as a packet loss. Right?


Massimo,
platform is M10i(10.4R3.4) which has CFEB with part number
750-010465(Internet Processor II). On a software side, it's:

root> show pfe version extensive
PFED release 10.4R3.4 built by builder on 2011-03-19 21:13:59 UTC
    warth.juniper.net:/volume/build/junos/10.4/release/10.4R3.4/obj-i386/junos/usr.sbin/pfed

root>


Christopher,
as in discussion with Stefan turned out, the policer indeed seems to
count the whole IP packet(including the L3 header). This "There is
also granularity in the policer.  When it drops, it cannot drop a
fractional packet...." is a good point as well. Thanks!


Chris,
Iperf UDP is indeed very bursty- I ran "iperf -c 192.168.2.1 -u -fm
-t60 -d -b 10m" and at the same time did "tcpdump -w iperf_dump.pcap
host 192.168.2.1". Then printed deltas between current and previous
line on each dump line(-ttt), printed only this first deltas
column(awk '{print $1}'), sorted those deltas by numerical value(sort
-n) and finally print uniq values with the count of the number of
times the line occurred in the dumpfile. Most popular values are
following:

[root@ ~]# tcpdump -ttt -r iperf_dump.pcap | awk '{print $1}' | sed
's/.*\.//' | sort -n | uniq -c | egrep ^[0-9]{4}
reading from file iperf_dump.pcap, link-type EN10MB (Ethernet)
1082 000010
11344 000011
12154 000012
4037 000013
1902 000014
1278 000015
1056 001173
1224 001174
1465 001175
1548 001176
1398 001177
1059 001178
[root@ ~]#


However, if I ran "nuttcp -4 -u -Ri10M -v -T1m -w2m 192.168.2.1" and
at the same time did "tcpdump -w nuttcp_dump.pcap host 192.168.2.1",
the most popular delta values between packets are way higher than with
Iperf:

[root@ ~]# tcpdump -ttt -r nuttcp_dump.pcap | awk '{print $1}' | sed
's/.*\.//' | sort -n | uniq -c | less | egrep ^[0-9]{4}
reading from file nuttcp_dump.pcap, link-type EN10MB (Ethernet)
2488 000817
12746 000818
22039 000819
15433 000820
4837 000821
[root@ ~]#

As you said, in case of nuttcp UDP flood, packets are much more evenly sent :)


Could you explain this "Your interface speed appears to be GigE, so
you are bursting out at line rate and relying on the buffers in the
router to accommodate these line-rate bursts.  You should have
different results if you connected the host at FastE vs GigE." a bit
further? I mean why should I see different results with 100BASE-TX or
10BASE-T?

In addition, how to start nuttcp in both directions simultaneously?
"nuttcp -4 -u -Ri10M -v -T1m -w2m 192.168.2.1" only sends from my
nuttcp client to nuttcp server.


regards,
martin


2011/4/4 Chris Tracy <ctracy at es.net>:
> Martin,
>
>> It might also be an idea to measure using different values of burst size.
>> I personally find the Juniper manuals to be somewhat lacking here...
>
> You may want to try a perf testing application which is less bursty.  iperf UDP traffic is very bursty -- it does not go out of its way to evenly space packets.  You can easily see this by capturing the perf traffic, then processing it with something like tcpdump -ttt -r file.pcap |  awk '{print $1}' | sed -e 's/00:00:00.//' | sort -n | uniq -c > deltas.  In the 'deltas' histogram that results you should see the overwhelming majority of the packets in your flow are spaced extremely close together (e.g., near the 0 end).
>
> Your interface speed appears to be GigE, so you are bursting out at line rate and relying on the buffers in the router to accommodate these line-rate bursts.  You should have different results if you connected the host at FastE vs GigE.
>
> nuttcp is a perf testing tool which, by default, is much less bursty -- you can control whether it bursts or not.  Go to www.nuttcp.org or grab the latest rev from http://lcp.nrl.navy.mil/nuttcp/beta/nuttcp-7.1.3.c.
>
> Use the -Ri10M option for a non-bursty 10Mbps flow.  You can add /# to make it burst # number of packets.  (This can be useful for seeing how "much" bursty traffic a device can take before it exhausts its buffers when there is a speed transition, for example..)
>
>        -Ri#[/#] instantaneous rate limit with optional packet burst
>
> If you look closely at the CPU usage of iperf vs nuttcp, you will find nuttcp (unlike iperf) consumes 100% of the sender's CPU when running in UDP mode because it puts itself into a very tight loop to get the most precise timing.
>
> -Chris
>
> --
> Chris Tracy <ctracy at es.net>
> Energy Sciences Network (ESnet)
> Lawrence Berkeley National Laboratory
>
>