[c-nsp] TCP behavior under strict CAR rate-limiting
Christopher Hunt
chunt at reachone.com
Thu Jun 19 16:32:35 EDT 2008
Phil,
I do understand that the lower burst (5000) is too low for production
and wouldn't use it. However, I can see that in a 10 second transfer I
can only push about 248Kbytes:
[root at localhost ~]# iperf -c 10.180.55.211 -P 1 -i 1 -t 10
------------------------------------------------------------
Client connecting to 10.180.55.211, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[ 3] local 192.168.10.2 port 33538 connected with 10.180.55.211 port 5001
[ ID] Interval Transfer Bandwidth
...
0.0-10.1 sec 248 KBytes 200 Kbits/sec
And yet the interface counters show an almost insignificant number of drops
(only 65 packets, although that is out of a total of 253 packets) :
Router#show int fa1/0 rate-limit
FastEthernet1/0
Input
matches: all traffic
params: 10000000 bps, 5000 limit, 5000 extended limit
conformed 188 packets, 260984 bytes; action: transmit
exceeded 65 packets, 97206 bytes; action: drop
last packet: 400ms ago, current burst: 2150 bytes
last cleared 00:00:15 ago, conformed 137000 bps, exceeded 51000 bps
So, that would seem to indicate that the interface did, in fact receive 10mbps
plus some burst. The big trouble is that the packets don't seem to be arriving
at the interface at rate. The "show interface" stats (see
http://users.reachone.com/drsmooth/ShowIntSmallBurst.txt ), taken manually
approximately once per second for the length of the 10 second test. Even after
3 seconds the interface has only received 113 packets input (147514 bytes) which
is nowhere near the CAR of 10mbps.
In addition, SNMP seems to reflect that the interface has not received anything
near the CAR (see http://users.reachone.com/drsmooth/smallburst.jpg ). I have
also monitored the packet counters on the testing client (that sends the bulk
data packets). It shows that after approx 3 seconds only 34,060 bytes in 36
packets, or 10,000bps. (As a frustrating aside, SNMP monitoring box of the
testing client interface actually shows a 5 second average of 1.7mbps with 1
second values near 7mbps).
[root at localhost ~]# ./showintstats
09.44.691896144
TX packets:221 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:18784 (18.3 KiB) TX bytes:33538 (32.7 KiB)
09.45.702834271
TX packets:221 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:18784 (18.3 KiB) TX bytes:33538 (32.7 KiB)
09.46.713474652
TX packets:257 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:20448 (19.9 KiB) TX bytes:67598 (66.0 KiB)
09.47.725242663
TX packets:257 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:20448 (19.9 KiB) TX bytes:67598 (66.0 KiB)
09.48.735970243
TX packets:304 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:22426 (21.9 KiB) TX bytes:128316 (125.3 KiB)
09.49.746649199
TX packets:304 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:22426 (21.9 KiB) TX bytes:128316 (125.3 KiB)
09.50.757276807
TX packets:347 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:24012 (23.4 KiB) TX bytes:184214 (179.8 KiB)
09.51.767121736
TX packets:347 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:24012 (23.4 KiB) TX bytes:184214 (179.8 KiB)
09.52.777232744
TX packets:405 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:26352 (25.7 KiB) TX bytes:258700 (252.6 KiB)
09.53.787515822
TX packets:405 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:26352 (25.7 KiB) TX bytes:258700 (252.6 KiB)
09.54.798234461
TX packets:463 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:28288 (27.6 KiB) TX bytes:337288 (329.3 KiB)
09.55.808966342
TX packets:463 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:28288 (27.6 KiB) TX bytes:337288 (329.3 KiB)
09.56.820653126
TX packets:515 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:30476 (29.7 KiB) TX bytes:405452 (395.9 KiB)
09.57.831554207
TX packets:515 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:30476 (29.7 KiB) TX bytes:405452 (395.9 KiB)
09.58.843085966
TX packets:552 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:32074 (31.3 KiB) TX bytes:448022 (437.5 KiB)
Christopher Hunt
Phil Bedard wrote:
> If the normal burst value is too low you may always be exceeding the
> normal burst limit. You can do a show int rate-limit to look at
> current burst values.
>
> Phil
>
> On Jun 19, 2008, at 12:00 PM, Christopher Hunt wrote:
>
>> Sorry,
>> The rate-limit statement that results in 0.2mbps throughput is:
>> rate-limit input 10000000 5000 5000 conform-action transmit
>> exceed-action drop
>>
>> -------- Original Message --------
>> Subject: TCP behavior under strict CAR rate-limiting
>> Date: Thu, 19 Jun 2008 08:11:36 -0700
>> From: Christopher Hunt <chunt at reachone.com>
>> Organization: ReachONE Internet Inc.
>> To: cisco-nsp at puck.nether.net
>>
>>
>>
>> Please excuse me if this is off-topic. I'm needing some expert TCP
>> analysis and not sure where to turn. I'm experimenting with CAR
>> rate-limiting using the Cisco IOS. I am using Iperf to test TCP
>> throughput, PRTG to poll the Cisco interface traffic levels each
>> second and Wireshark to capture the packets. This is a lab, so there
>> are no public IPs in use. Using the rate-limit statement
>> "rate-limit input 10000000 1875000 3750000 conform-action transmit
>> exceed-action drop" I see expected behavior, that is an average of
>> 10mbps with bursts up to ~18mbps.
>> Using the rate-limit statement "rate-limit input 10000000 1875000
>> 3750000 conform-action transmit exceed-action drop" I see an average
>> of 0.02mbps (200 bits/second) with no bursting at all.
>> I expected that the cause would be excessive drops by the Cisco
>> interface causing the TCPs on either end to negotiate a low
>> tcp_receive_window, but the packets don't bear that out. The window
>> sizes are staying high. The cisco interface doesn't show many drops
>> nor does it ever seem to receive the 10mbps rate, even for a second.
>> I expect that even retransmits, duplicate ACKS etc. would increment
>> packet counters on the Cisco.Any ideas what else could cause low
>> throughput besides a low tcp_receive_window?
>> --
>> Christopher Hunt
>>
>>
>>
>> --
>> Christopher Hunt
>> ReachONE Internet, Inc.
>> (888)820-7559
>>
>> _______________________________________________
>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
More information about the cisco-nsp
mailing list