[c-nsp] understanding interface traffic counters of Cisco router and Cisco switch

Nigel Roy nigel at theroys.me.uk
Sun Nov 13 02:06:50 EST 2011 

Martin, 

In my last role, we did some in depth testing for QoS. Some simple tests using specific packet sizes and a bit of maths showed this to be the case. If memory serves it was also confirmed by our Cisco support team!

Regards Nigel 
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Martin T <m4rtntns at gmail.com> wrote:

Saku,
yes, SNMP counters show exactly the same results as interface counters
under CLI.

Counters under CLI:

Cisco2950#show interfaces Fa0/1 | i packets input|packets output
510257 packets input, 773479509 bytes, 0 no buffer
510270 packets output, 773484727 bytes, 0 underruns
Cisco2950#show interfaces Fa0/2 | i packets input|packets output
510232 packets input, 773479047 bytes, 0 no buffer
510229 packets output, 773478942 bytes, 0 underruns
Cisco2950#show interfaces Fa0/23 | i packets input|packets output
510224 packets input, 773476532 bytes, 0 no buffer
510241 packets output, 773482027 bytes, 0 underruns
Cisco2950#show interfaces Fa0/24 | i packets input|packets output
510221 packets input, 773476198 bytes, 0 no buffer
510238 packets output, 773481663 bytes, 0 underruns
Cisco2950#


C3640#show interfaces Fa2/0 | i packets input|packets output
510236 packets input, 771438752 bytes
510232 packets output, 771436264 bytes, 0 underruns
C3640#show interfaces Fa3/0 | i packets input|packets output
510227 packets input, 771437906 bytes
510222 packets output, 771435374 bytes, 0 underruns
C3640#


Counters under SNMP:

Cisco2950:
iso.3.6.1.2.1.2.2.1.10.1 = Counter32: 773479509
iso.3.6.1.2.1.2.2.1.16.1 = Counter32: 773488104
iso.3.6.1.2.1.2.2.1.10.2 = Counter32: 773479047
iso.3.6.1.2.1.2.2.1.16.2 = Counter32: 773478942
iso.3.6.1.2.1.2.2.1.10.23 = Counter32: 773476532
iso.3.6.1.2.1.2.2.1.16.23 = Counter32: 773482545
iso.3.6.1.2.1.2.2.1.10.24 = Counter32: 773476198
iso.3.6.1.2.1.2.2.1.16.24 = Counter32: 773481663

C3640:
iso.3.6.1.2.1.2.2.1.10.1 = Counter32: 771438344
iso.3.6.1.2.1.2.2.1.16.1 = Counter32: 771436264
iso.3.6.1.2.1.2.2.1.10.2 = Counter32: 771437906
iso.3.6.1.2.1.2.2.1.16.2 = Counter32: 771435374


Nigel,
what makes you think that router counters doesn't include FCS? At
least both router interfaces and switch interfaces have CRC field
present..


regards,
martin


2011/11/12 Harold 'Buz' Dale <buz.dale at usg.edu>:
> Only other thing I can think of is autonegotiation...
>_____________________________________________

> From: cisco-nsp-bounces at puck.nether.net [cisco-nsp-bounces at puck.nether.net] On Behalf Of Martin T [m4rtntns at gmail.com]
> Sent: Friday, November 11, 2011 6:37 PM
> To: Erik Soosalu
> Cc: cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] understanding interface traffic counters of Cisco router and Cisco switch
>
> Erik, Harold:
> I already had disabled CDP and BPDU's. At the moment all switch
> interfaces involved in this setup have following configuration:
>
>  switchport access vlan 333
>  switchport mode access
>  switchport nonegotiate
>  no keepalive
>  no cdp enable
>  spanning-tree bpdufilter enable
>  spanning-tree bpduguard enable
>
> ..and spanning-tree on VLAN 333 is disabled("no spanning-tree vlan
> 333"). Updated drawing is here:
> http://img525.imageshack.us/img525/5736/interfacestrafficcounte.png
>
> On top of all this I configured SPAN which had Fa0/1 as a source
> interface and Fa0/3 as a destination one:
>
> monitor session 1 source interface Fa0/1
> monitor session 1 destination interface Fa0/3
>
> ..and PC with "tcpdump -nei eth0 not host 10.10.10.2" running was
> listening port Fa0/3. Throughout the 900 seconds long test(iperf -c
> 10.10.11.2 -u -d -b 20m -t 900) all that tcpdump captured were ARP
> requests and replies. In other words it looks like there are no
> protocols running on the switch which might cause such overhead..
>
>
> In this case, as I mentioned, I did a 900s test with 20Mbps in both
> directions and difference between switch interfaces and router
> interfaces were 0.3% as usual:
>
> Cisco2950#show interfaces Fa0/1 | i packets input|packets output
>     1530640 packets input, 2320402324 bytes, 0 no buffer
>     1530646 packets output, 2320409968 bytes, 0 underruns
> Cisco2950#show interfaces Fa0/2 | i packets input|packets output
>     1530640 packets input, 2320409584 bytes, 0 no buffer
>     1530636 packets output, 2320402068 bytes, 0 underruns
> Cisco2950#show interfaces Fa0/23 | i packets input|packets output
>     1530645 packets input, 2320409904 bytes, 0 no buffer
>     1530641 packets output, 2320402388 bytes, 0 underruns
> Cisco2950#show interfaces Fa0/24 | i packets input|packets output
>     1530636 packets input, 2320402362 bytes, 0 no buffer
>     1530641 packets output, 2320409648 bytes, 0 underruns
> Cisco2950#
>
>
> C3640#show interfaces Fa2/0 | i packets input|packets output
>     1530641 packets input, 2314279824 bytes
>     1530645 packets output, 2314287324 bytes, 0 underruns
> C3640#show interfaces Fa3/0 | i packets input|packets output
>     1530641 packets input, 2314287084 bytes
>     1530635 packets output, 2314279464 bytes, 0 underruns
> C3640#
>
>
> Any additional ideas? :)
>
>
> regards,
> martin
>
>
> 2011/11/11 Erik Soosalu <erik.soosalu at calyxinc.com>:
>> What about all the other control packet stuff that might be running on the switch (CDP, Spanning Tree, VTP, etc)?
>>
>>
>> Thanks,
>> Erik Soosalu
>>
>> -----Original Message-----
>> From: cisco-nsp-bounces at puck.nether.net [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Martin T
>> Sent: Friday, November 11, 2011 2:12 PM
>> To: Christopher J. Pilkington
>> Cc: cisco-nsp at puck.nether.net
>> Subject: Re: [c-nsp] understanding interface traffic counters of Cisco router and Cisco switch
>>
>> Sergey, Christopher:
>> I doubt that it's the VLAN tag which adds this additional 0.3% traffic
>> to switch interface counters when compared to router interface
>> counters. As far as I understand, VLAN tag is added in case when frame
>> leaves the switch via trunk(802.1Q) port, but this is not a case in my
>> test- all the switch ports are in "switchport mode access". Traffic
>> between switch ports in the switch should have no VLAN information
>> applied..
>>
>> Any other ideas? Or am I wrong that traffic inside the
>> "switch-internal-VLAN" has no VLAN tag information?
>>
>>
>> regards,
>> martin
>>
>>
>> 2011/11/11 Christopher J. Pilkington <cjp at 0x1.net>:
>>> Fa0/1 is an access port, not a 802.1q trunk, the traffic on that
>>> interface is not tagged, so the monitor destination will "see"
>>> untagged traffic.
>>>
>>>
>>>
>>> On Nov 10, 2011, at 19:38, Martin T <m4rtntns at gmail.com> wrote:
>>>
>>>> Sergey,
>>>> I modified the setup a little:
>>>>
>>>> http://img64.imageshack.us/img64/5736/interfacestrafficcounte.png
>>>>
>>>> ..so now port Fa0/3 in the switch is in "monitoring" state and all the
>>>> traffic from switch port Fa0/1 is copied to Fa0/3, which is connected
>>>> to eth1 interface on "ubuntu" machine. Now if I start "tcpdump -nei
>>>> eth1 -c10" in "ubuntu" machine in the middle of the iperf test, then
>>>> results are:
>>>>
>>>> root at ubuntu:~# tcpdump -nei eth1 -c10
>>>> tcpdump: WARNING: eth1: no IPv4 address assigned
>>>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>>>> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
>>>> 00:10:30.167558 10:40:10:40:10:40 > 00:06:d7:4d:c0:61, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.10.2.54064 > 10.10.11.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.167563 00:06:d7:4d:c0:61 > 10:40:10:40:10:40, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.11.2.46531 > 10.10.10.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.168556 10:40:10:40:10:40 > 00:06:d7:4d:c0:61, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.10.2.54064 > 10.10.11.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.168805 00:06:d7:4d:c0:61 > 10:40:10:40:10:40, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.11.2.46531 > 10.10.10.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.169805 10:40:10:40:10:40 > 00:06:d7:4d:c0:61, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.10.2.54064 > 10.10.11.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.170055 00:06:d7:4d:c0:61 > 10:40:10:40:10:40, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.11.2.46531 > 10.10.10.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.171054 10:40:10:40:10:40 > 00:06:d7:4d:c0:61, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.10.2.54064 > 10.10.11.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.171303 00:06:d7:4d:c0:61 > 10:40:10:40:10:40, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.11.2.46531 > 10.10.10.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.172304 10:40:10:40:10:40 > 00:06:d7:4d:c0:61, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.10.2.54064 > 10.10.11.2.5001: UDP, length
>>>> 1470
>>>> 00:10:30.172308 00:06:d7:4d:c0:61 > 10:40:10:40:10:40, ethertype IPv4
>>>> (0x0800), length 1512: 10.10.11.2.46531 > 10.10.10.2.5001: UDP, length
>>>> 1470
>>>> 10 packets captured
>>>> 10 packets received by filter
>>>> 0 packets dropped by kernel
>>>> root at ubuntu:~#
>>>>
>>>> In other words it looks like traffic isn't VLAN-tagged("ethertype"
>>>> should be 0x8100 in this case). Or might this be some sort of
>>>> switch-internal VLAN tag?
>>>>
>>>>
>>>> regards,
>>>> martin
>>>>
>>>> 2011/11/10 Sergey Nikitin <oldnick at oldnick.ru>:
>>>>> Hi,
>>>>>
>>>>> Most likely this is because of 802.1Q tag (4 bytes) added to the counter on
>>>>> a switch interface (and obviously you don't see this tag on a router
>>>>> interface). For example, interfaces Fa3/0 and Fa0/24:
>>>>> 773476480 - 771435576 = 2040904
>>>>> 2040904 / 510226 = 4
>>>>>
>>>>> HTH
>>>>>
>>>>> Martin T wrote:
>>>>>>
>>>>>> I made a following setup:
>>>>>>
>>>>>> http://img828.imageshack.us/img828/5736/interfacestrafficcounte.png
>>>>>>
>>>>>> ..and executed "iperf -s -u -fm" in "ubuntu" machine and "iperf -c
>>>>>> 10.10.11.2 -fm -u -d -b 10m -t600" in "PE860" machine. Before the test
>>>>>> I cleared all interface counters. Iperf results were following:
>>>>>>
>>>>>> root at PE860:~# iperf -c 10.10.11.2 -fm -u -d -b 10m -t600
>>>>>>_____________________________________________

>>>>>> Server listening on UDP port 5001
>>>>>> Receiving 1470 byte datagrams
>>>>>> UDP buffer size: 0.12 MByte (default)
>>>>>>_____________________________________________

>>>>>>_____________________________________________

>>>>>> Client connecting to 10.10.11.2, UDP port 5001
>>>>>> Sending 1470 byte datagrams
>>>>>> UDP buffer size: 0.12 MByte (default)
>>>>>>_____________________________________________

>>>>>> [  3] local 10.10.10.2 port 44911 connected with 10.10.11.2 port 5001
>>>>>> [  4] local 10.10.10.2 port 5001 connected with 10.10.11.2 port 49469
>>>>>> [ ID] Interval       Transfer     Bandwidth       Jitter   Lost/Total
>>>>>> Datagrams
>>>>>> [  4]  0.0-600.0 sec    715 MBytes  10.0 Mbits/sec  0.008 ms    0/510205
>>>>>> (0%)
>>>>>> [  4]  0.0-600.0 sec  1 datagrams received out-of-order
>>>>>> [  3]  0.0-600.0 sec    715 MBytes  10.0 Mbits/sec
>>>>>> [  3] Sent 510206 datagrams
>>>>>> [  3] Server Report:
>>>>>> [  3]  0.0-600.0 sec    715 MBytes  10.0 Mbits/sec  0.026 ms
>>>>>> 2/510205 (0.00039%)
>>>>>> [  3]  0.0-600.0 sec  1 datagrams received out-of-order
>>>>>> root at PE860:~#
>>>>>>
>>>>>>
>>>>>> For some reason, the interface counters in switch(Fa0/1, Fa0/2, Fa0/23
>>>>>> and Fa0/24):
>>>>>>
>>>>>> Cisco2950#show interfaces Fa0/1 | i packets input|packets output
>>>>>>     510227 packets input, 773472188 bytes, 0 no buffer
>>>>>>     510236 packets output, 773484380 bytes, 0 underruns
>>>>>> Cisco2950#show interfaces Fa0/2 | i packets input|packets output
>>>>>>     510225 packets input, 773476416 bytes, 0 no buffer
>>>>>>     510223 packets output, 773471932 bytes, 0 underruns
>>>>>> Cisco2950#show interfaces Fa0/23 | i packets input|packets output
>>>>>>     510230 packets input, 773476736 bytes, 0 no buffer
>>>>>>     510233 packets output, 773479832 bytes, 0 underruns
>>>>>> Cisco2950#show interfaces Fa0/24 | i packets input|packets output
>>>>>>     510222 packets input, 773471868 bytes, 0 no buffer
>>>>>>     510226 packets output, 773476480 bytes, 0 underruns
>>>>>> Cisco2950#
>>>>>>
>>>>>> ..show little bit different results than router counters:
>>>>>>
>>>>>> C3640#show interfaces Fa2/0 | i packets input|packets output
>>>>>>     510228 packets input, 771431340 bytes
>>>>>>     510230 packets output, 771435816 bytes, 0 underruns
>>>>>> C3640#show interfaces Fa3/0 | i packets input|packets output
>>>>>>     510226 packets input, 771435576 bytes
>>>>>>     510222 packets output, 771430980 bytes, 0 underruns
>>>>>> C3640#
>>>>>>
>>>>>> I tried this test multiple times with different bandwidth(-b) values
>>>>>> and always the difference between router counters and switch counters
>>>>>> were ~0.3%. If the difference were 1.2% - 1.3% then it would make
>>>>>> sense because probably in this case router counts only up to IP
>>>>>> header, but switch includes L2 header as well, but what might cause
>>>>>> this 0.3% difference?
>>>>>>
>>>>>>
>>>>>> regards,
>>>>>> martin
>>>>>>_____________________________________________

>>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>>>>
>>>>>
>>>>
>>>>_____________________________________________

>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
>>_____________________________________________

>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
>>_____________________________________________

>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>
>_____________________________________________

> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>

_____________________________________________

cisco-nsp mailing list cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

More information about the cisco-nsp mailing list