[c-nsp] jumbo MTU on WS-X6704-10GE
David Sinn
dsinn at dsinn.com
Wed Jul 5 23:30:57 EDT 2006
Inline
On Jul 5, 2006, at 2:36 PM, Ian Cox wrote:
>
> There is no "giant" counter for a vlan interface since it is a
> logical interface.
No disagreement.
> The accounting of a packet being a giant or not is done against the
> physical interface the packet is sent or received on.
I would expect that, but that's not what I'm seeing for Jumbo's when
using .1q on 6704's under 12.2(18)SXF4. I only see the giants
increment on non-.1q interfaces in the path.
> You can see the port counters for the WS-X cards via show counters
> int ten x/y, and compare the oversize pkt counter there to what is
> reported on show interface x/y.
Below is the client facing interface where a 1x20kpps 9000-byte data
stream along with a 1x20kpps 1500-byte data stream enters my
network. (The 1500-byte stream is hair-pinning back down the same
interface on the other VLAN, hence the outbound 20kpps value.)
TenGigabitEthernet2/2 is up, line protocol is up (connected)
Hardware is C6k 10000Mb 802.3, address is 0014.1c38.bad6 (bia
0014.694c.cf39)
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 6/255, rxload 40/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 10Gb/s
input flow-control is off, output flow-control is off
ARP type: ARPA, ARP Timeout 04:00:00
Last input never, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/30991/0 (size/max/drops/flushes); Total output
drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1604259000 bits/sec, 41682 packets/sec
5 minute output rate 255307000 bits/sec, 20997 packets/sec
60467394548 packets input, 295640517510295 bytes, 0 no buffer
Received 14505692810 broadcasts (1615578523 multicasts)
0 runts, 29065 giants, 0 throttles
30991 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
40954542071 packets output, 137035929772498 bytes, 0 underruns
0 output errors, 0 collisions, 3 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
32 bit counters:
0. rxCRCAlignErrors = 0
1. rxUndersizedPkts = 0
2. rxOversizedPkts = 29065
3. rxFragmentPkts = 0
4. rxJabbers = 0
5. txCollisions = 0
6. ifInErrors = 30991
7. ifOutErrors = 0
8. ifInDiscards = 30991
9. ifInUnknownProtos = 30991
10. ifOutDiscards = 0
11. txDelayExceededDiscards = 0
12. txCRC = 0
13. linkChange = 3
14. wrongEncapFrames = 30991
But the next hop up, which is not a .1q interface, is reflecting the
giants on it's inbound interface:
TenGigabitEthernet3/4 is up, line protocol is up (connected)
Hardware is C6k 10000Mb 802.3, address is 000f.34b2.f480 (bia 000f.
34b2.f480)
Internet address is 198.48.76.1/30
MTU 9216 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 34/255
Encapsulation ARPA, loopback not set
Keepalive not set
Full-duplex, 10Gb/s
input flow-control is off, output flow-control is off
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/1386/9 (size/max/drops/flushes); Total output
drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 1348313000 bits/sec, 20682 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
L2 Switched: ucast: 16706 pkt, 1899373 bytes - mcast: 993577 pkt,
98910667 bytes
L3 in Switched: ucast: 9666618823 pkt, 76802712446329 bytes -
mcast: 3740031 pkt, 30495654993 bytes mcast
L3 out Switched: ucast: 180250744 pkt, 41134494588 bytes mcast:
7889315942 pkt, 10759980763094 bytes
10017733342 packets input, 77232077437968 bytes, 20 no buffer
Received 7229020 broadcasts (3741768 IP multicasts)
0 runts, 820576021 giants, 20 throttles
2 input errors, 1 CRC, 1 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
0 input packets with dribble condition detected
4249133500 packets output, 5472930262718 bytes, 0 underruns
0 output errors, 0 collisions, 4 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped out
32 bit counters:
0. rxCRCAlignErrors = 1
1. rxUndersizedPkts = 0
2. rxOversizedPkts = 820992473
3. rxFragmentPkts = 0
4. rxJabbers = 0
5. txCollisions = 0
6. ifInErrors = 2
7. ifOutErrors = 0
8. ifInDiscards = 2
9. ifInUnknownProtos = 0
10. ifOutDiscards = 0
11. txDelayExceededDiscards = 0
12. txCRC = 0
13. linkChange = 9
14. wrongEncapFrames = 0
Thanks!
David
> Ian
>
>
> At 01:49 PM 7/5/2006 -0700, David Sinn wrote:
>> So, your comments made me re-look at what I'm seeing and while it may
>> not be related to configuration changes as I first thought, I'm not
>> seeing giant's incrementing where I would expect based on the
>> comments.
>>
>> I have two interfaces that are .1q'ed and neither are showing giant
>> counters incrementing. One is with dot-sub-interfaces, the other is
>> a more traditional trunk going to VLAN interfaces. One is currently
>> pushing 40Kpps of 9k packets, and the other was receiving 9k packets
>> earlier in the week (troubleshooting a jumbo clean path... ;)
>>
>> TenGigabitEthernet2/2 is up, line protocol is up (connected)
>> 5 minute input rate 1604184000 bits/sec, 41687 packets/sec
>> 5 minute output rate 255314000 bits/sec, 21001 packets/sec
>> 0 runts, 29065 giants, 0 throttles <----- Not incrementing.
>> Vlan60 is up, line protocol is up
>> 5 minute input rate 1617612000 bits/sec, 42076 packets/sec
>> 5 minute output rate 1000 bits/sec, 1 packets/sec
>> L2 Switched: ucast: 161486 pkt, 850433676 bytes - mcast: 4441382
>> pkt, 299456186 bytes
>> L3 in Switched: ucast: 45629470992 pkt, 176148351510954 bytes -
>> mcast: 7271147944 pkt, 59289495996248 bytes mcast
>> L3 out Switched: ucast: 60201400 pkt, 20653413425 bytes mcast:
>> 4054464289 pkt, 33087720212902 bytes
>> 0 runts, 0 giants, 48 throttles
>> Vlan61 is up, line protocol is up
>> 5 minute input rate 0 bits/sec, 0 packets/sec
>> 5 minute output rate 248949000 bits/sec, 20531 packets/sec
>> L2 Switched: ucast: 715 pkt, 47876 bytes - mcast: 752824 pkt,
>> 51563616 bytes
>> L3 in Switched: ucast: 127053321 pkt, 13294286484 bytes - mcast:
>> 7229010760 pkt, 58960443776240 bytes mcast
>> L3 out Switched: ucast: 36734974702 pkt, 103607468431523 bytes
>> mcast: 0 pkt, 0 bytes
>> 0 runts, 0 giants, 48 throttles
>>
>>
>> The interfaces I earlier attributed to having been "modified" are
>> also different because they are routed, not trunk interfaces:
>>
>> (The next router "up the stack: from the router above)
>> TenGigabitEthernet3/4 is up, line protocol is up (connected)
>> 5 minute input rate 1348287000 bits/sec, 20685 packets/sec
>> 5 minute output rate 4000 bits/sec, 4 packets/sec
>> L2 Switched: ucast: 16565 pkt, 1887327 bytes - mcast: 985198 pkt,
>> 98077197 bytes
>> L3 in Switched: ucast: 9163641553 pkt, 72704490444324 bytes -
>> mcast: 3740031 pkt, 30495654993 bytes mcast
>> L3 out Switched: ucast: 180121319 pkt, 41103115390 bytes mcast:
>> 7889315942 pkt, 10759980763094 bytes
>> 0 runts, 317600238 giants, 20 throttles <--- Incrementing at
>> 20KPPS
>>
>> All of these are 6704-10GE ports.
>>
>> Are the giants on the VLAN ports being masked?
>>
>> David
>>
>>
>> On Jul 5, 2006, at 1:11 PM, Ian Cox wrote:
>>
>>> At 01:52 PM 7/5/2006 -0500, Charles Spurgeon wrote:
>>>> I would expect to see the Ethernet oversized error counter
>>>> increment
>>>> for every frame received that was larger than 1518 bytes (make that
>>>> 1522 bytes for frames with a VLAN tag in them). The Ethernet
>>>> standard
>>>> specifies that any frame larger than the max in the standard is an
>>>> oversize error.
>>>>
>>>> Jumbo frames are not part of the Ethernet standard. Therefore, the
>>>> frame counters below are correctly incrementing for non-standard
>>>> oversized frames.
>>>>
>>>> Presumably the 1GigE interface should have been incrementing the
>>>> oversized counters as well. But since you report in a followup
>>>> that
>>>> the counter isn't incrementing, then perhaps it's a bug in the
>>>> 1GigE
>>>> int code.
>>>>
>>>> Or it might be that when a non-standard Ethernet MTU is configured
>>>> they decided not to increment the oversized frame error counter to
>>>> avoid getting worried calls from the customers. Only Cisco knows
>>>> for
>>>> sure...
>>>
>>> If a frame is greater than 1518 / 1522 with a 1q tag, the giant
>>> counter is incremented. Only the WS-X6148 cards at present have
>>> additional counters that take into configured MTU on the interface.
>>>
>>>
>>> Ian
>>>
>>>> Thanks,
>>>>
>>>> -Charles
>>>>
>>>> Charles E. Spurgeon / UTnet
>>>> UT Austin ITS / Networking
>>>> c.spurgeon at its.utexas.edu / 512.475.9265
>>>>
>>>> On Wed, Jul 05, 2006 at 01:02:07PM +0100, A.L.M.Buxey at lboro.ac.uk
>>>> wrote:
>>>>> hi,
>>>>>
>>>>> run into an interesting one, wonder if someone can help.
>>>>>
>>>>> we have several 10GbE links, run from WS-X6704-10GE modules
>>>>> on Sup720 powered 6509e chassis. 2 of them dont have any
>>>>> requirement
>>>>> to carry jumbo frames...and on those we see no problem. however,
>>>>> one of the links is taking over from a 1GbE link (fed from WS-
>>>>> X6724-SFP
>>>>> modules) which had 9000 MTU traffic running through it. from the
>>>>> specs, the 10GbE module has a max of 9216..which is what the
>>>>> system jumbomtu has been set for..as well as an instruction on the
>>>>> interface to handle...so, the config for the interface looks like:
>>>>>
>>>>> interface TenGigabitEthernet2/1
>>>>> description 10GbE jumbomtu interlink
>>>>> mtu 9216
>>>>> no ip address
>>>>> logging event link-status
>>>>> switchport
>>>>> switchport trunk encapsulation dot1q
>>>>> switchport trunk native vlan 708
>>>>> switchport trunk allowed vlan 100,120,280,708,736
>>>>> switchport mode trunk
>>>>>
>>>>> however, in the counters, we see this:
>>>>>
>>>>> 32 bit counters:
>>>>> 0. rxCRCAlignErrors = 0
>>>>> 1. rxUndersizedPkts = 0
>>>>> 2. rxOversizedPkts = 203234
>>>>> 3. rxFragmentPkts = 0
>>>>> 4. rxJabbers = 0
>>>>> 5. txCollisions = 0
>>>>> 6. ifInErrors = 37
>>>>> 7. ifOutErrors = 0
>>>>> 8. ifInDiscards = 37
>>>>> 9. ifInUnknownProtos = 37
>>>>> 10. ifOutDiscards = 0
>>>>> 11. txDelayExceededDiscards = 0
>>>>> 12. txCRC = 0
>>>>> 13. linkChange = 0
>>>>> 14. wrongEncapFrames = 37
>>>>>
>>>>>
>>>>> on our other links, which dont have jumbo/mtu , all these
>>>>> values are
>>>>> a nice , beautiful '0'..which is exactly what we like to see :-)
>>>>>
>>>>> can anyone shed some info on this peculiarity? IOS is 12.2(17d)
>>>>> SXB
>>>>> on all involved routers (ie those with happy 10GbE and those with
>>>>> errors)
>>>>>
>>>>> alan
>>>>> _______________________________________________
>>>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>> _______________________________________________
>>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>> _______________________________________________
>>> cisco-nsp mailing list cisco-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : https://puck.nether.net/pipermail/cisco-nsp/attachments/20060705/02872226/attachment-0001.bin
More information about the cisco-nsp
mailing list