[j-nsp] RFC2544 on Juniper MX960 10G ports

[Chris Evans]

Update..

JTAC came back stating they were able to get line rate @ 64bytes on a L3
config, but did get some loss on the l2 configuration. I tested with them to
show my packet loss that I was getting.. Turns out they tested on the 4 x
10Gig line module, I am testing with the combo card. They then tested with a
combo card in their lab and duplicated the results I was seeing..

So to the person asking about the combo card, there is an issue with it..
Once again, this is with 9.6R3.8

On Sun, Mar 14, 2010 at 4:13 PM, Chris Evans <chrisccnpspam2 at gmail.com>wrote:

> Alex,
>
> Thanks for you input. While what you stay is true, there are still issues
> with the MX platform. As stated in my previous postings, I can only push
> line rate or 94.78% max before drops start.. This is that % on a full duplex
> stream.. If I shutdown one of the streams I can then push 100% line rate,
> packet loss is only there when there are full duplex flows.. It's really
> either down to a bug or just the PFE cannot forward the PPS. If you take the
> frame size up to 69 bytes, you can do 100% line rate full duplex.
>
> Putting the MX into a layer2 mode (meaning setting it up as a typical
> ethernet switch) brings another issue with even much larger % of packet loss
> unfortunately.
>
> I am using an IXIA XM12 appliance and IXNetwork.
>
> On Sun, Mar 14, 2010 at 5:04 PM, Alex <alex.arseniev at gmail.com> wrote:
>
>> Chris,
>> Line rate on one 10G port could be different from line-rate on another 10G
>> port because Ethernet is not bit-synchronous.
>> LAN-PHY 10GBASE-S allowed transmitter clock tolerance is 10.3125Gbd +/1
>> 100ppm (parts per million) and LAN-PHY 10GBASE-S allowed receiver clock
>> tolerance is also 10.3125Gbd +/- 100ppm. See IEEE 802.3-2008 spec sections
>> 52.5.1 and 52.5.2.
>> So in reality the Rx on ingress port could run at 10.31353125Gbd and Tx on
>> egress port could run at 10.31146875Gbd. This 0.0020625Gbd=2.0625Mbd
>> difference causes ingress buffer to grow until there is no more room and
>> eventually an "overspill" and packet drop. Please run Your tests at 99% line
>> rate, I am sure there will be no packet loss at all.
>> Rgds
>> Alex
>>
>>
>> ----- Original Message ----- From: "Chris Evans" <
>> chrisccnpspam2 at gmail.com>
>> To: "Joerg Staedele" <js at tnib.de>
>> Cc: "juniper-nsp" <juniper-nsp at puck.nether.net>
>> Sent: Sunday, March 14, 2010 7:28 PM
>>
>> Subject: Re: [j-nsp] RFC2544 on Juniper MX960 10G ports
>>
>>
>>  Joerg,
>>>
>>> The hardware we have in our lab is the 20xSFP + 2x10Gig.. JTAC says this
>>> 'should' work but obviously it doesn't.. I tested it on an EX switch and
>>> it
>>> had no issues.. In a simple L2 mode the MX lost about 47% packets at
>>> 64byte
>>> 10Gig line rates. In L3 mode is lost about 5.2%.. This is when testing
>>> full
>>> duplex flows. This was with 9.6R3.8.. There is a known PR related to this
>>> issue.
>>>
>>> Hope to have some resolution sometime this week..
>>>
>>> On Sun, Mar 14, 2010 at 3:14 PM, Joerg Staedele <js at tnib.de> wrote:
>>>
>>>  Hi,
>>>>
>>>> so this means that this Linecard is not able to do line-rate forwarding
>>>> with small frame sizes? What about other cards (20xSFP+2x10G) .. I guess
>>>> they use exactly the same PFE hardware? So they have this limitation
>>>> aswell?
>>>>
>>>> I am really confused now because in every document you read that the
>>>> DPCE's
>>>> are able to do line-rate at any frame-size?
>>>>
>>>> Regards,
>>>>  Joerg
>>>>
>>>> -----Original Message-----
>>>> From: juniper-nsp-bounces at puck.nether.net [mailto:
>>>> juniper-nsp-bounces at puck.nether.net] On Behalf Of Jonathan Lassoff
>>>> Sent: Sunday, March 14, 2010 6:55 PM
>>>> To: Serge Vautour
>>>> Cc: juniper-nsp
>>>> Subject: Re: [j-nsp] RFC2544 on Juniper MX960 10G ports
>>>>
>>>> Excerpts from Serge Vautour's message of Thu Feb 18 16:28:44 -0800 2010:
>>>> > Hello,
>>>> >
>>>> > We recently used a traffic generator to run RFC2544 tests against a
>>>> Juniper MX960. The 1G ports work flawlessly. 0% packet loss at all frame
>>>> sizes.
>>>> >
>>>> > The 10G ports  (4x10G "R" card) didn't do as well. They dropped up to
>>>> > 25%
>>>> packets with certain small frames (ex: 70 byte frames). The packet loss
>>>> goes
>>>> away almost completely for frames larger than 100 bytes. Our SE tells us
>>>> this is normal and is due to how the MX chops the frames up into 64 byte
>>>> cells inside the PFE. The 4x10G cards have 4 separate PFEs (1 per 10G
>>>> port)
>>>> and each of them has 10G of bandwidth. 10G of small frames essentially
>>>> creates more than 10G of traffic inside the PFE. That explanation may
>>>> not be
>>>> 100% correct but I think it paints the right picture.
>>>> >
>>>> > Now the questions. Is this a problem on production networks with real
>>>> world traffic? What about on VPN networks with alot of small frames like
>>>> VoIP? Has anyone seen this problem creep it's head in production?
>>>>
>>>> Isn't the minimum Ethernet frame size 64 bytes? I think Ethernet II /
>>>> Ethernet 802.3 requires this.
>>>>
>>>> Wouldn't this make the problem moot if you're just running Ethernet?
>>>>
>>>> Might be a problem with small ATM cells?
>>>>
>>>> Cheers,
>>>> jof
>>>> _______________________________________________
>>>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>>>
>>>>
>>>> _______________________________________________
>>>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>>>
>>>>  _______________________________________________
>>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>>
>>>
>>
>