[j-nsp] Going Juniper

Tue Apr 10 12:39:36 EDT 2018

FSVO tens Gbps and easily.

I think INTC is quoting DPDK for hundreds of Gbps for latest XEONs.

But everything depends on what you're actually doing, as it's
run-to-completion it has highly variant pps performance depending on
what is being done. Do you have filters? How large FIB? uRPF?

10Gbps is 15Mpps on smallest size frames.

Also XEON isn't actually in any meaningful way cheap lookup chip, cost
per Gbps is very large compared to BRCM stuff.

But I can understand in some environments the compute is sunk cost, so
it doesn't really matter and the the pps requirement is very modest.
So there are use cases, but probably not as much as people intuitively
think.

On 10 April 2018 at 19:06, Josh Reynolds <josh at kyneticwifi.com> wrote:
> There are x86 based routing platforms doing many tens of Gbps easily in
> software. Things like vpp.io, DPDK, and others are driving things like FRR,
> Cumulus, and now TNSR drastically forward.
>
> On Tue, Apr 10, 2018, 10:40 AM <mike+jnsp at willitsonline.com> wrote:
>
>> On 04/09/2018 08:07 PM, Chris via juniper-nsp wrote:
>> > Hi,
>> >
>> > On 10/04/2018 9:45 AM, mike+jnsp at willitsonline.com wrote:
>> >>      I see there is a terrific amount of used mx104 and mx240 out there
>> >> and the specs all seem great. What I'm looking to do is have 2x 10g
>> >> feeds, route bgp, do flow exporting, and do a certain amount of ingress
>> >> filtering to protect the network from ddos.Id even like to do cgnat for
>> >> up to 5000 users but not sure if a single box setup would be wise.
>> >
>> > I can't speak for the MX240, but we have some deployments of the
>> > MX104, MX80 and the vMX.
>> >
>> > For the MX104 (and the MX80) the main limitation they have is that the
>> > CPU on the routing engine is terribly slow. This can be a problem for
>> > you if you are taking multiple full tables with BGP. Even without
>> > taking full tables, the RE CPU on the MX104's I have is basically
>> > always at 100%. Commits are pretty slow as well. This shouldn't be
>> > such an issue with the MX240 as it has a wider range of routing
>> > engines available with much better specs.
>> >
>> > The MX104's (and MX80's) have the MS-MIC-16G installed. We use the
>> > MS-MIC-16G for IPSEC, NAT and stateful firewalling (service filters
>> > are used to only send certain traffic to the stateful firewall). So
>> > far there has only been 1 issue that I have personally encountered
>> > with the MS-MIC-16 - the card has crashed on a previous release of
>> > JunOS when adding a large number of IPSEC peers. Since upgraded I have
>> > not experienced the same issue though.
>> >
>> > I also have some vMX's deployed (they are running on top of Dell
>> > R740's with 3 x Intel X710 cards to give 12 x 10G interfaces). The
>> > painful part on getting the vMX to work was the host setup with KVM -
>> > the documents are severly lacking on Junipers side (but I have written
>> > up the exact instructions to get the most recent 18.1R1 release
>> > working on CentOS with no issues).
>> >
>> > So far after getting the issues with the KVM host ironed out I have
>> > been very happy with the performance of the vMX. Since 17.4R1 you can
>> > deploy a virtual MS-MPC (which requires extra CPU resources) which
>> > will give you NAT support as well as stateful firewalling support.
>> > Since its virtualised and the RE runs as a seperate VM you can assign
>> > more or less resources to it as needed - I have 16G of RAM allocated
>> > with 6 cores and the time to process/install a full table is only a
>> > few seconds. They have survived some DDoS attacks that were large
>> > enough to fill up the transit links with no issues as well. The
>> > biggest thing is to make sure you get NIC's that support SR-IOV and
>> > make sure the CPU is fast enough/has enough cores for your
>> > requirements (you cannot over-allocate the cores!). For my use case, I
>> > don't think I will be buying any more physical MX's unless I have an
>> > actual reason to need their hardware, the vMX suites my needs just
>> > fine. Juniper does provide a (limited) demo of the vMX, happy to send
>> > you the install guide I wrote up for getting it working on KVM with
>> > CentOS 7.4 (Ubuntu is also supported for KVM but the install process
>> > is basically terrible).
>>
>> I have a lot of experience and confidence running embedded linux as a
>> router doing bgp/ospf and so forth. Also running pfsense in virtual
>> machines for various features. Never knew juniper had a software only
>> option, that is pretty interesting to me.
>>
>> I know it can be set up and run like a champ and do some (undefined)
>> number of gigabits without issue. What concerns me is that there are
>> performance limitations in these software only platforms based on your
>> processor/bus/card choices, and of course the performance of a software
>> hash vs a hardware cam for forwarding table lookups. And usually
>> (imho),  you hit the platform limits like a truck running into a brick
>> wall. However, if I knew I was only going to have just a few gbps (3?),
>> I likely would be very interested doing a live deployment. However, with
>> that said, it certainly is interesting enough to investigate and I'd
>> love to see your writeup. At a minimum it sounds very useful and I may
>> use vMX for pre-deployment testing purposes.
>>
>> On your mx104 you said cpu was pegged at %100 - operationally does this
>> cause you any grief? How long does it take for your routes to recover
>> after a peer flaps? (eg: your sending traffic to a dead peer before
>> forwarding is updated to remove those). If you are logged in doing
>> normal network stuff like looking up routes or making minor config
>> updates, is the cli slogwash or can you move around and work?
>>
>> Thank you so much for the feedback!
>>
>> Mike-
>>
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp

-- 
  ++ytti