[j-nsp] EX4550 or QFX5100 for Core

[Thomas Bellman]

On 2018-08-03 16:39, Giovanni Bellac via juniper-nsp wrote:

> So, we want something new with JTAC support. We need (1/10G)-Base-T,
> VLAN, L3, nothing fancy, but stable. We have 3k ARP entries.
>
> Option 1) 2x EX4550
>
> Option 2) 2x QFX5100
>
> We want to keep simplicity in and therefore want to use VC. We are
> pushing some Gbit/s from Rack-to-Rack (backups) and to our two
> upstreams around 500-600Mbit/s.
> QFX5100 hardware seems to be MUCH better than EX4550 hardware. The ARP
> table size, hash table size etc. on EX4550 is relatively small.

We have two QFX5100-48S in our datacenter core/spine, and have been
pretty happy with them.  They have been running for four years, and
I currently expect that we will replace them in another three years
(i.e. sometime in 2021).

We are however *not* running virtual chassis, but run them as stand-
alone units, with a mix of OSPF, Spanning Tree and VRRP to connect
them, and the leaf switches/routers, together.

We also have much smaller ARP (and MAC) tables than you.  Partly
because we mostly run L3 (OSPF) out to our leafs.  Currently we
have about 150 ARP entries, 70 IPv6 neighbour entries and 150 MAC
addresses in each.  We have about 1200 IPv4 routes and 450 IPv6
routes in our routing tables (that includes four VRFs).


We are currently not running BGP on them.  (The border routers are
MX480s, provided by our ISP, the Swedish NREN, and they talk BGP to
the NREN core routers, but we only talk OSPF to the border routers.)


We have almost only fiber, and mostly single-mode.  Just a couple of
1000Base-T connections.

If you don't need 10Gbase-T, and don't need all of the ports in a
QFX5100, you might want to take a look at EX4600.  The same hardware
as in QFX5100 (Broadcom Trident II), but fewer ports, and cheaper.
I think there are a few MPLS features that are disabled in the
EX4600.  We have recently bought a few of those to have as leaf
routers/switches in our datacenter, but I'm in the middle of taking
the first of them into production, so don't have much experience.
Given the similarities to QFX5100, I don't *expect* problems, though.

(If there had been a 10GBase-T version of the EX4600, I think we
would have bought that for some of the leafs instead, since TP cables
are much easier to handle in a rack than fiber or DAC, and cheaper
as well.)



There are several reasons for not running virtual chassis:

 - Standards-based protocols (OSPFv2, OSPFv3, Spanning Tree, VRRP).
   If we want to change vendors, or just generations with the same
   vendor, that is much easier than if they run a proprietary VC
   protocol.  Just add the new spines, move connections one by one
   to the new hardware, and finally turn off the old ones.

   With a virtual chassis, you would need to add routing protocols,
   STP, VRRP, et.c first between the old and new VC.  And that is
   not always possible to do without (short) downtimes.

 - Independant control planes.
   A shared control plane can cause a bug to take out both routers.

 - Loose coupling
   Virtual chassis requires much more state to be shared between the
   units, which makes the implementation more complicated.  Being a
   former programmer, I'm averse to that. :-)  I have also heard too
   many stories (about many vendors) about limitations or problems
   with virtual chassis to feel comfortable with that.


-- 
Thomas Bellman,  National Supercomputer Centre,  Linköping Univ., Sweden
"We don't understand the software, and sometimes we don't understand
 the hardware, but we can *see* the blinking lights!"

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <https://puck.nether.net/pipermail/juniper-nsp/attachments/20180803/e11ce685/attachment.sig>