[c-nsp] in praise of the cat6500 Re: Flow tools

[Jeff Bacon]

> Date: Fri, 20 Jan 2012 20:00:56 +0000
> From: Alessandra Forti <Alessandra.Forti at cern.ch>
> 
> Hi,
> 
> I got some money to upgrade my network infrastructure from 1Gbps to 10Gbps.
> 
> At the moment I have a cat6509E with a Sup720. This system has been
> working fine for 6 years. The upgrade will have to last a similar number
> of years and our main requirement is throughput with minimal routing if
> we are going to double the link to the outside world. My initial
> combination to support 16 racks at 10Gbps was to simply buy
> 4x6716-10T-3C blades and keep the Sup720. I then got enough money to
> upgrade the Sup720 to a Sup2T with (6816-10T-2T blades). I was wondering
> if this is really necessary or if the Sup720 will last that long i.e.
> another 6 years. I'm not an expert and would appreciate your comments if
> I go down this route because the alternative is to replace the 6509
> altogether (most likely with a Force10 Z9000).
> 

OK. So, I'm a little late here, and it's not normally what I
get into, but.

What strikes me here is "throughput with minimal routing". What
is the 6500 actually _doing_? Is it doing primarily layer-2 with
some VLAN SVIs and light layer-3 with some routing protocol?

If that's the case, well, you could easily use a bunch of 6704s as
one poster suggested to get a bunch of cheap line-rate ports, or 
use a 6716 and oversubscribe... 

But if it were me? I'd toss the 6500 entirely and get an Arista 7050.
If you need more ports, then use a 40G aggregator and fan-out on
7050s. Or insert some other vendor that's doing 10G on commodity
silicon here. 

(I'd also suggest ditching that idea of 10G-T and just go twinax. you
can reach 7 meters or even more, and it's more reliable and draws
way less power than 10G-T - not to mention error rates. The cables
will cost you a bit more but overall it's worth it. Of course
it depends on what you're using for TOR.)


The Cat6k got its start as an L2 device. It was that until some
bright boy decided to gut a 7200 NPE and glue it into the supervisor
and create the MSFC. But we've come a long way since then. The
Cat6500 at this point functionally resembles a very high throughput
mid-range-capability switching router that happens to also be able
to play a dumb L2/L3 switch when necessary. 

As an L2 or basic L3 switch, it's out-matched and out-classed. (IMO,
so is the entire cat4k line, anymore, except in certain situations.) 
As a 10G L3 switch, it's massively out-classed. If that's all you want,
buy something else. Your per-port cost for 10G is so high on a cat6500
that it's just ridiculous. 

The cat6k has graduated to being a high-touch device. MPLS, ATOM,
QoS, Netflow (yes RD it's got flaws but it still HAS Netflow),
complex configurations - got it. No, it's not an ASR9k. But it
can do a fair bit of what an ASR9k can do for way less and with
latency/mpps rates that an ASR9k could only dream of. It's in the
middle somewhere. 

If you're not going to use any of those features, there's plenty
of better cheaper alternatives than a cat6500. And it doesn't
sound like the OP intends to. 

Myself - yes, I have a mesh of 6500s. Which, for any site with
any density of hosts, immediately drops down into an Arista L3
dist/fan-out layer, because 10G ports are way cheaper on an Arista.
The 6500s own the 10G MAN links and split the traffic out into
its various layers, and really it breaks down to a single 
vs720/X6708 or sup2t/6908, and all the traffic lives on the plane
of the 6x08 with maybe a 67xx-SFP feeding in some 1G traffic. 

Just another $0.02 in the pot.

-bacon