[c-nsp] DWDM optics on 6500s

[Nick Hilliard]

On 02/10/2009 17:44, Jeff Bacon wrote:
> I am looking at getting some metro waves (mostly 20-40km) between sites;
> I'm working with a provider who is using passive splitters on dark runs
> and they're willing to split me out a wave for near the same cost as
> just running a gig switched.

I went through this some while back, and on the basis that:

- coloured xenpaks are exotic, expensive and only produced by a single 
manufacturer in the world (opnext, as you ask)
- coloured xenpaks will only last as long as your 6704 card, meaning that 
when you retire this kit, your entire coloured optics investment is lost
- transponders were not hugely more expensive than the prices I was quoted 
for cisco coloured optics

... I decided that coloured xenpaks, while marginally cheaper in the short 
term, were actually a bad strategic move in the long term.   Given the way 
that our network has changed since we made that decision, it turns out that 
it was a good decision to make, as we're completely flexible about what kit 
we use at each end of the link, and have chosen to exercise that flexibility.

There is also a much better selection of coloured XFPs on the market than 
coloured xenpak.

> The goal is primarily serialization latency delay reduction, not
> actually running 10G of traffic - I'll be lucky to run 1-2GB (though
> it'll mostly be 60-100byte packets).

If you want to cut delay for switching, you may want to consider the new 
top-of-rack 10G boxes, which are typically cut-through.  You may find that 
these boxes + SR SFP+ + wdm transponders is quite cost favourable compared 
to c6500 chassis space + 6704 + coloured xenpak.  Cisco N5K may be a good 
option here.  But other vendors have similar style boxes (Brocade Ti24X, 
Extreme X650, F10 S2410, Arista Networks *.*, etc).  Oh, and the SFP+ boxes 
will also run 1G ethernet on SFPs (although the N2K has some limitations). 
  This is a nice feature win.

> 1) The cisco optics appear to be in short supply and damn expensive. I
> may have to go third-party. I know it's a gamble. Any other issues I
> Should think about besides what's been discussed here?

coloured xenpaks are a single vendor product and I have heard that they are 
mostly made to order, hence the delay.

> 2) Is XENPAK on 6704 viable? Any gotchas I should know about with
> XENPAKs vs X2?

X2 == xenpak version 2.  Their power draw is slightly less than xenpak, but 
lots more than xfp / sfp+. Only HP and Cisco use X2 for ethernet switches - 
everyone else uses XFP and latterly SFP+, which means that there is less 
pricing pressure and and more vendor lock-in if you go down the X2 route.

Personally, I have a bit of a thing against X2, but that's just me.  Make 
your own mind up.

> 3) Does 6500 switching performance blow super-hard, or just so-so hard?
> (6-15us is ok.) Yes a 4900M might be faster, or a J-product, but I don't
> want to change platform really, I need NAT and don't want to use
> routers, I want to keep box count down (co-lo), and having a whole box
> just for passing 10G doesn't IMO make sense because I'd still have to
> get it into the 6500 anyway.

The 6500 is a great 1G switch platform, but doesn't excel in the 10G range, 
particularly with 6704 blades.

Nick