[c-nsp] ONSs, DWDM SFPs, and the 3560/3750E

[Eric Van Tol]

> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net [mailto:cisco-nsp-
> bounces at puck.nether.net] On Behalf Of Justin Shore
> Sent: Monday, April 07, 2008 12:28 AM
> To: 'Cisco-nsp'
> Subject: [c-nsp] ONSs, DWDM SFPs, and the 3560/3750E
>
> So I'm working on a solution involving a pair of 15454s to transport
> numerous GigE links between a pair of sites over diverse paths and
> still
> give us a 10G upgrade path someday.  Unfortunately I know little
> about
> the ONSs at this time even though I've been staring at data sheets,
> presentations and the Dynamic Config Tool for weeks if not months.
> Like
> where do I use the filters?
>

I was in the same boat for the past year, so I feel your pain!

> I've been told we could use 15454s to build a fully-redundant chassis
> (PSUs, CPUs, etc) and then buy a pair of Xponder cards for each
> chassis.
>   Each Xponder card would output a single 10G link and we'd ship that
> link over one path and other 10G link from the other Xponder over the
> other path (for PtP links or use both 10G interfaces for L2 VLAN
> redundancy).  That's the basic plan as laid out in this doc (the only
> Xponder doc I could find):

This sounds about right.

> At one site we meet both of our upstreams with fiber (one provider
> with
> a Fuji and another with a 15454).  That site also has a data center
> which has numerous links between it and the other site.  And at the
> other site is the core of the ISP and all our offices.
>
> I've been told that the Xponder card can only accept GigE fiber
> inputs
> using the DWDM SFPs and that we'll have to convert to DWDM optics
> with
> an external switch if we have to use copper or other fiber links.  Is
> this true?

It doesn't sound like you are using dark fiber if your fiber is passing through your provier's electronics.  You're not going to be able to run DWDM over a lit pair that's handed off to you from a provider.  What you are describing is a lit service where they're handing off a p2p GE or 10GE to you.  Is this the case, or have I misunderstood?

> The one page I found on the Xponder card contradicts what
> this person is telling me.  I haven't had any luck finding good
> design
> or implementation docs on this card or exactly how it's used.  Both
> of
> our upstreams hand off as copper.  Fiber is not an option with one of
> the upstreams and with the other it's not something that we've
> discussed.  Either way it wouldn't be with DWDM optics.  The current
> data center hardware can only accept copper, for now.  Our internal
> connections can be fiber.

See previous response.  You cannot run DWDM over copper or over an already lit fiber.  Your provider might be willing to sell you different wavelengths on their fiber and you can mux/aggregate multiple GEs or 10GEs at your switch, but you're not going to be able to mux/demux at the wavelength level yourself, unless you're the one generating the light.

> The first solution that comes to mind is to stack a 3750G-12S with a
> copper 3750G and use that to map VLANs between copper and DWDM ports.
> However I can't find any mention of DWDM SFP support in the 3750G.
> Then
> I looked at the 3750E.  However there isn't a SFP-based chassis with
> the
> 3750E.  There is however the 3560E-12D and 12SD.  Since they are the
> exact same switches, sans the stacking interface, why isn't there an
> all-SFP or X2 3750E?  I hate to take a guess.
>
> Then I started thinking I could take the 3560E-12D and put TwinGig
> modules in it.  One side of the TwinGig would be the copper or
> standard
> fiber SFP and the other side would be the DWDM optic.  Then I read
> the
> data sheet for the 3560E and found out that DWDM SFPs aren't
> supported
> in either the 12D or 12SD chassis (but are supported in all the other
> 3560Es).  Is there a technical reason behind this?

Previous answers aside, I doubt there is any technical reason behind any Cisco switch from supporting any SFP.

> So I'm rather stuck.  This really isn't making any sense; I think I'm
> missing something here.  I'm short on design and implementation
> information for a DWDM deployment with the 15454s.  What info I do
> have
> seems to be contradicted by the data sheets, but if it's right is
> difficult to work around due to a lack of support for DWDM SFPs in
> various access switch platforms.  I can't even find the 15454-GE-XP
> when
> I try to build a 15454 on the Dynamic Config Tool.  I suspect there's
> more to it than what I'd seeing too.

If you've already engaged your Cisco SE, I'm surprised that he/she has not mentioned to you that, with the physical setup you describe above, you're not going to be able to accomplish what you want.

If you want to learn about Cisco-centric design and implementation, I suggest the Cisco Press book, "Optical Network Design and Implementation" (http://safari.ciscopress.com/1587051052).  It gives a very detailed, scientific overview of how WDM works, then provides some great info on the various optical platforms that Cisco offers, as well as case studies.

> Other solutions are always welcomed too.  I was considering a pure L2
> solution with the 4900Ms but I'd have to double up hardware on each
> end
> to avoid single-points of failure.  Plus that solution is only useful
> up
> to 10G of throughput unless I want to burn more fiber for more 10G
> links
> (something we want to avoid).  I also considered a L3 option with
> MPLS.
>   However the cost for full-redundancy dwarfs that of the L2 solution
> and should still be considerably more than the DWDM option.
>
> Does anyone have any pointers or tips on any of these problems?  Any
> guidance would be much appreciated.  What am I missing?
>
> Thanks
>   Justin

If I have your setup correct, then doing L2/L3 redundancy is your only option at this point.  Unless you can get dark fiber from your providers, WDM is not going to work.  I can say that moving to dark fiber can be costly at first, especially if your provider is a major player, but the long term benefits and cost savings are huge, since WDM offers almost limitless possibilities.

-evt