[c-nsp] T3 or Ethernet delivery?

[Bill Wichers]

I've found that some carriers consider Ethernet something of a "toy"
whereas TDM and SONET circuits are considered more "mission critical".
Basically our local engineering gusy say that the Ethernet links are
just a "bunch of jumpers in COs", and by that they mean a single link
patched through to where it needs to go with no protection or management
anywhere. The T3 links, while not always path diverse, are typically at
least provisioned as 4 fiber handoffs within the carrier's network so
you at least have some protection against a dead optic. This seems to be
especially an issue for intercity links since the T3s are typically
protected around a ring between the cities and the Ethernet rarely, if
ever, is protected at all. This is just the ILEC (ATT here) though, most
of the CLECs offer protection options for their Ethernet offerings.

Personally I've been burned before with carriers not provisioning
circuits as "protected" as one would expect (which includes TDM/SONET
links). I try to keep all our core links on our own fiber where we
control the physical routing and protection, but we have a few remote
POPs that are not economical to build fiber to and those are the ones
with the leased links. I'm not a big fan of Ethernet for the links to
these POPs, but the Ethernet links we use from our gear to the customer
premises do tend to work OK.

Regarding monitoring, use a routing protocol that has keepalives to
detect an outage. If you are using a switch you can probably determine
link state on the circuit too (although this probably won't give you an
indication of end-to-end circuit status since the carrier probably has a
switch serving you that will give you a link regardless of the "rest" of
the circuit working). 

   -Bill


> One of my carriers has given me a choice for a new circuit delivery:
T3
> or Ethernet. My outside world circuit experience is all non-Ethernet,
so
> I have a few questions the sales group wasn't able to answer. I'd love
> to hear some real world experience. The cost difference between the
two
> is not significant enough to be the sole deciding factor and I'm not
> using pure-Ethernet platforms so it's just a matter of adding the
right
> interface card.
> 
> How do you detect a "down" condition on Ethernet? My experience is
that
> the interface could be up/up because Ethernet doesn't know about
> anything further down the line and ends up throwing packets into a
> magical black hole. Or worse, secret packet loss.
> 
> Can you even troubleshoot Ethernet? Normally if I'm seeing something
> like out of frame errors or AIS, I can say "hey, there's a problem and
> it's X". It scares me to think of opening trouble tickets as "it's
> broken and I can't really tell you why".
> 
> With a T3 I can be fairly certain that if there aren't any alarms that
> my end is happily talking to the other end. How does one accomplish
the
> same with Ethernet? A periodic "ping" seems rather ambiguous as a
health
> check.
> 
> Since this is an outside world connection (i.e. I'm not in a colo) the
> slightly lower cost and convenience factor of Ethernet doesn't
override
> my desire to stick with a T3 for its management properties and the
> sleeping good at night feeling I get knowing there are no alarms. My
gut
> tells me to stick with it even though Ethernet delivery is what all
the
> cool kids are doing these days, so any insight is appreciated. Thanks!
> 
> ~Seth
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/