[c-nsp] T3 or Ethernet delivery?

[Lee]

For us, price =is= the deciding factor.  A 45Mb ethernet service costs
us much less than a real T3.

We replaced a T3 circuit with a 45Mb ethernet service and then
discovered that the RTT went from 12ms on the T3 to 39ms on the
ethernet circuit.  Much discussion with the provider about
re-engineering the circuit to get the RTT down and then much more
waiting for them to schedule a service window ... and we've now got a
35ms RTT.

Another 'gotcha' is MTU size.  It's trivially easy to run IPSec over a
T3 without fragmenting packets.  Ethernet however...  we ended up
dumping one provider because they (w|c)ouldn't give us more than a
1524 [?not sure] byte MTU.

A nice thing about getting ethernet service is that more bandwidth is
just a phone call away.  We bumped the speed up from 45 to 100Mb and
are still paying less for the 100Mb ethernet service than we were for
the T3.   Still have that 35ms RTT though..

Input access lists that end with "deny ip any any log-input" are your
friend.  We just brought up a new circuit & I was seeing strange stuff
hitting our router.  Call the provider (who is my new love - the
person answering the phone was the person that fixed the problem ...
while I was on the phone!! ), give 'em the offending IP address and
get told it's a pure L2 network on their side.  *sigh*  change the
access-list from log to log-input, give him the offending MAC address,
he finds the offending box & fixes the config.

If you care at all about keeping your data private, put everything
inside an IPSec tunnel.  You have no idea who/what else is on that
same ethernet circuit.

If you care at all about throwing your packets into a black-hole, run
a routing protocol over the tunnel.

If you care at all about actually using the bandwidth you're paying
for, get the hardware crypto accelerator card for your platform.


> ... It scares me to think of opening trouble tickets as "it's
> broken and I can't really tell you why".

Welcome to user-land :)   Just remember to act like a real user and
lie when they ask you to reboot the box & see if that fixes the
problem.

HTH,
Lee


On 4/8/09, Seth Mattinen <sethm at rollernet.us> wrote:
> One of my carriers has given me a choice for a new circuit delivery: T3
> or Ethernet. My outside world circuit experience is all non-Ethernet, so
> I have a few questions the sales group wasn't able to answer. I'd love
> to hear some real world experience. The cost difference between the two
> is not significant enough to be the sole deciding factor and I'm not
> using pure-Ethernet platforms so it's just a matter of adding the right
> interface card.
>
> How do you detect a "down" condition on Ethernet? My experience is that
> the interface could be up/up because Ethernet doesn't know about
> anything further down the line and ends up throwing packets into a
> magical black hole. Or worse, secret packet loss.
>
> Can you even troubleshoot Ethernet? Normally if I'm seeing something
> like out of frame errors or AIS, I can say "hey, there's a problem and
> it's X". It scares me to think of opening trouble tickets as "it's
> broken and I can't really tell you why".
>
> With a T3 I can be fairly certain that if there aren't any alarms that
> my end is happily talking to the other end. How does one accomplish the
> same with Ethernet? A periodic "ping" seems rather ambiguous as a health
> check.
>
> Since this is an outside world connection (i.e. I'm not in a colo) the
> slightly lower cost and convenience factor of Ethernet doesn't override
> my desire to stick with a T3 for its management properties and the
> sleeping good at night feeling I get knowing there are no alarms. My gut
> tells me to stick with it even though Ethernet delivery is what all the
> cool kids are doing these days, so any insight is appreciated. Thanks!
>
> ~Seth
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>