[outages] Cox -> nLayer connectivity issues

[Brandon Whaley]

Hi Jeremy, thanks for getting back to me.

That is indeed our edge router, I was just giving an example of the path.

We have all return traffic preferenced to leave the network via our Level3
connection right now, and we've got our AS prepended with our nLayer
session multiple times.  Cox is choosing to use our nLayer route inbound
regardless and we're not going to consider taking it down to resolve the
problem.

The trace path I sent is not unusual for our connection (though the latency
is, we've been in contact with Cox about that), I was providing it as an
example of the inbound path only.

The issue is affecting all Cox users in our area, so it's not isolated to
our office or even our city.

The fact that ICMP is unaffected leads me to believe that a malfunctioning
device performing some sort of "traffic shaping" may be at fault.


On Wed, Dec 19, 2012 at 6:34 PM, Jeremy Chadwick <jdc at koitsu.org> wrote:

> Brandon,
>
> Do you have return-path traceroutes from 198.46.80.1 to wherever you did
> the below traceroute (and what is that source IP)?  Also, is 198.46.80.1
> a host/system or a router?  If a router, please do not do that -- please
> use a host/system as a destination.
>
> I would say the problem looks like it's happening between hops #6 and #7
> in the below traceroute (jump in latency from 5ms to 210ms), but that
> may be normal -- do you have traceroutes for what things look like
> normally?
>
> The latency seen may also be caused by issues on the return-path.
>
> P.S. -- This would be the first I've heard in a long time where ICMP
> *was not* affected, but TCP and UDP were.  Usually the reports are
> "everything is impacted" or "only ICMP is impacted" (result of ICMP
> deprio and a reporter not having familiarity with that fact).
>
> --
> | Jeremy Chadwick                                   jdc at koitsu.org |
> | UNIX Systems Administrator                http://jdc.koitsu.org/ |
> | Mountain View, CA, US                                            |
> | Making life hard for others since 1977.             PGP 4BD6C0CB |
>
> On Wed, Dec 19, 2012 at 06:24:46PM -0500, Brandon Whaley wrote:
> > We've been seeing intermittent TCP/UDP connectivity issues from Cox
> > Communications in Virginia to any location that routes over nLayer.  UDP
> > traceroutes are fine, but DNS lookups time out for minutes at a time,
> then
> > work again for ~5 minutes before repeating the problem.  ICMP is never
> > affected during the outages.
> >
> > traceroute to 198.46.80.1 (198.46.80.1), 30 hops max, 60 byte packets
> >  1  router36f24c.local (192.168.14.1)  0.560 ms  0.531 ms  0.753 ms
> >  2  wsip-174-77-92-169.hr.hr.cox.net (174.77.92.169)  2.532 ms  2.581 ms
> >  3.009 ms
> >  3  172.21.224.153 (172.21.224.153)  3.995 ms  4.067 ms  4.111 ms
> >  4  172.21.249.101 (172.21.249.101)  4.185 ms  4.396 ms  4.561 ms
> >  5  172.21.249.73 (172.21.249.73)  4.916 ms  4.900 ms  5.128 ms
> >  6  172.21.249.18 (172.21.249.18)  5.517 ms  5.124 ms  5.043 ms
> >  7  ip-216-54-33-22.coxfiber.net (216.54.33.22)  210.486 ms  210.477 ms
> >  210.466 ms
> >  8  68.1.4.139 (68.1.4.139)  221.950 ms  222.460 ms  232.268 ms
> >  9  * xe-5-0-7.ar1.iad1.us.nlayer.net (69.31.10.81)  226.332 ms
>  226.866 ms
> > 10  as54641.xe-9-0-1.ar1.iad1.us.nlayer.net (69.31.31.42)  223.718 ms
> >  225.083 ms  225.744 ms
> > 11  198.46.80.1 (198.46.80.1)  225.706 ms  226.670 ms  226.698 ms
> >
> > Is anyone with Cox on the list that can investigate/contact me?
> >
> > --
> > Best Regards,
> > Brandon W.
>
> > _______________________________________________
> > Outages mailing list
> > Outages at outages.org
> > https://puck.nether.net/mailman/listinfo/outages
>
>


-- 
Best Regards,
Brandon W.
Tier 3 System Administrator
InMotion Hosting Inc.

888-321-4678
757-416-6575 (Int'l)
NEW: 24x7 EMAIL and PHONE Technical Support

Did you know?
We'll Build, Update and Promote Your Site for You! Visit
www.inmotionhosting.com/webdesign
Answers to commonly asked questions, as well as other useful tools, can be
found at http://support.inmotionhosting.com

How am I doing? Please feel free to email my manager at
manager_feedback at inmotion.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20121219/5197e86d/attachment.htm>