<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Something else that just clicked, I have been having a number of issues reaching arin.net today from one of my servers in Los Angeles that uses nLayer as its
upstream. Request response times are between 20 and 40 seconds as opposed to 2 to 4 seconds on our office connection. Looking at my trace from LA, we are going LA<->Cox<->ARIN.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">C:\Users\jake>tracert arin.net<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tracing route to arin.net [192.149.252.76]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">over a maximum of 30 hops:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 1 <1 ms 1 ms <1 ms v403.er01.lax.ubiquity.io [72.37.224.129]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 2 1 ms 5 ms 1 ms xe-1-0-3.ar1.lax2.us.nlayer.net [69.31.127.45]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 3 <1 ms <1 ms <1 ms ae1-80g.cr1.lax1.us.nlayer.net [69.31.127.129]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 4 2 ms 5 ms 2 ms ae2-50g.ar1.lax1.us.nlayer.net [69.31.127.142]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 5 <1 ms <1 ms <1 ms as22773.ae12.ar1.lax1.us.nlayer.net [69.31.127.230]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 6 70 ms 111 ms 70 ms mrfddsrj01-ae0.0.rd.dc.cox.net [68.1.1.5]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 7 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 8 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> 9 72 ms 73 ms 82 ms wsip-98-172-152-14.dc.dc.cox.net [98.172.152.14]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">10 72 ms 72 ms 82 ms host-252-131.arin.net [192.149.252.131]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">11 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">12 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">13 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">14 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">15 * * * Request timed out.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> outages-bounces@outages.org [mailto:outages-bounces@outages.org]
<b>On Behalf Of </b>Jake Mertel<br>
<b>Sent:</b> Wednesday, December 19, 2012 4:45 PM<br>
<b>To:</b> 'Brandon Whaley'; 'outages@outages.org'<br>
<b>Subject:</b> Re: [outages] Cox -> nLayer connectivity issues<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We have received a report of similar issues today. The client has servers with us in several locations where we use nLayer and/or PacketExchagne and his monitoring
system is on a network that uses Cox as its preferred upstream. He shutdown his Cox upstream and didn’t have any issues reaching the servers over his backup provider. The issues were sporadic and did not affect all protocols – ICMP pings worked, snmpwalk was
fine, but UDP traces were dying somewhere on the reverse path. Seems to be very similar to what you are seeing.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:outages-bounces@outages.org">outages-bounces@outages.org</a> [<a href="mailto:outages-bounces@outages.org">mailto:outages-bounces@outages.org</a>]
<b>On Behalf Of </b>Brandon Whaley<br>
<b>Sent:</b> Wednesday, December 19, 2012 4:25 PM<br>
<b>To:</b> <a href="mailto:outages@outages.org">outages@outages.org</a><br>
<b>Subject:</b> [outages] Cox -> nLayer connectivity issues<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">We've been seeing intermittent TCP/UDP connectivity issues from Cox Communications in Virginia to any location that routes over nLayer. UDP traceroutes are fine, but DNS lookups time out for minutes at a time, then work again for ~5 minutes
before repeating the problem. ICMP is never affected during the outages.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">traceroute to 198.46.80.1 (198.46.80.1), 30 hops max, 60 byte packets<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 1 router36f24c.local (192.168.14.1) 0.560 ms 0.531 ms 0.753 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 2 <a href="http://wsip-174-77-92-169.hr.hr.cox.net">wsip-174-77-92-169.hr.hr.cox.net</a> (174.77.92.169) 2.532 ms 2.581 ms 3.009 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 3 172.21.224.153 (172.21.224.153) 3.995 ms 4.067 ms 4.111 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 4 172.21.249.101 (172.21.249.101) 4.185 ms 4.396 ms 4.561 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 5 172.21.249.73 (172.21.249.73) 4.916 ms 4.900 ms 5.128 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 6 172.21.249.18 (172.21.249.18) 5.517 ms 5.124 ms 5.043 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 7 <a href="http://ip-216-54-33-22.coxfiber.net">ip-216-54-33-22.coxfiber.net</a> (216.54.33.22) 210.486 ms 210.477 ms 210.466 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 8 68.1.4.139 (68.1.4.139) 221.950 ms 222.460 ms 232.268 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> 9 * <a href="http://xe-5-0-7.ar1.iad1.us.nlayer.net">xe-5-0-7.ar1.iad1.us.nlayer.net</a> (69.31.10.81) 226.332 ms 226.866 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">10 <a href="http://as54641.xe-9-0-1.ar1.iad1.us.nlayer.net">as54641.xe-9-0-1.ar1.iad1.us.nlayer.net</a> (69.31.31.42) 223.718 ms 225.083 ms 225.744 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">11 198.46.80.1 (198.46.80.1) 225.706 ms 226.670 ms 226.698 ms<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Is anyone with Cox on the list that can investigate/contact me?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <br>
Best Regards, <br>
Brandon W.<o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>