<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">On Aug 5, 2015, at 10:59 AM, James Milko <<a href="mailto:jmilko@bandwidth.com">jmilko@bandwidth.com</a>> wrote:<br><div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">What is everyone's network path to Softlayer? We just had an issue where traffic in one direction, from one network would fail. The really interesting bit is that our SIP traffic was failing but ICMP worked fine.</div></blockquote><div><br></div>From NAC, which was the only place I was seeing trouble with ICMP not making it through (yes, that was odd - nagios alerted on the host down, but http and other checks were OK, traceroute also showed a response on the last hop), the path is pretty simple, NAC and SL peer with each other:</div><div><br></div><div><div> 3 0.e1-1.<a href="http://tbr1.mmu.nac.net">tbr1.mmu.nac.net</a> (209.123.10.18) 0.352 ms 0.324 ms 0.358 ms</div><div> 4 0.e1-2.<a href="http://tbr1.tl9.nac.net">tbr1.tl9.nac.net</a> (209.123.10.130) 1.356 ms 1.312 ms 10.229 ms</div><div> 5 0.e2-1.<a href="http://pr2.tl9.nac.net">pr2.tl9.nac.net</a> (209.123.11.142) 1.101 ms 1.196 ms 1.104 ms</div><div> 6 te1-7.bbr01.<a href="http://tl01.nyc01.networklayer.com">tl01.nyc01.networklayer.com</a> (198.32.160.27) 1.354 ms 1.446 ms 1.478 ms</div><div> 7 ae7.bbr02.<a href="http://tl01.nyc01.networklayer.com">tl01.nyc01.networklayer.com</a> (173.192.18.177) 1.731 ms 1.696 ms 1.859 ms</div><div> 8 ae1.bbr01.<a href="http://eq01.chi01.networklayer.com">eq01.chi01.networklayer.com</a> (173.192.18.132) 22.718 ms 21.803 ms 22.602 ms</div><div> 9 ae20.bbr01.<a href="http://eq01.dal03.networklayer.com">eq01.dal03.networklayer.com</a> (173.192.18.136) 40.673 ms 40.780 ms 39.721 ms</div><div>10 ae5.<a href="http://dar01.dal05.networklayer.com">dar01.dal05.networklayer.com</a> (173.192.18.215) 40.550 ms 40.576 ms 40.515 ms</div><div>11 po1.fcr02.<a href="http://sr02.dal05.networklayer.com">sr02.dal05.networklayer.com</a> (173.192.118.139) 40.958 ms 40.768 ms 41.833 ms</div><div><br></div><div>Same in both directions.</div><div><br></div><div>It also fixed itself at 11:52 eastern.</div><div><br></div><div>I know long ago SL noted that they had all sorts of nifty IDS stuff in place. I’m not sure they still do, but if they do, maybe it went nuts trying to “mitigate” something that didn’t need mitigating. Always weird when you lose one particular protocol (me ICMP, you UDP and possibly only particular ports).</div><div><br></div><div>Charles</div><div><br></div><blockquote type="cite"><div dir="ltr"><div><br></div><div>We noticed that our transit of choice flipped last night, so we manually flipped it back and the issue resolved.</div><div><br></div><div>2828 3356 36351 - no SIP in one direction<br></div><div>174 1299 36351 - Works fine</div><div><br></div><div>We've fired tickets off to everyone involved hoping someone finds something.</div><div><br></div><div class="gmail_extra"><div><div class="gmail_signature"><div dir="ltr"><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:bold;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">James Milko</span><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:italic;font-variant:normal;text-decoration:none;vertical-align:baseline"></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:italic;font-variant:normal;text-decoration:none;vertical-align:baseline">Architect, Network Engineering<br></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#888888;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">900 Main Campus Drive</span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#888888;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Raleigh, NC 27606</span><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="text-decoration:underline;font-size:13px;font-family:Arial;color:rgb(25,98,207);font-weight:normal;font-style:normal;font-variant:normal;vertical-align:baseline;background-color:rgb(255,255,255)"><a href="http://www.bandwidth.com/business/" style="text-decoration:none" target="_blank">Bandwidth</a></span></div><span style="font-size:13px;font-family:Arial;color:#1962cf;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline"></span></div></div></div>
<br><div class="gmail_quote">On Wed, Aug 5, 2015 at 10:37 AM, Justin Head via Outages <span dir="ltr"><<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
I've got a VM with Linode in Dallas (SoftLayer) running a monitoring
system and it's seeing connectivity issues over SNMP (UDP) to
basically all locations. This has been on and off all morning since
at least 5am CDT.<br>
<pre cols="72">--
Justin Head
Owner / Chief Operating Officer
CubedHost, LLC
<a href="https://cubedhost.com/" target="_blank">https://cubedhost.com</a></pre><div><div class="h5">
<div>On 8/5/15 9:23 AM, Charles Sprickman
via Outages wrote:<br>
</div>
<blockquote type="cite">
<div>
<div>
<div>I haven’t had time to look at this and I’m going
out the door right now, but I’ve had some odd alerts on a
softlayer-hosted box. It has multiple IPs and from <a href="http://nac.net/" target="_blank">nac.net</a>
in NJ, I am unable to ping some IPs but not others.</div>
<div><br>
</div>
<div>I won’t have time to poke at it more until this
afternoon (eastern) though.</div>
<div><br>
</div>
<div>Charles</div>
<div><br>
</div>
<div>Izzy Goldstein - TeleGo via Outages <<a href="mailto:outages@outages.org" target="_blank"></a><a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:</div>
</div>
<div><br>
</div>
<blockquote type="cite">
<div>
<div dir="ltr">
<div style="font-family:monospace,monospace">seems like monitis is having some issues?</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace">cant load web page <a href="http://www.monitis.com/" target="_blank"></a><a href="http://www.monitis.com/" target="_blank">www.monitis.com</a></div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace">here is a traceroute output</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div>
<div style="font-family:monospace,monospace">[root@pbx81 ~]# traceroute <a href="http://monitis.com/" target="_blank">monitis.com</a></div>
<div style="font-family:monospace,monospace">traceroute to <a href="http://monitis.com/" target="_blank">monitis.com</a> (208.76.244.202), 30 hops
max, 60 byte packets</div>
<div style="font-family:monospace,monospace"> 1 209.200.42.35
(209.200.42.35) 0.516 ms 0.568 ms 0.549 ms</div>
<div style="font-family:monospace,monospace"> 2 <a href="http://csc180.gsc.webair.net/" target="_blank">csc180.gsc.webair.net</a>
(173.239.28.37) 0.678 ms 0.812 ms 0.795 ms</div>
<div style="font-family:monospace,monospace"> 3 <a href="http://es0.nyc4.webair.net/" target="_blank">es0.nyc4.webair.net</a>
(173.239.0.25) 1.211 ms 1.335 ms 1.317 ms</div>
<div style="font-family:monospace,monospace"> 4 208.178.245.149
(208.178.245.149) 0.896 ms 1.031 ms 1.009 ms</div>
<div style="font-family:monospace,monospace"> 5 * * *</div>
<div style="font-family:monospace,monospace"> 6 * * *</div>
<div style="font-family:monospace,monospace"> 7 <a href="http://colo4-dalla.ear1.dallas1.level3.net/" target="_blank">COLO4-DALLA.ear1.Dallas1.Level3.net</a>
(8.9.232.74) 35.522 ms 35.481 ms 35.475 ms</div>
<div style="font-family:monospace,monospace"> 8 * * *</div>
<div style="font-family:monospace,monospace"> 9 72.249.131.13
(72.249.131.13) 38.657 ms 38.888 ms 39.113 ms</div>
<div style="font-family:monospace,monospace">10 <a href="http://67.208.119.94-static.reverse.crucialx.net/" target="_blank">67.208.119.94-static.reverse.crucialx.net</a>
(67.208.119.94) 35.806 ms 36.365 ms 36.474 ms</div>
<div style="font-family:monospace,monospace">11 <a href="http://monitis.com/" target="_blank">monitis.com</a>
(208.76.244.202) 35.439 ms 35.247 ms 35.596 ms</div>
<div style="font-family:monospace,monospace">[root@pbx81 ~]#</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div>
<div><font face="monospace, monospace"><br>
</font></div>
<div><font face="monospace, monospace">Tracing route to <a href="http://monitis.com/" target="_blank">monitis.com</a>
[208.76.244.202]</font></div>
<div><font face="monospace, monospace">over a maximum of 30
hops:</font></div>
<div><font face="monospace, monospace"><br>
</font></div>
<div><font face="monospace, monospace"> 1 <1 ms
<1 ms <1 ms
Wireless_Broadband_Router.home [192.168.1.1]</font></div>
<div><font face="monospace, monospace"> 2 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 3 10 ms 8 ms
9 ms <a href="http://451be041.cst.lightpath.net/" target="_blank">451be041.cst.lightpath.net</a>
[65.19.98.65]</font></div>
<div><font face="monospace, monospace"> 4 11 ms 9 ms
11 ms <a href="http://ool-4353dda4.dyn.optonline.net/" target="_blank">ool-4353dda4.dyn.optonline.net</a>
[67.83.221.164]</font></div>
<div><font face="monospace, monospace"> 5 9 ms 11 ms
10 ms <a href="http://451be075.cst.lightpath.net/" target="_blank">451be075.cst.lightpath.net</a>
[65.19.99.117]</font></div>
<div><font face="monospace, monospace"> 6 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 7 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 8 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 9 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 10 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 11 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 12 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 13 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 14 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 15 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 16 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 17 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 18 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 19 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 20 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 21 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 22 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 23 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 24 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 25 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 26 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 27 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 28 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 29 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"> 30 * *
* Request timed out.</font></div>
<div><font face="monospace, monospace"><br>
</font></div>
<div><font face="monospace, monospace">Trace complete.</font></div>
</div>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div style="font-family:monospace,monospace"><br>
</div>
<div><br>
</div>
-- <br>
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr" style="font-size:small">
<div style="font-family:monospace,monospace"></div>
<br>
</div>
<div dir="ltr" style="font-size:small">
<div style="font-size:12.8000001907349px"><br>
</div>
<div>
<div style="font-size:12.8000001907349px"><font color="#444444">Izzy Goldstein</font></div>
<div style="font-size:12.8000001907349px"><font color="#444444"><br>
</font></div>
<div style="font-size:12.8000001907349px"><font color="#444444">Main: (212)
477.1000<br>
Fax: <a href="tel:%28212%29%20477.8900" value="+12124778900" target="_blank">(212) 477.8900</a></font></div>
<div style="font-size:12.8000001907349px"><font color="#444444"><br>
</font></div>
<div style="font-size:12.8000001907349px"><font color="#444444"><a href="mailto:igoldstein@telego.com" style="color:rgb(17,85,204)" target="_blank"></a><a href="mailto:igoldstein@telego.com" target="_blank">igoldstein@telego.com</a></font></div>
<div style="font-size:12.8000001907349px"><font color="#444444"><br>
</font></div>
<div style="font-size:12.8000001907349px"><font color="#444444"><a href="http://www.telego.com/" style="color:rgb(17,85,204)" target="_blank">Website</a> | <a href="https://www.linkedin.com/company/telego" style="color:rgb(17,85,204)" target="_blank">LinkedIn</a> | <a href="http://www.telego.com/blog" style="color:rgb(17,85,204)" target="_blank">Blog</a> |</font></div>
<div style="font-size:12.8000001907349px"><font color="#666666"><br>
</font></div>
<div style="font-size:12.8000001907349px"><font color="#666666"><br>
</font></div>
<div><font color="#cccccc"><font face="arial, helvetica,
sans-serif" size="1">Confidentially
Notice: This e-mail may contain
confidential and/or
privileged information. If you are
not the intended recipient or have
received this e-mail in error
please notify us immediately by
email reply and destroy this
e-mail. Any unauthorized
copying, disclosure
or distribution of the material in
this e-mail is strictly forbidden.
Any views or opinions presented in
this email are solely those of the
author and do not necessarily
represent those of TeleGo (T).
Employees of TeleGo are expressly
required not to make defamatory
statements and not to infringe or
authorize any infringement of
copyright or any other legal right
by email communications. Any such
communication is contrary to
TeleGo policy and outside the
scope of the employment of the
individual concerned. TeleGo will
not accept any liability in
respect of such communication, and
the employee responsible will be
personally liable for any damages
or other liability arising.</font><br>
</font></div>
<div style="font-size:12.8000001907349px">
<div><font face="arial, helvetica,
sans-serif" color="#cccccc" size="1"><br>
</font></div>
<div><font face="arial, helvetica,
sans-serif" color="#cccccc" size="1">Think before you print.</font></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Outages mailing list<br>
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></div>
</blockquote>
<br>
</div>
<div><br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Outages mailing list
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a>
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Outages mailing list<br>
<a href="mailto:Outages@outages.org">Outages@outages.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/outages" rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div><br></body></html>