<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">On Aug 5, 2015, at 10:59 AM, James Milko <<a href="mailto:jmilko@bandwidth.com">jmilko@bandwidth.com</a>> wrote:<br><div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr">What is everyone's network path to Softlayer?  We just had an issue where traffic in one direction, from one network would fail.  The really interesting bit is that our SIP traffic was failing but ICMP worked fine.</div></blockquote><div><br></div>From NAC, which was the only place I was seeing trouble with ICMP not making it through (yes, that was odd - nagios alerted on the host down, but http and other checks were OK, traceroute also showed a response on the last hop), the path is pretty simple, NAC and SL peer with each other:</div><div><br></div><div><div> 3  0.e1-1.<a href="http://tbr1.mmu.nac.net">tbr1.mmu.nac.net</a> (209.123.10.18)  0.352 ms  0.324 ms  0.358 ms</div><div> 4  0.e1-2.<a href="http://tbr1.tl9.nac.net">tbr1.tl9.nac.net</a> (209.123.10.130)  1.356 ms  1.312 ms  10.229 ms</div><div> 5  0.e2-1.<a href="http://pr2.tl9.nac.net">pr2.tl9.nac.net</a> (209.123.11.142)  1.101 ms  1.196 ms  1.104 ms</div><div> 6  te1-7.bbr01.<a href="http://tl01.nyc01.networklayer.com">tl01.nyc01.networklayer.com</a> (198.32.160.27)  1.354 ms  1.446 ms  1.478 ms</div><div> 7  ae7.bbr02.<a href="http://tl01.nyc01.networklayer.com">tl01.nyc01.networklayer.com</a> (173.192.18.177)  1.731 ms  1.696 ms  1.859 ms</div><div> 8  ae1.bbr01.<a href="http://eq01.chi01.networklayer.com">eq01.chi01.networklayer.com</a> (173.192.18.132)  22.718 ms  21.803 ms  22.602 ms</div><div> 9  ae20.bbr01.<a href="http://eq01.dal03.networklayer.com">eq01.dal03.networklayer.com</a> (173.192.18.136)  40.673 ms  40.780 ms  39.721 ms</div><div>10  ae5.<a href="http://dar01.dal05.networklayer.com">dar01.dal05.networklayer.com</a> (173.192.18.215)  40.550 ms  40.576 ms  40.515 ms</div><div>11  po1.fcr02.<a href="http://sr02.dal05.networklayer.com">sr02.dal05.networklayer.com</a> (173.192.118.139)  40.958 ms  40.768 ms  41.833 ms</div><div><br></div><div>Same in both directions.</div><div><br></div><div>It also fixed itself at 11:52 eastern.</div><div><br></div><div>I know long ago SL noted that they had all sorts of nifty IDS stuff in place.  I’m not sure they still do, but if they do, maybe it went nuts trying to “mitigate” something that didn’t need mitigating.  Always weird when you lose one particular protocol (me ICMP, you UDP and possibly only particular ports).</div><div><br></div><div>Charles</div><div><br></div><blockquote type="cite"><div dir="ltr"><div><br></div><div>We noticed that our transit of choice flipped last night, so we manually flipped it back and the issue resolved.</div><div><br></div><div>2828 3356 36351 - no SIP in one direction<br></div><div>174 1299 36351 - Works fine</div><div><br></div><div>We've fired tickets off to everyone involved hoping someone finds something.</div><div><br></div><div class="gmail_extra"><div><div class="gmail_signature"><div dir="ltr"><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:bold;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">James Milko</span><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:italic;font-variant:normal;text-decoration:none;vertical-align:baseline"></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:italic;font-variant:normal;text-decoration:none;vertical-align:baseline">Architect, Network Engineering<br></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#888888;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">900 Main Campus Drive</span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size:13px;font-family:Arial;color:#888888;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Raleigh, NC 27606</span><span style="font-size:13px;font-family:Arial;color:#999999;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"></span></div><div style="line-height: 1; margin-top: 0pt; margin-bottom: 0pt;"><span style="text-decoration:underline;font-size:13px;font-family:Arial;color:rgb(25,98,207);font-weight:normal;font-style:normal;font-variant:normal;vertical-align:baseline;background-color:rgb(255,255,255)"><a href="http://www.bandwidth.com/business/" style="text-decoration:none" target="_blank">Bandwidth</a></span></div><span style="font-size:13px;font-family:Arial;color:#1962cf;background-color:#ffffff;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline"></span></div></div></div>
<br><div class="gmail_quote">On Wed, Aug 5, 2015 at 10:37 AM, Justin Head via Outages <span dir="ltr"><<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    I've got a VM with Linode in Dallas (SoftLayer) running a monitoring
    system and it's seeing connectivity issues over SNMP (UDP) to
    basically all locations. This has been on and off all morning since
    at least 5am CDT.<br>
    <pre cols="72">--
Justin Head
Owner / Chief Operating Officer
CubedHost, LLC
<a href="https://cubedhost.com/" target="_blank">https://cubedhost.com</a></pre><div><div class="h5">
    <div>On 8/5/15 9:23 AM, Charles Sprickman
      via Outages wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div>
        <div>
          <div>I haven’t had time to look at this and I’m going
            out the door right now, but I’ve had some odd alerts on a
            softlayer-hosted box. It has multiple IPs and from <a href="http://nac.net/" target="_blank">nac.net</a>
            in NJ, I am unable to ping some IPs but not others.</div>
          <div><br>
          </div>
          <div>I won’t have time to poke at it more until this
            afternoon (eastern) though.</div>
          <div><br>
          </div>
          <div>Charles</div>
          <div><br>
          </div>
          <div>Izzy Goldstein - TeleGo via Outages <<a href="mailto:outages@outages.org" target="_blank"></a><a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:</div>
        </div>
        <div><br>
        </div>
        <blockquote type="cite">
          <div>
            <div dir="ltr">
              <div style="font-family:monospace,monospace">seems like monitis is having some issues?</div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace">cant load web page <a href="http://www.monitis.com/" target="_blank"></a><a href="http://www.monitis.com/" target="_blank">www.monitis.com</a></div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace">here is a traceroute output</div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div>
                <div style="font-family:monospace,monospace">[root@pbx81 ~]# traceroute <a href="http://monitis.com/" target="_blank">monitis.com</a></div>
                <div style="font-family:monospace,monospace">traceroute to <a href="http://monitis.com/" target="_blank">monitis.com</a> (208.76.244.202), 30 hops
                  max, 60 byte packets</div>
                <div style="font-family:monospace,monospace"> 1  209.200.42.35
                  (209.200.42.35)  0.516 ms  0.568 ms  0.549 ms</div>
                <div style="font-family:monospace,monospace"> 2  <a href="http://csc180.gsc.webair.net/" target="_blank">csc180.gsc.webair.net</a>
                  (173.239.28.37)  0.678 ms  0.812 ms  0.795 ms</div>
                <div style="font-family:monospace,monospace"> 3  <a href="http://es0.nyc4.webair.net/" target="_blank">es0.nyc4.webair.net</a>
                  (173.239.0.25)  1.211 ms  1.335 ms  1.317 ms</div>
                <div style="font-family:monospace,monospace"> 4  208.178.245.149
                  (208.178.245.149)  0.896 ms  1.031 ms  1.009 ms</div>
                <div style="font-family:monospace,monospace"> 5  * * *</div>
                <div style="font-family:monospace,monospace"> 6  * * *</div>
                <div style="font-family:monospace,monospace"> 7  <a href="http://colo4-dalla.ear1.dallas1.level3.net/" target="_blank">COLO4-DALLA.ear1.Dallas1.Level3.net</a>
                  (8.9.232.74)  35.522 ms  35.481 ms  35.475 ms</div>
                <div style="font-family:monospace,monospace"> 8  * * *</div>
                <div style="font-family:monospace,monospace"> 9  72.249.131.13
                  (72.249.131.13)  38.657 ms  38.888 ms  39.113 ms</div>
                <div style="font-family:monospace,monospace">10  <a href="http://67.208.119.94-static.reverse.crucialx.net/" target="_blank">67.208.119.94-static.reverse.crucialx.net</a>
                  (67.208.119.94)  35.806 ms  36.365 ms  36.474 ms</div>
                <div style="font-family:monospace,monospace">11  <a href="http://monitis.com/" target="_blank">monitis.com</a>
                  (208.76.244.202)  35.439 ms  35.247 ms  35.596 ms</div>
                <div style="font-family:monospace,monospace">[root@pbx81 ~]#</div>
                <div style="font-family:monospace,monospace"><br>
                </div>
                <div style="font-family:monospace,monospace"><br>
                </div>
                <div>
                  <div><font face="monospace, monospace"><br>
                    </font></div>
                  <div><font face="monospace, monospace">Tracing route to <a href="http://monitis.com/" target="_blank">monitis.com</a>
                      [208.76.244.202]</font></div>
                  <div><font face="monospace, monospace">over a maximum of 30
                      hops:</font></div>
                  <div><font face="monospace, monospace"><br>
                    </font></div>
                  <div><font face="monospace, monospace">  1    <1 ms  
                       <1 ms    <1 ms
                       Wireless_Broadband_Router.home [192.168.1.1]</font></div>
                  <div><font face="monospace, monospace">  2     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace">  3    10 ms     8 ms
                          9 ms  <a href="http://451be041.cst.lightpath.net/" target="_blank">451be041.cst.lightpath.net</a>
                      [65.19.98.65]</font></div>
                  <div><font face="monospace, monospace">  4    11 ms     9 ms
                         11 ms  <a href="http://ool-4353dda4.dyn.optonline.net/" target="_blank">ool-4353dda4.dyn.optonline.net</a>
                      [67.83.221.164]</font></div>
                  <div><font face="monospace, monospace">  5     9 ms    11 ms
                         10 ms  <a href="http://451be075.cst.lightpath.net/" target="_blank">451be075.cst.lightpath.net</a>
                      [65.19.99.117]</font></div>
                  <div><font face="monospace, monospace">  6     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace">  7     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace">  8     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace">  9     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 10     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 11     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 12     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 13     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 14     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 15     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 16     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 17     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 18     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 19     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 20     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 21     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 22     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 23     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 24     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 25     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 26     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 27     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 28     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 29     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"> 30     *        *    
                         *     Request timed out.</font></div>
                  <div><font face="monospace, monospace"><br>
                    </font></div>
                  <div><font face="monospace, monospace">Trace complete.</font></div>
                </div>
              </div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div style="font-family:monospace,monospace"><br>
              </div>
              <div><br>
              </div>
              -- <br>
              <div>
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div>
                        <div dir="ltr">
                          <div dir="ltr">
                            <div dir="ltr" style="font-size:small">
                              <div style="font-family:monospace,monospace">​</div>
                              <br>
                            </div>
                            <div dir="ltr" style="font-size:small">
                              <div style="font-size:12.8000001907349px"><br>
                              </div>
                              <div>
                                <div style="font-size:12.8000001907349px"><font color="#444444">Izzy Goldstein</font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444"><br>
                                  </font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444">Main:   (212)
                                    477.1000<br>
                                    Fax:     <a href="tel:%28212%29%20477.8900" value="+12124778900" target="_blank">(212) 477.8900</a></font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444"><br>
                                  </font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444"><a href="mailto:igoldstein@telego.com" style="color:rgb(17,85,204)" target="_blank"></a><a href="mailto:igoldstein@telego.com" target="_blank">igoldstein@telego.com</a></font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444"><br>
                                  </font></div>
                                <div style="font-size:12.8000001907349px"><font color="#444444"><a href="http://www.telego.com/" style="color:rgb(17,85,204)" target="_blank">Website</a> | <a href="https://www.linkedin.com/company/telego" style="color:rgb(17,85,204)" target="_blank">LinkedIn</a> | <a href="http://www.telego.com/blog" style="color:rgb(17,85,204)" target="_blank">Blog</a> |</font></div>
                                <div style="font-size:12.8000001907349px"><font color="#666666"><br>
                                  </font></div>
                                <div style="font-size:12.8000001907349px"><font color="#666666"><br>
                                  </font></div>
                                <div><font color="#cccccc"><font face="arial, helvetica,
                                      sans-serif" size="1">Confidentially
                                      Notice: This e-mail may contain
                                      confidential and/or
                                      privileged information. If you are
                                      not the intended recipient or have
                                      received this e-mail in error
                                      please notify us immediately by
                                      email reply and destroy this
                                      e-mail. Any unauthorized
                                      copying, disclosure
                                      or distribution of the material in
                                      this e-mail is strictly forbidden.
                                      Any views or opinions presented in
                                      this email are solely those of the
                                      author and do not necessarily
                                      represent those of TeleGo (T).
                                      Employees of TeleGo are expressly
                                      required not to make defamatory
                                      statements and not to infringe or
                                      authorize any infringement of
                                      copyright or any other legal right
                                      by email communications. Any such
                                      communication is contrary to
                                      TeleGo policy and outside the
                                      scope of the employment of the
                                      individual concerned. TeleGo will
                                      not accept any liability in
                                      respect of such communication, and
                                      the employee responsible will be
                                      personally liable for any damages
                                      or other liability arising.</font><br>
                                  </font></div>
                                <div style="font-size:12.8000001907349px">
                                  <div><font face="arial, helvetica,
                                      sans-serif" color="#cccccc" size="1"><br>
                                    </font></div>
                                  <div><font face="arial, helvetica,
                                      sans-serif" color="#cccccc" size="1">Think before you print.</font></div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
            _______________________________________________<br>
            Outages mailing list<br>
            <a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br>
            <a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></div>
        </blockquote>
        <br>
      </div>
      <div><br>
      </div>
      <br>
      <fieldset></fieldset>
      <br>
      <pre>_______________________________________________
Outages mailing list
<a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a>
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a>
</pre>
    </blockquote>
    <br>
  </div></div></div>

<br>_______________________________________________<br>
Outages mailing list<br>
<a href="mailto:Outages@outages.org">Outages@outages.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/outages" rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>
<br></blockquote></div><br></div></div>
</blockquote></div><br></body></html>