<div dir="ltr">It looks like the situation has resolved itself in the last minute - I'm still waiting for an update from Level 3, but from 2 off-list replies it looks like there were a few people who've experienced routing issues with Level 3 in Houston today.</div>
<div class="gmail_extra"><br clear="all"><div>Regards,<br><br>Chris Boulton<br>Bigcommerce<br><br>Email: <a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a><br><br>Web: <a href="http://www.bigcommerce.com" target="_blank">http://www.bigcommerce.com/</a></div>
<br><br><div class="gmail_quote">On Fri, Jan 25, 2013 at 6:45 AM, Chris Boulton <span dir="ltr"><<a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Ah, my apologies on the confusion. They're two distinct paths out, not traceroutes from each end.<div><br></div><div>I chose not to include the destination IP because the issues we're seeing a more wide spread than to a single IP.</div>
<div><br></div><div>Level 3 managed to get back to us - they were originally reporting back to us that they are experiencing connectivity issues for some of their clients in Austin, which they're now denying and still continuing to investigate.</div>
</div><div class="gmail_extra"><div class="im"><br clear="all"><div>Regards,<br><br>Chris Boulton<br>Bigcommerce<br><br>Email: <a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a><br><br>Web: <a href="http://www.bigcommerce.com" target="_blank">http://www.bigcommerce.com/</a></div>
<br><br></div><div><div class="h5"><div class="gmail_quote">On Fri, Jan 25, 2013 at 6:39 AM, Jeremy Chadwick <span dir="ltr"><<a href="mailto:jdc@koitsu.org" target="_blank">jdc@koitsu.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Chris, is there some reason you're not providing actual<br>
source/destination IPs for each of your mtrs?<br>
<br>
The reason I'm asking is because I was left to believe what you were<br>
providing was both ends of an asymmetric connection (i.e. mtr #1 was<br>
for x.x.x.x to y.y.y.y, mtr #2 was for y.y.y.y to x.x.x.x).<br>
<br>
However, your 2nd paragraph below ("we're seeing issues over one<br>
connections but not another") seems to indicate what you provided are<br>
two mtrs across completely different circuits.<br>
<br>
So which is it?<br>
<div><br>
--<br>
| Jeremy Chadwick <a href="mailto:jdc@koitsu.org" target="_blank">jdc@koitsu.org</a> |<br>
| UNIX Systems Administrator <a href="http://jdc.koitsu.org/" target="_blank">http://jdc.koitsu.org/</a> |<br>
| Mountain View, CA, US |<br>
| Making life hard for others since 1977. PGP 4BD6C0CB |<br>
<br>
</div><div><div>On Fri, Jan 25, 2013 at 06:14:05AM +1100, Chris Boulton wrote:<br>
> Thanks, I am aware of what the traceroute shows and how a traceroute works.<br>
> The intent of the traceroute was primarily to show where I see our paths<br>
> between Level 3 and TW Telecom converge.<br>
><br>
> Plain and simple - we are experiencing connectivity issues over our Level 3<br>
> connection, which we're not seeing over our TW Telecom connection for<br>
> traffic destined to the same location.<br>
><br>
> Regards,<br>
><br>
> Chris Boulton<br>
> Bigcommerce<br>
><br>
> Email: <a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a><br>
><br>
> Web: <a href="http://www.bigcommerce.com/" target="_blank">http://www.bigcommerce.com/</a><br>
><br>
><br>
> On Fri, Jan 25, 2013 at 6:11 AM, Jeremy Chadwick <<a href="mailto:jdc@koitsu.org" target="_blank">jdc@koitsu.org</a>> wrote:<br>
><br>
> > Please see my below post on this matter, which I posted only 2 days ago:<br>
> ><br>
> > <a href="https://puck.nether.net/pipermail/outages/2013-January/005111.html" target="_blank">https://puck.nether.net/pipermail/outages/2013-January/005111.html</a><br>
> ><br>
> > What you've shown in your mtr does not show any issue of any kind with<br>
> > Level 3. The packet loss shown at hops 4/5/6 does not "trickle down"<br>
> > each subsequent hop (look at hop 7 onward for proof), nor is there any<br>
> > sign of latency increase either.<br>
> ><br>
> > On the other hand, the 2nd mtr you provided shows some degree of packet<br>
> > loss starting at hop 2 (TW Telecom) which *does* "trickle down" through<br>
> > all subsequent hops (egress). If you want to know how many packets were<br>
> > lost, look at the --order flag to mtr (I recommend --order=LSRNABW).<br>
> ><br>
> > Plain and simple: talk to TW Telecom about the loss in their network.<br>
> ><br>
> > --<br>
> > | Jeremy Chadwick <a href="mailto:jdc@koitsu.org" target="_blank">jdc@koitsu.org</a> |<br>
> > | UNIX Systems Administrator <a href="http://jdc.koitsu.org/" target="_blank">http://jdc.koitsu.org/</a> |<br>
> > | Mountain View, CA, US |<br>
> > | Making life hard for others since 1977. PGP 4BD6C0CB |<br>
> ><br>
> > On Fri, Jan 25, 2013 at 05:46:27AM +1100, Chris Boulton wrote:<br>
> > > Seeing connectivity issues as traffic passes through Level 3 in Dallas -<br>
> > > not seeing the same issues for where routes converge (<a href="http://above.net" target="_blank">above.net</a> in DFW)<br>
> > > over our TW Telecom link.<br>
> > ><br>
> > > Anybody else experiencing this as well?<br>
> > ><br>
> > > Level 3:<br>
> > ><br>
> > > Host Loss% Snt<br>
> > > Last Avg Best Wrst StDev<br>
> > > 1. <a href="http://vlan358.car1.Houston1.Level3.net" target="_blank">vlan358.car1.Houston1.Level3.net</a> 0.0% 167<br>
> > > 105.9 80.5 31.7 276.4 59.8<br>
> > > 2. <a href="http://ae-2-5.bar1.Houston1.Level3.net" target="_blank">ae-2-5.bar1.Houston1.Level3.net</a> 0.0% 167<br>
> > > 32.5 34.6 32.2 93.8 7.3<br>
> > > 3. <a href="http://ae-13-13.ebr1.Dallas1.Level3.net" target="_blank">ae-13-13.ebr1.Dallas1.Level3.net</a> 0.0% 167<br>
> > > 39.5 37.4 36.7 41.2 0.9<br>
> > > 4. <a href="http://ae-91-91.csw4.Dallas1.Level3.net" target="_blank">ae-91-91.csw4.Dallas1.Level3.net</a> 65.1% 167<br>
> > > 44.5 43.5 39.7 53.8 4.0<br>
> > > 5. <a href="http://ae-4-90.edge2.Dallas3.Level3.net" target="_blank">ae-4-90.edge2.Dallas3.Level3.net</a> 5.4% 166<br>
> > > 37.1 37.9 36.1 75.6 5.1<br>
> > > 6. <a href="http://ex-3-1-0.er1.dfw2.us.above.net" target="_blank">ex-3-1-0.er1.dfw2.us.above.net</a> 66.1% 166<br>
> > > 39.5 43.5 38.9 104.8 12.5<br>
> > > 7. <a href="http://xe-3-1-0.cr1.dfw2.us.above.net" target="_blank">xe-3-1-0.cr1.dfw2.us.above.net</a> 0.0% 166<br>
> > > 37.3 40.4 37.0 114.1 10.6<br>
> > > 8. <a href="http://xe-0-0-0.er3.dfw2.us.above.net" target="_blank">xe-0-0-0.er3.dfw2.us.above.net</a> 0.0% 166<br>
> > > 36.2 39.8 36.1 99.3 10.2<br>
> > ><br>
> > > Host Loss% Snt<br>
> > > Last Avg Best Wrst StDev<br>
> > > 1. 50.58.18.1 0.0% 829<br>
> > > 4.3 23.9 4.1 121.2 14.5<br>
> > > 2. <a href="http://dal2-pr2-xe-0-3-0-0.us.twtelecom.net" target="_blank">dal2-pr2-xe-0-3-0-0.us.twtelecom.net</a> 0.1% 829<br>
> > > 55.4 31.2 9.4 121.0 17.2<br>
> > > <a href="http://dal2-pr2-xe-2-3-0-0.us.twtelecom.net" target="_blank">dal2-pr2-xe-2-3-0-0.us.twtelecom.net</a><br>
> > > 3. <a href="http://xe-3-0-0.er2.dfw2.us.above.net" target="_blank">xe-3-0-0.er2.dfw2.us.above.net</a> 0.1% 829<br>
> > > 44.8 31.6 9.5 143.7 17.9<br>
> > > 4. <a href="http://xe-0-1-0.cr2.dfw2.us.above.net" target="_blank">xe-0-1-0.cr2.dfw2.us.above.net</a> 0.2% 829<br>
> > > 40.5 31.2 10.3 116.2 15.9<br>
> > > <a href="http://xe-3-0-0.cr2.dfw2.us.above.net" target="_blank">xe-3-0-0.cr2.dfw2.us.above.net</a><br>
> > > 5. <a href="http://xe-0-2-0.er3.dfw2.us.above.net" target="_blank">xe-0-2-0.er3.dfw2.us.above.net</a> 0.2% 829<br>
> > > 42.7 32.4 9.6 109.2 16.5<br>
> > ><br>
> > ><br>
> > ><br>
> > > Regards,<br>
> > ><br>
> > > Chris Boulton<br>
> > > Bigcommerce<br>
> > ><br>
> > > Email: <a href="mailto:chris@bigcommerce.com" target="_blank">chris@bigcommerce.com</a><br>
> > ><br>
> > > Web: <a href="http://www.bigcommerce.com/" target="_blank">http://www.bigcommerce.com/</a><br>
> ><br>
> > > _______________________________________________<br>
> > > Outages mailing list<br>
> > > <a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br>
> > > <a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>
> ><br>
> ><br>
</div></div></blockquote></div><br></div></div></div>
</blockquote></div><br></div>