[outages] Intermittent Level 3 outage/connectivity issues on Dallas edges?
Chris Boulton
chris at bigcommerce.com
Thu Jan 24 15:02:29 EST 2013
It looks like the situation has resolved itself in the last minute - I'm
still waiting for an update from Level 3, but from 2 off-list replies it
looks like there were a few people who've experienced routing issues with
Level 3 in Houston today.
Regards,
Chris Boulton
Bigcommerce
Email: chris at bigcommerce.com
Web: http://www.bigcommerce.com/
On Fri, Jan 25, 2013 at 6:45 AM, Chris Boulton <chris at bigcommerce.com>wrote:
> Ah, my apologies on the confusion. They're two distinct paths out, not
> traceroutes from each end.
>
> I chose not to include the destination IP because the issues we're seeing
> a more wide spread than to a single IP.
>
> Level 3 managed to get back to us - they were originally reporting back to
> us that they are experiencing connectivity issues for some of their clients
> in Austin, which they're now denying and still continuing to investigate.
>
> Regards,
>
> Chris Boulton
> Bigcommerce
>
> Email: chris at bigcommerce.com
>
> Web: http://www.bigcommerce.com/
>
>
> On Fri, Jan 25, 2013 at 6:39 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:
>
>> Chris, is there some reason you're not providing actual
>> source/destination IPs for each of your mtrs?
>>
>> The reason I'm asking is because I was left to believe what you were
>> providing was both ends of an asymmetric connection (i.e. mtr #1 was
>> for x.x.x.x to y.y.y.y, mtr #2 was for y.y.y.y to x.x.x.x).
>>
>> However, your 2nd paragraph below ("we're seeing issues over one
>> connections but not another") seems to indicate what you provided are
>> two mtrs across completely different circuits.
>>
>> So which is it?
>>
>> --
>> | Jeremy Chadwick jdc at koitsu.org |
>> | UNIX Systems Administrator http://jdc.koitsu.org/ |
>> | Mountain View, CA, US |
>> | Making life hard for others since 1977. PGP 4BD6C0CB |
>>
>> On Fri, Jan 25, 2013 at 06:14:05AM +1100, Chris Boulton wrote:
>> > Thanks, I am aware of what the traceroute shows and how a traceroute
>> works.
>> > The intent of the traceroute was primarily to show where I see our paths
>> > between Level 3 and TW Telecom converge.
>> >
>> > Plain and simple - we are experiencing connectivity issues over our
>> Level 3
>> > connection, which we're not seeing over our TW Telecom connection for
>> > traffic destined to the same location.
>> >
>> > Regards,
>> >
>> > Chris Boulton
>> > Bigcommerce
>> >
>> > Email: chris at bigcommerce.com
>> >
>> > Web: http://www.bigcommerce.com/
>> >
>> >
>> > On Fri, Jan 25, 2013 at 6:11 AM, Jeremy Chadwick <jdc at koitsu.org>
>> wrote:
>> >
>> > > Please see my below post on this matter, which I posted only 2 days
>> ago:
>> > >
>> > > https://puck.nether.net/pipermail/outages/2013-January/005111.html
>> > >
>> > > What you've shown in your mtr does not show any issue of any kind with
>> > > Level 3. The packet loss shown at hops 4/5/6 does not "trickle down"
>> > > each subsequent hop (look at hop 7 onward for proof), nor is there any
>> > > sign of latency increase either.
>> > >
>> > > On the other hand, the 2nd mtr you provided shows some degree of
>> packet
>> > > loss starting at hop 2 (TW Telecom) which *does* "trickle down"
>> through
>> > > all subsequent hops (egress). If you want to know how many packets
>> were
>> > > lost, look at the --order flag to mtr (I recommend --order=LSRNABW).
>> > >
>> > > Plain and simple: talk to TW Telecom about the loss in their network.
>> > >
>> > > --
>> > > | Jeremy Chadwick jdc at koitsu.org |
>> > > | UNIX Systems Administrator http://jdc.koitsu.org/ |
>> > > | Mountain View, CA, US |
>> > > | Making life hard for others since 1977. PGP 4BD6C0CB |
>> > >
>> > > On Fri, Jan 25, 2013 at 05:46:27AM +1100, Chris Boulton wrote:
>> > > > Seeing connectivity issues as traffic passes through Level 3 in
>> Dallas -
>> > > > not seeing the same issues for where routes converge (above.net in
>> DFW)
>> > > > over our TW Telecom link.
>> > > >
>> > > > Anybody else experiencing this as well?
>> > > >
>> > > > Level 3:
>> > > >
>> > > > Host Loss%
>> Snt
>> > > > Last Avg Best Wrst StDev
>> > > > 1. vlan358.car1.Houston1.Level3.net 0.0%
>> 167
>> > > > 105.9 80.5 31.7 276.4 59.8
>> > > > 2. ae-2-5.bar1.Houston1.Level3.net 0.0%
>> 167
>> > > > 32.5 34.6 32.2 93.8 7.3
>> > > > 3. ae-13-13.ebr1.Dallas1.Level3.net 0.0%
>> 167
>> > > > 39.5 37.4 36.7 41.2 0.9
>> > > > 4. ae-91-91.csw4.Dallas1.Level3.net 65.1%
>> 167
>> > > > 44.5 43.5 39.7 53.8 4.0
>> > > > 5. ae-4-90.edge2.Dallas3.Level3.net 5.4%
>> 166
>> > > > 37.1 37.9 36.1 75.6 5.1
>> > > > 6. ex-3-1-0.er1.dfw2.us.above.net 66.1%
>> 166
>> > > > 39.5 43.5 38.9 104.8 12.5
>> > > > 7. xe-3-1-0.cr1.dfw2.us.above.net 0.0%
>> 166
>> > > > 37.3 40.4 37.0 114.1 10.6
>> > > > 8. xe-0-0-0.er3.dfw2.us.above.net 0.0%
>> 166
>> > > > 36.2 39.8 36.1 99.3 10.2
>> > > >
>> > > > Host Loss%
>> Snt
>> > > > Last Avg Best Wrst StDev
>> > > > 1. 50.58.18.1 0.0%
>> 829
>> > > > 4.3 23.9 4.1 121.2 14.5
>> > > > 2. dal2-pr2-xe-0-3-0-0.us.twtelecom.net 0.1%
>> 829
>> > > > 55.4 31.2 9.4 121.0 17.2
>> > > > dal2-pr2-xe-2-3-0-0.us.twtelecom.net
>> > > > 3. xe-3-0-0.er2.dfw2.us.above.net 0.1%
>> 829
>> > > > 44.8 31.6 9.5 143.7 17.9
>> > > > 4. xe-0-1-0.cr2.dfw2.us.above.net 0.2%
>> 829
>> > > > 40.5 31.2 10.3 116.2 15.9
>> > > > xe-3-0-0.cr2.dfw2.us.above.net
>> > > > 5. xe-0-2-0.er3.dfw2.us.above.net 0.2%
>> 829
>> > > > 42.7 32.4 9.6 109.2 16.5
>> > > >
>> > > >
>> > > >
>> > > > Regards,
>> > > >
>> > > > Chris Boulton
>> > > > Bigcommerce
>> > > >
>> > > > Email: chris at bigcommerce.com
>> > > >
>> > > > Web: http://www.bigcommerce.com/
>> > >
>> > > > _______________________________________________
>> > > > Outages mailing list
>> > > > Outages at outages.org
>> > > > https://puck.nether.net/mailman/listinfo/outages
>> > >
>> > >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20130125/86b71bec/attachment.htm>
More information about the Outages
mailing list