[outages] Intermittent Level 3 outage/connectivity issues on Dallas edges?

Chris Boulton chris at bigcommerce.com
Thu Jan 24 14:45:54 EST 2013


Ah, my apologies on the confusion. They're two distinct paths out, not
traceroutes from each end.

I chose not to include the destination IP because the issues we're seeing a
more wide spread than to a single IP.

Level 3 managed to get back to us - they were originally reporting back to
us that they are experiencing connectivity issues for some of their clients
in Austin, which they're now denying and still continuing to investigate.

Regards,

Chris Boulton
Bigcommerce

Email: chris at bigcommerce.com

Web: http://www.bigcommerce.com/


On Fri, Jan 25, 2013 at 6:39 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:

> Chris, is there some reason you're not providing actual
> source/destination IPs for each of your mtrs?
>
> The reason I'm asking is because I was left to believe what you were
> providing was both ends of an asymmetric connection (i.e. mtr #1 was
> for x.x.x.x to y.y.y.y, mtr #2 was for y.y.y.y to x.x.x.x).
>
> However, your 2nd paragraph below ("we're seeing issues over one
> connections but not another") seems to indicate what you provided are
> two mtrs across completely different circuits.
>
> So which is it?
>
> --
> | Jeremy Chadwick                                   jdc at koitsu.org |
> | UNIX Systems Administrator                http://jdc.koitsu.org/ |
> | Mountain View, CA, US                                            |
> | Making life hard for others since 1977.             PGP 4BD6C0CB |
>
> On Fri, Jan 25, 2013 at 06:14:05AM +1100, Chris Boulton wrote:
> > Thanks, I am aware of what the traceroute shows and how a traceroute
> works.
> > The intent of the traceroute was primarily to show where I see our paths
> > between Level 3 and TW Telecom converge.
> >
> > Plain and simple - we are experiencing connectivity issues over our
> Level 3
> > connection, which we're not seeing over our TW Telecom connection for
> > traffic destined to the same location.
> >
> > Regards,
> >
> > Chris Boulton
> > Bigcommerce
> >
> > Email: chris at bigcommerce.com
> >
> > Web: http://www.bigcommerce.com/
> >
> >
> > On Fri, Jan 25, 2013 at 6:11 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:
> >
> > > Please see my below post on this matter, which I posted only 2 days
> ago:
> > >
> > > https://puck.nether.net/pipermail/outages/2013-January/005111.html
> > >
> > > What you've shown in your mtr does not show any issue of any kind with
> > > Level 3.  The packet loss shown at hops 4/5/6 does not "trickle down"
> > > each subsequent hop (look at hop 7 onward for proof), nor is there any
> > > sign of latency increase either.
> > >
> > > On the other hand, the 2nd mtr you provided shows some degree of packet
> > > loss starting at hop 2 (TW Telecom) which *does* "trickle down" through
> > > all subsequent hops (egress).  If you want to know how many packets
> were
> > > lost, look at the --order flag to mtr (I recommend --order=LSRNABW).
> > >
> > > Plain and simple: talk to TW Telecom about the loss in their network.
> > >
> > > --
> > > | Jeremy Chadwick                                   jdc at koitsu.org |
> > > | UNIX Systems Administrator                http://jdc.koitsu.org/ |
> > > | Mountain View, CA, US                                            |
> > > | Making life hard for others since 1977.             PGP 4BD6C0CB |
> > >
> > > On Fri, Jan 25, 2013 at 05:46:27AM +1100, Chris Boulton wrote:
> > > > Seeing connectivity issues as traffic passes through Level 3 in
> Dallas -
> > > > not seeing the same issues for where routes converge (above.net in
> DFW)
> > > > over our TW Telecom link.
> > > >
> > > > Anybody else experiencing this as well?
> > > >
> > > > Level 3:
> > > >
> > > >  Host                                                     Loss%   Snt
> > > > Last   Avg  Best  Wrst StDev
> > > >  1. vlan358.car1.Houston1.Level3.net                       0.0%
> 167
> > > >  105.9  80.5  31.7 276.4  59.8
> > > >  2. ae-2-5.bar1.Houston1.Level3.net                        0.0%
> 167
> > > > 32.5  34.6  32.2  93.8   7.3
> > > >  3. ae-13-13.ebr1.Dallas1.Level3.net                       0.0%
> 167
> > > > 39.5  37.4  36.7  41.2   0.9
> > > >  4. ae-91-91.csw4.Dallas1.Level3.net                      65.1%
> 167
> > > > 44.5  43.5  39.7  53.8   4.0
> > > >  5. ae-4-90.edge2.Dallas3.Level3.net                       5.4%
> 166
> > > > 37.1  37.9  36.1  75.6   5.1
> > > >  6. ex-3-1-0.er1.dfw2.us.above.net                        66.1%
> 166
> > > > 39.5  43.5  38.9 104.8  12.5
> > > >  7. xe-3-1-0.cr1.dfw2.us.above.net                         0.0%
> 166
> > > > 37.3  40.4  37.0 114.1  10.6
> > > >  8. xe-0-0-0.er3.dfw2.us.above.net                         0.0%
> 166
> > > > 36.2  39.8  36.1  99.3  10.2
> > > >
> > > >  Host                                                     Loss%   Snt
> > > > Last   Avg  Best  Wrst StDev
> > > >  1. 50.58.18.1                                             0.0%   829
> > > >  4.3  23.9   4.1 121.2  14.5
> > > >  2. dal2-pr2-xe-0-3-0-0.us.twtelecom.net                   0.1%
> 829
> > > > 55.4  31.2   9.4 121.0  17.2
> > > >     dal2-pr2-xe-2-3-0-0.us.twtelecom.net
> > > >  3. xe-3-0-0.er2.dfw2.us.above.net                         0.1%
> 829
> > > > 44.8  31.6   9.5 143.7  17.9
> > > >  4. xe-0-1-0.cr2.dfw2.us.above.net                         0.2%
> 829
> > > > 40.5  31.2  10.3 116.2  15.9
> > > >     xe-3-0-0.cr2.dfw2.us.above.net
> > > >  5. xe-0-2-0.er3.dfw2.us.above.net                         0.2%
> 829
> > > > 42.7  32.4   9.6 109.2  16.5
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Chris Boulton
> > > > Bigcommerce
> > > >
> > > > Email: chris at bigcommerce.com
> > > >
> > > > Web: http://www.bigcommerce.com/
> > >
> > > > _______________________________________________
> > > > Outages mailing list
> > > > Outages at outages.org
> > > > https://puck.nether.net/mailman/listinfo/outages
> > >
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20130125/21697391/attachment.htm>


More information about the Outages mailing list