[outages] Intermittent Level 3 outage/connectivity issues on Dallas edges?
Jeremy Chadwick
jdc at koitsu.org
Thu Jan 24 15:06:58 EST 2013
Gotcha, now we're on the same page! :-)
So can you provide return-path mtr or traceroutes that are associated
with both of your below links? Level 3 will probably want this as well,
as most routing today is asymmetrical.
Rephrased verbosely: we need to see a total of 4 mtrs/traceroutes:
1) From x.x.x.x to y.y.y.y (your Level 3 link) -- you provided this
2) From y.y.y.y to x.x.x.x -- need this
3) From a.a.a.a to b.b.b.b (your TW Telecom link) -- you provided this
4) From b.b.b.b to a.a.a.a -- need this
What you showed in Item #1 (your L3 link) shows no actual/real issue.
What you showed in Item #3 (your TW Telecom link) **does** show packet
loss, but I cannot tell (without Item #4) if it's on the return-path or
the forward-path. I understand "your TW Telecom link is working fine"
but the 0.1-0.2% packet loss indicates it may not be as good as you
think. That may be a separate issue however, or may be by design
(QoS-like rules applied on router, etc.).
Make sense? :-)
--
| Jeremy Chadwick jdc at koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
On Fri, Jan 25, 2013 at 06:45:54AM +1100, Chris Boulton wrote:
> Ah, my apologies on the confusion. They're two distinct paths out, not
> traceroutes from each end.
>
> I chose not to include the destination IP because the issues we're seeing a
> more wide spread than to a single IP.
>
> Level 3 managed to get back to us - they were originally reporting back to
> us that they are experiencing connectivity issues for some of their clients
> in Austin, which they're now denying and still continuing to investigate.
>
> Regards,
>
> Chris Boulton
> Bigcommerce
>
> Email: chris at bigcommerce.com
>
> Web: http://www.bigcommerce.com/
>
>
> On Fri, Jan 25, 2013 at 6:39 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:
>
> > Chris, is there some reason you're not providing actual
> > source/destination IPs for each of your mtrs?
> >
> > The reason I'm asking is because I was left to believe what you were
> > providing was both ends of an asymmetric connection (i.e. mtr #1 was
> > for x.x.x.x to y.y.y.y, mtr #2 was for y.y.y.y to x.x.x.x).
> >
> > However, your 2nd paragraph below ("we're seeing issues over one
> > connections but not another") seems to indicate what you provided are
> > two mtrs across completely different circuits.
> >
> > So which is it?
> >
> > --
> > | Jeremy Chadwick jdc at koitsu.org |
> > | UNIX Systems Administrator http://jdc.koitsu.org/ |
> > | Mountain View, CA, US |
> > | Making life hard for others since 1977. PGP 4BD6C0CB |
> >
> > On Fri, Jan 25, 2013 at 06:14:05AM +1100, Chris Boulton wrote:
> > > Thanks, I am aware of what the traceroute shows and how a traceroute
> > works.
> > > The intent of the traceroute was primarily to show where I see our paths
> > > between Level 3 and TW Telecom converge.
> > >
> > > Plain and simple - we are experiencing connectivity issues over our
> > Level 3
> > > connection, which we're not seeing over our TW Telecom connection for
> > > traffic destined to the same location.
> > >
> > > Regards,
> > >
> > > Chris Boulton
> > > Bigcommerce
> > >
> > > Email: chris at bigcommerce.com
> > >
> > > Web: http://www.bigcommerce.com/
> > >
> > >
> > > On Fri, Jan 25, 2013 at 6:11 AM, Jeremy Chadwick <jdc at koitsu.org> wrote:
> > >
> > > > Please see my below post on this matter, which I posted only 2 days
> > ago:
> > > >
> > > > https://puck.nether.net/pipermail/outages/2013-January/005111.html
> > > >
> > > > What you've shown in your mtr does not show any issue of any kind with
> > > > Level 3. The packet loss shown at hops 4/5/6 does not "trickle down"
> > > > each subsequent hop (look at hop 7 onward for proof), nor is there any
> > > > sign of latency increase either.
> > > >
> > > > On the other hand, the 2nd mtr you provided shows some degree of packet
> > > > loss starting at hop 2 (TW Telecom) which *does* "trickle down" through
> > > > all subsequent hops (egress). If you want to know how many packets
> > were
> > > > lost, look at the --order flag to mtr (I recommend --order=LSRNABW).
> > > >
> > > > Plain and simple: talk to TW Telecom about the loss in their network.
> > > >
> > > > --
> > > > | Jeremy Chadwick jdc at koitsu.org |
> > > > | UNIX Systems Administrator http://jdc.koitsu.org/ |
> > > > | Mountain View, CA, US |
> > > > | Making life hard for others since 1977. PGP 4BD6C0CB |
> > > >
> > > > On Fri, Jan 25, 2013 at 05:46:27AM +1100, Chris Boulton wrote:
> > > > > Seeing connectivity issues as traffic passes through Level 3 in
> > Dallas -
> > > > > not seeing the same issues for where routes converge (above.net in
> > DFW)
> > > > > over our TW Telecom link.
> > > > >
> > > > > Anybody else experiencing this as well?
> > > > >
> > > > > Level 3:
> > > > >
> > > > > Host Loss% Snt
> > > > > Last Avg Best Wrst StDev
> > > > > 1. vlan358.car1.Houston1.Level3.net 0.0%
> > 167
> > > > > 105.9 80.5 31.7 276.4 59.8
> > > > > 2. ae-2-5.bar1.Houston1.Level3.net 0.0%
> > 167
> > > > > 32.5 34.6 32.2 93.8 7.3
> > > > > 3. ae-13-13.ebr1.Dallas1.Level3.net 0.0%
> > 167
> > > > > 39.5 37.4 36.7 41.2 0.9
> > > > > 4. ae-91-91.csw4.Dallas1.Level3.net 65.1%
> > 167
> > > > > 44.5 43.5 39.7 53.8 4.0
> > > > > 5. ae-4-90.edge2.Dallas3.Level3.net 5.4%
> > 166
> > > > > 37.1 37.9 36.1 75.6 5.1
> > > > > 6. ex-3-1-0.er1.dfw2.us.above.net 66.1%
> > 166
> > > > > 39.5 43.5 38.9 104.8 12.5
> > > > > 7. xe-3-1-0.cr1.dfw2.us.above.net 0.0%
> > 166
> > > > > 37.3 40.4 37.0 114.1 10.6
> > > > > 8. xe-0-0-0.er3.dfw2.us.above.net 0.0%
> > 166
> > > > > 36.2 39.8 36.1 99.3 10.2
> > > > >
> > > > > Host Loss% Snt
> > > > > Last Avg Best Wrst StDev
> > > > > 1. 50.58.18.1 0.0% 829
> > > > > 4.3 23.9 4.1 121.2 14.5
> > > > > 2. dal2-pr2-xe-0-3-0-0.us.twtelecom.net 0.1%
> > 829
> > > > > 55.4 31.2 9.4 121.0 17.2
> > > > > dal2-pr2-xe-2-3-0-0.us.twtelecom.net
> > > > > 3. xe-3-0-0.er2.dfw2.us.above.net 0.1%
> > 829
> > > > > 44.8 31.6 9.5 143.7 17.9
> > > > > 4. xe-0-1-0.cr2.dfw2.us.above.net 0.2%
> > 829
> > > > > 40.5 31.2 10.3 116.2 15.9
> > > > > xe-3-0-0.cr2.dfw2.us.above.net
> > > > > 5. xe-0-2-0.er3.dfw2.us.above.net 0.2%
> > 829
> > > > > 42.7 32.4 9.6 109.2 16.5
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > > Chris Boulton
> > > > > Bigcommerce
> > > > >
> > > > > Email: chris at bigcommerce.com
> > > > >
> > > > > Web: http://www.bigcommerce.com/
> > > >
> > > > > _______________________________________________
> > > > > Outages mailing list
> > > > > Outages at outages.org
> > > > > https://puck.nether.net/mailman/listinfo/outages
> > > >
> > > >
> >
More information about the Outages
mailing list