[outages] Packet loss to 8.8.8.8 and other google IPs (Dallas)

[Damian Menscher]

This appears to be a problem on the Level3 network in the DFW region, which
is why only some of you see it.  (Those who don't see it are in other
regions, peering with Google directly, or using other transit providers.)

For those seeing issues, I suggest escalating to Level3, perhaps after
trying a reverse-traceroute from http://lg.level3.net/ (I can't do this
test since you didn't specify your source IP.)

Damian

On Mon, Aug 29, 2016 at 11:14 AM, Dan White <dwhite at olp.net> wrote:

> We are seeing packet loss to google hosts, hosted in Dallas. Any anyone
> seeing this?
>
> MTR output:
>
> Host                              Loss%   Snt   Last   Avg  Best  Wrst
> StDev
> 1. 10.0.2.1                        0.0%    51    2.4   1.9   0.5   5.2
>  0.7
> 2. olp-67-217-152-5.olp.net        0.0%    51    0.4   3.7   0.3  27.8
>  4.9
> 3. olp-67-217-158-29.olp.net       0.0%    50    0.3   4.5   0.3  22.0
>  5.3
> 4. olp-67-217-151-125.olp.net      0.0%    50    0.6   4.9   0.6  33.3
>  6.8
> 5. olp-67-217-158-30.olp.net       0.0%    50    0.6   6.4   0.5  64.0
> 10.5
> 6. olp-67-217-158-21.olp.net       0.0%    50   11.4   4.4   1.1  13.6
>  4.1
> 7. xe-8-3-0.bar2.tulsa2.level3.net 0.0%    50    1.2   2.7   1.2  27.5
>  5.5
> 8. ae-0-11.bar1.tulsa2.level3.net  0.0%    50    2.0   4.1   1.9  44.5
>  7.8
> 9. ae-3-3.ebr4.dallas1.level3.net 91.8%    50   20.5  20.5  20.5  20.5
>  0.0
> 10. ae-8-0.ear3.dallas1.level3.net 93.9%    50   20.6  20.6  20.6  20.7
>  0.0
> 11. google-level3-100g.dallas1.leve28.0%    50   20.5  20.7  20.4  29.6
>  1.5
> 12. 108.170.240.193                22.0%    50   20.8  20.7  20.7  20.9
>  0.0
> 13. 64.233.175.9                   26.0%    50   20.7  20.7  20.7  20.8
>  0.0
> 14. google-public-dns-a.google.com 24.0%    50   20.5  20.6  20.5  25.9
>  0.9
>
> The problematic hop appears to be either 10 or 11 (level3-100g). Compare
> to:
>
> Host                              Loss%   Snt   Last   Avg  Best  Wrst
> StDev
> 1. 10.0.2.1                        0.0%   124    0.8   1.6   0.5   4.6
>  0.5
> 2. olp-67-217-152-5.olp.net        0.0%   124    3.8   4.3   0.3  29.9
>  5.6
> 3. olp-67-217-158-29.olp.net       0.0%   124    0.3   3.5   0.3  26.1
>  4.9
> 4. olp-67-217-151-125.olp.net      0.0%   124    6.6   3.3   0.6  20.0
>  3.7
> 5. olp-67-217-158-30.olp.net       0.0%   124    0.6   7.1   0.6  44.1
>  9.8
> 6. olp-67-217-158-21.olp.net       0.8%   124   11.9   4.8   1.1  14.9
>  4.3
> 7. xe-8-3-0.bar2.tulsa2.level3.net 0.0%   123    1.3   4.0   1.1  68.3
> 11.3
> 8. ae-0-11.bar1.tulsa2.level3.net  0.8%   123    2.1   3.9   1.9  62.3
>  8.1
> 9. ae-3-3.ebr4.dallas1.level3.net 94.3%   123   20.5  20.6  20.5  20.6
>  0.0
> 10. ae-2-3512.edge2.dallas1.level3. 0.0%   123   20.4  22.1  20.3  68.0
>  7.1
> 11. b.resolvers.level3.net          0.0%   123   20.2  20.3  20.2  22.8
>  0.3
>
> --
> Dan White
> BTC Broadband
> Network Admin Lead
> Ph  918.366.0248 (direct)   main: (918)366-8000
> Fax 918.366.6610            email: dwhite at olp.net
> http://www.btcbroadband.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "external-netops" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to external-netops+unsubscribe at google.com.
> To post to this group, send email to external-netops at google.com.
> Visit this group at https://groups.google.com/a/go
> ogle.com/group/external-netops/.
> For more options, visit https://groups.google.com/a/google.com/d/optout.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20160829/f2413b5c/attachment.htm>