[outages] Comcast LA issue?

Thu Feb 19 18:33:12 EST 2015

I'll take a look. West coast appears to be haunted today.

John

On Thu, Feb 19, 2015 at 4:07 PM, Jeremy Chadwick via Outages <
outages at outages.org> wrote:

> Something weird is going on with Comcast in Los Angeles: small packets
> appear to be getting discarded with a loss rate of ~6-10% while larger
> packets (I've tried 1000 and 1200) seem to make it through without any
> loss.  Things like SSH and IMAP+SSL are impacted consistently
> (particularly during key exchange), while HTTP does not appear to be, so
> the issue appears to be affecting both TCP and ICMP (have not tested
> UDP).
>
> First sign of this issue, according to my logs, is at 11:51 PT (UTC-8),
> and has gradually gotten worse over the past few hours.
>
> Looks like the issue is with the forward path (Source->Destination
> listed below), as the return path only shows loss at the final
> destination.  (I currently cannot provide a return-path mtr because the
> only way I can access my box is through VNC, and I don't like doing
> things like screenshots of a VNC window, so you'll just have to trust
> me)
>
> Nothing anomalous shows up when using L3's looking glass to ping things,
> and the only Comcast LG I know of is in NYC.
>
> I most likely won't see Email responses from anyone (directly or to the
> list) until this problem is addressed, since my mail is hosted on shown
> box and cannot currently be fetched due to the problem.  Frustrating.
>
> Source IP: 69.181.142.213
> Destination IP: 174.136.108.146
>
> ICMP 56 bytes:
>
>
> Packets               Pings
>  Host                                                               Loss%
>  Snt   Rcv  Last   Avg  Best  Wrst
>  1. gw.home.lan (192.168.1.1)                                        0.0%
>  185   185   0.2   0.2   0.2   0.3
>  2. 69.181.136.1                                                     0.0%
>  185   185   8.9   9.9   7.0  21.1
>  3. te-0-2-0-5-sur03.santaclara.ca.sfba.comcast.net (68.86.249.69)
>  0.0%   185   185   8.7   9.9   7.9  17.0
>  4. te-0-0-0-13-sur03.sanrafael.ca.sfba.comcast.net (162.151.78.134
> 0.0%   185   185  10.7  12.9   9.7  20.7
>  5. he-1-5-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.93)
> 98.4%   184     3 49496 49006 47906 49615
>  6. he-2-9-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.102)
>  5.4%   184   174  11.9  14.1  10.9  23.5
>  7. 50.242.148.34                                                    6.5%
>  184   172  11.7  13.4  10.4  69.9
>  8. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142)
>  66.7%   184    61  44.9  20.6  18.0  44.9
>  9. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142)
>  59.0%   184    75  18.7  19.8  18.0  26.0
> 10. ae-27-27.car2.LosAngeles1.Level3.net (4.69.201.18)
>  2.2%   184   180  20.8  63.1  18.2 293.3
> 11. GIGLINX-INC.car2.LosAngeles1.Level3.net (4.30.195.86)
> 6.0%   184   173  19.2  22.2  19.1  39.3
> 12. mambo.koitsu.org (174.136.108.146)
>  6.5%   184   172  18.8  20.6  18.5  36.9
>
> Note: Loss shown at above hops #5, #8, and #9 are normal according to
> historic data (i.e. ICMP prioritisation is in effect) -- however loss at
> hop #6 is new.
>
>
> ICMP 1200 bytes:
>
>
> Packets               Pings
>  Host                                                               Loss%
>  Snt   Rcv  Last   Avg  Best  Wrst
>  1. gw.home.lan (192.168.1.1)                                        0.0%
>   43    43   0.3   0.3   0.2   0.4
>  2. 69.181.136.1                                                     0.0%
>   43    43   9.7  10.4   8.6  14.3
>  3. te-0-2-0-5-sur03.santaclara.ca.sfba.comcast.net (68.86.249.69)
>  0.0%    42    42  10.0  10.0   8.7  16.0
>  4. te-0-0-0-13-sur03.sanrafael.ca.sfba.comcast.net (162.151.78.134
> 0.0%    42    42  11.7  13.3  10.3  18.9
>  5. ???
>  6. he-2-9-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.102)
>  0.0%    42    42  14.4  15.9  12.4  25.1
>  7. 50.242.148.34                                                    0.0%
>   42    42  12.7  13.6  11.2  23.1
>  8. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142)
> 9.8%    42    37  20.0  20.7  18.8  25.0
>  9. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142)
> 0.0%    42    42  19.1  21.1  18.9  29.9
> 10. ae-27-27.car2.LosAngeles1.Level3.net (4.69.201.18)
>  0.0%    42    42  64.3 105.7  20.9 412.5
> 11. GIGLINX-INC.car2.LosAngeles1.Level3.net (4.30.195.86)
> 0.0%    42    42  20.6  22.5  19.9  31.2
> 12. mambo.koitsu.org (174.136.108.146)
>  0.0%    42    42  20.7  22.0  20.1  26.4
>
> --
> | Jeremy Chadwick                                   jdc at koitsu.org |
> | UNIX Systems Administrator                http://jdc.koitsu.org/ |
> | Making life hard for others since 1977.             PGP 4BD6C0CB |
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20150219/086d617d/attachment.htm>