[outages] Comcast LA issue?
Jeremy Chadwick
jdc at koitsu.org
Thu Feb 19 18:25:40 EST 2015
Here's another destination showing the same problem (I can't even load
http://www.dslreports.com/ at present -- handshake works, but either
client->server GET request gets lost, or payload response gets lost,
followed by TCP retrans, eventually RST from the server, rinse lather
repeat), with things again looking centralised around LA.
Hopefully having multiple test endpoints will come in handy.
Source IP: 69.181.142.213
Destination IP: 64.91.255.98
Packets Pings
Host Loss% Snt Rcv Last Avg Best Wrst
1. gw.home.lan (192.168.1.1) 0.0% 171 171 0.2 0.2 0.2 0.4
2. 69.181.136.1 0.0% 171 171 12.7 9.6 7.7 19.2
3. te-0-2-0-5-sur03.santaclara.ca.sfba.comcast.net (68.86.249.69) 0.0% 171 171 9.4 9.2 8.0 17.1
4. te-0-0-0-13-sur03.sanrafael.ca.sfba.comcast.net (162.151.78.134 0.0% 171 171 12.1 12.2 9.5 18.4
5. he-1-5-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.93) 98.8% 171 2 50456 50095 49735 50456
6. he-2-9-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.102) 2.9% 171 166 14.1 13.5 10.9 22.7
7. he-2-12-0-0-cr01.seattle.wa.ibone.comcast.net (68.86.86.229) 1.8% 171 168 29.2 30.2 27.4 36.3
8. he-0-15-0-0-cr01.denver.co.ibone.comcast.net (68.86.86.213) 9.9% 171 154 50.1 47.6 44.9 53.1
9. he-5-13-0-0-cr01.350ecermak.il.ibone.comcast.net (68.86.85.245) 4.1% 171 163 72.9 72.3 68.6 84.7
10. hu-0-10-0-1.pe03.350ecermak.il.ibone.comcast.net (68.86.86.6) 5.3% 170 161 69.8 69.4 68.1 84.7
11. 50.242.150.126 7.1% 170 158 106.0 89.7 67.4 400.5
50.242.150.130
12. lw-dc3-core1-te8-16.rtr.liquidweb.com (209.59.157.244) 9.4% 170 154 75.7 80.9 75.2 242.7
13. lw-dc3-dist15-po5.rtr.liquidweb.com (69.167.128.241) 2.4% 170 166 75.0 75.9 74.7 83.6
14. www.dslreports.com (64.91.255.98) 5.3% 170 161 74.2 74.3 73.2 82.5
--
| Jeremy Chadwick jdc at koitsu.org |
| UNIX Systems Administrator http://jdc.koitsu.org/ |
| Making life hard for others since 1977. PGP 4BD6C0CB |
On Thu, Feb 19, 2015 at 03:07:04PM -0800, Jeremy Chadwick wrote:
> Something weird is going on with Comcast in Los Angeles: small packets
> appear to be getting discarded with a loss rate of ~6-10% while larger
> packets (I've tried 1000 and 1200) seem to make it through without any
> loss. Things like SSH and IMAP+SSL are impacted consistently
> (particularly during key exchange), while HTTP does not appear to be, so
> the issue appears to be affecting both TCP and ICMP (have not tested
> UDP).
>
> First sign of this issue, according to my logs, is at 11:51 PT (UTC-8),
> and has gradually gotten worse over the past few hours.
>
> Looks like the issue is with the forward path (Source->Destination
> listed below), as the return path only shows loss at the final
> destination. (I currently cannot provide a return-path mtr because the
> only way I can access my box is through VNC, and I don't like doing
> things like screenshots of a VNC window, so you'll just have to trust
> me)
>
> Nothing anomalous shows up when using L3's looking glass to ping things,
> and the only Comcast LG I know of is in NYC.
>
> I most likely won't see Email responses from anyone (directly or to the
> list) until this problem is addressed, since my mail is hosted on shown
> box and cannot currently be fetched due to the problem. Frustrating.
>
> Source IP: 69.181.142.213
> Destination IP: 174.136.108.146
>
> ICMP 56 bytes:
>
> Packets Pings
> Host Loss% Snt Rcv Last Avg Best Wrst
> 1. gw.home.lan (192.168.1.1) 0.0% 185 185 0.2 0.2 0.2 0.3
> 2. 69.181.136.1 0.0% 185 185 8.9 9.9 7.0 21.1
> 3. te-0-2-0-5-sur03.santaclara.ca.sfba.comcast.net (68.86.249.69) 0.0% 185 185 8.7 9.9 7.9 17.0
> 4. te-0-0-0-13-sur03.sanrafael.ca.sfba.comcast.net (162.151.78.134 0.0% 185 185 10.7 12.9 9.7 20.7
> 5. he-1-5-0-0-cr01.sanjose.ca.ibone.comcast.net (68.86.90.93) 98.4% 184 3 49496 49006 47906 49615
> 6. he-2-9-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.102) 5.4% 184 174 11.9 14.1 10.9 23.5
> 7. 50.242.148.34 6.5% 184 172 11.7 13.4 10.4 69.9
> 8. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142) 66.7% 184 61 44.9 20.6 18.0 44.9
> 9. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142) 59.0% 184 75 18.7 19.8 18.0 26.0
> 10. ae-27-27.car2.LosAngeles1.Level3.net (4.69.201.18) 2.2% 184 180 20.8 63.1 18.2 293.3
> 11. GIGLINX-INC.car2.LosAngeles1.Level3.net (4.30.195.86) 6.0% 184 173 19.2 22.2 19.1 39.3
> 12. mambo.koitsu.org (174.136.108.146) 6.5% 184 172 18.8 20.6 18.5 36.9
>
> Note: Loss shown at above hops #5, #8, and #9 are normal according to
> historic data (i.e. ICMP prioritisation is in effect) -- however loss at
> hop #6 is new.
>
>
> ICMP 1200 bytes:
>
> Packets Pings
> Host Loss% Snt Rcv Last Avg Best Wrst
> 1. gw.home.lan (192.168.1.1) 0.0% 43 43 0.3 0.3 0.2 0.4
> 2. 69.181.136.1 0.0% 43 43 9.7 10.4 8.6 14.3
> 3. te-0-2-0-5-sur03.santaclara.ca.sfba.comcast.net (68.86.249.69) 0.0% 42 42 10.0 10.0 8.7 16.0
> 4. te-0-0-0-13-sur03.sanrafael.ca.sfba.comcast.net (162.151.78.134 0.0% 42 42 11.7 13.3 10.3 18.9
> 5. ???
> 6. he-2-9-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.86.102) 0.0% 42 42 14.4 15.9 12.4 25.1
> 7. 50.242.148.34 0.0% 42 42 12.7 13.6 11.2 23.1
> 8. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142) 9.8% 42 37 20.0 20.7 18.8 25.0
> 9. ae-3-80.edge6.LosAngeles1.Level3.net (4.69.144.142) 0.0% 42 42 19.1 21.1 18.9 29.9
> 10. ae-27-27.car2.LosAngeles1.Level3.net (4.69.201.18) 0.0% 42 42 64.3 105.7 20.9 412.5
> 11. GIGLINX-INC.car2.LosAngeles1.Level3.net (4.30.195.86) 0.0% 42 42 20.6 22.5 19.9 31.2
> 12. mambo.koitsu.org (174.136.108.146) 0.0% 42 42 20.7 22.0 20.1 26.4
>
> --
> | Jeremy Chadwick jdc at koitsu.org |
> | UNIX Systems Administrator http://jdc.koitsu.org/ |
> | Making life hard for others since 1977. PGP 4BD6C0CB |
>
More information about the Outages
mailing list