[outages] Comcast in Silicon Valley -- Oakland or thereabouts just broke?

Grant Ridder shortdudey123 at gmail.com
Wed Oct 2 20:41:55 EDT 2013


>From Mountain View i do not see web browsing issues, however traceroutes to
different websites (like google.com) break down in Oakland.

-Grant


On Wed, Oct 2, 2013 at 5:18 PM, John Neiberger <jneiberger at gmail.com> wrote:

> I just checked into this. Looks like some sort of transport issue in sfba.
> The appropriate teams have been working on it for a little while. I haven't
> heard what the expected time to repair is yet.
>
> John
> On Oct 2, 2013 5:57 PM, "Jeremy Chadwick" <jdc at koitsu.org> wrote:
>
>> This just happened -- looks like something in Comcast's network within
>> Oakland broke, reconverged (quickly and seamless), but issues remain.
>>
>> Ren Provo et al, if you're lurking, might want to make someone aware of
>> this (if they aren't already).  :-)
>>
>> src: 76.102.14.35
>> dst: 208.79.90.130
>>
>> === Wed Oct  2 16:14:00 PDT 2013  (1380755640)
>> Start: Wed Oct  2 16:14:00 2013
>> HOST: icarus.home.lan             Loss%   Snt   Rcv  Last   Avg  Best
>>  Wrst
>>   1.|-- 192.168.1.1                0.0%    40    40   0.4   0.3   0.2
>> 0.4
>>   2.|-- 76.102.12.1                0.0%    40    40   9.6  10.8   8.1
>>  73.5
>>   3.|-- 68.86.249.253              0.0%    40    40   8.9  10.2   8.3
>>  62.2
>>   4.|-- 69.139.199.98              0.0%    40    40  10.6  13.1   9.8
>>  47.4
>>   5.|-- 68.85.155.14               0.0%    40    40  12.0  16.8  11.4
>>  87.7
>>   6.|-- 68.86.90.157               0.0%    40    40  14.6  17.8  12.6
>>  87.5
>>   7.|-- 68.86.89.210               0.0%    40    40  23.7  24.7  20.6
>>  92.9
>>   8.|-- 68.86.87.210               0.0%    40    40  25.9  25.0  20.7
>>  99.0
>>   9.|-- 173.167.57.138             0.0%    40    40  22.1  24.7  21.3
>>  89.0
>>  10.|-- 69.31.127.141              0.0%    40    40  20.5  22.0  19.8
>>  67.1
>>  11.|-- 69.31.124.114              0.0%    40    40  20.6  21.6  20.0
>>  45.9
>>  12.|-- 67.199.135.102             0.0%    40    40  39.9  36.5  21.1
>> 231.0
>>  13.|-- 208.79.90.130              0.0%    40    40  21.2  20.7  20.2
>>  21.7
>> === END
>>
>> === Wed Oct  2 16:15:00 PDT 2013  (1380755700)
>> Start: Wed Oct  2 16:15:00 2013
>> HOST: icarus.home.lan             Loss%   Snt   Rcv  Last   Avg  Best
>>  Wrst
>>   1.|-- 192.168.1.1                0.0%    40    40   0.2   0.3   0.2
>> 0.4
>>   2.|-- 76.102.12.1                0.0%    40    40   8.1   9.4   8.1
>>  13.6
>>   3.|-- 68.86.249.253              0.0%    40    40   8.7   9.7   8.4
>>  14.1
>>   4.|-- 68.86.143.98               0.0%    40    40  13.1  13.1  10.2
>>  21.1
>>   5.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   6.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   7.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   8.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   9.|-- 173.167.57.138             0.0%    40    40  33.0  33.9  31.6
>>  37.1
>>  10.|-- 69.31.127.141              5.0%    40    38  30.2  31.4  29.7
>>  39.3
>>  11.|-- 69.31.124.114              7.5%    40    37  31.8  38.5  29.8
>> 104.0
>>  12.|-- 67.199.135.102             7.5%    40    37  31.1  36.6  30.6
>> 119.7
>>  13.|-- 208.79.90.130              7.5%    40    37  30.9  31.9  30.4
>>  38.9
>> === END
>>
>> 68.86.249.253 = te-0-2-0-5-ur06.santaclara.ca.sfba.comcast.net
>> 69.139.199.98 = te-1-1-0-11-ar01.oakland.ca.sfba.comcast.net
>> 68.85.155.14  = be-90-ar01.sfsutro.ca.sfba.comcast.net
>> 68.86.143.98  = te-1-1-0-5-ar01.oakland.ca.sfba.comcast.net
>>
>> And packet loss has gradually been getting worse (impacting all forms of
>> packets, not just TCP).
>>
>> === Wed Oct  2 16:45:00 PDT 2013  (1380757500)
>> Start: Wed Oct  2 16:45:00 2013
>> HOST: icarus.home.lan             Loss%   Snt   Rcv  Last   Avg  Best
>>  Wrst
>>   1.|-- 192.168.1.1                0.0%    40    40   0.2   0.3   0.2
>> 0.4
>>   2.|-- 76.102.12.1                0.0%    40    40   8.6   9.9   8.1
>>  14.3
>>   3.|-- 68.86.249.253              0.0%    40    40   9.2  10.2   8.4
>>  16.7
>>   4.|-- 68.86.143.98               0.0%    40    40  13.9  13.1  10.6
>>  16.6
>>   5.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   6.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   7.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   8.|-- ???                       100.0    40     0   0.0   0.0   0.0
>> 0.0
>>   9.|-- 173.167.57.138            15.0%    40    34  34.6  34.8  31.7
>>  41.7
>>  10.|-- 69.31.127.141             25.0%    40    30  31.3  32.6  30.0
>>  41.7
>>  11.|-- 69.31.124.114              5.0%    40    38  31.8  33.5  30.5
>>  58.6
>>  12.|-- 67.199.135.102            12.8%    39    34  30.9  37.1  30.9
>>  77.3
>>  13.|-- 208.79.90.130              7.7%    39    36  32.9  32.3  30.8
>>  37.3
>> === END
>>
>> And here's what I see on the return path, for a brief run:
>>
>> src: 208.79.90.130
>> dst: 76.102.14.35
>>
>>  Host                                           Loss%   Snt   Rcv  Last
>> Avg  Best  Wrst
>>  1. 208.79.90.129                                0.0%    93    93   1.5
>>   6.1   0.9 163.0
>>  2. s7.lax.arpnetworks.com                       0.0%    93    93   0.7
>>  11.2   0.5 150.5
>>  3. ge0-15.as01.lax07.mzima.net                  0.0%    93 93 4.1 2.4
>> 0.4  64.6
>>  4. ae3-50g.cr1.lax1.us.nlayer.net               0.0%    93 93 0.6 2.2
>> 0.4  56.0
>>  5. ae1-50g.ar1.lax1.us.nlayer.net               1.1%    93 92 3.1 3.3
>> 1.3   6.4
>>  6. be-102-pe01.600wseventh.ca.ibone.comcast.ne  0.0%    93 93 3.8 3.1
>> 1.1   5.4
>>  7. pos-0-8-0-0-cr01.losangeles.ca.ibone.comcas  0.0%    93    93   3.0
>>   2.9   1.5   6.1
>>  8. he-2-15-0-0-cr01.sanjose.ca.ibone.comcast.n  0.0%    92    92   8.6
>>  10.3   8.1  12.5
>>  9. he-0-9-0-0-ar01.sfsutro.ca.sfba.comcast.net  0.0%    92    92  12.5
>>  11.6   9.8  13.6
>> 10. be-100-ar01.oakland.ca.sfba.comcast.net      3.3%    92    88  22.0
>>  25.2  21.9  49.6
>> 11. te-0-4-0-2-ur06.santaclara.ca.sfba.comcast.  3.3%    92    89  23.8
>>  23.8  23.0  25.4
>> 12. te-6-0-acr03.santaclara.ca.sfba.comcast.net  9.8%    92    83  23.8
>>  24.4  22.6  46.3
>> 13. c-76-102-14-35.hsd1.ca.comcast.net           6.5%    92    86  36.3
>>  31.7  30.5  38.3
>>
>> --
>> | Jeremy Chadwick                                   jdc at koitsu.org |
>> | UNIX Systems Administrator                http://jdc.koitsu.org/ |
>> | Making life hard for others since 1977.             PGP 4BD6C0CB |
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20131002/ced2f855/attachment.htm>


More information about the Outages mailing list