[outages] Comcast in Silicon Valley -- Oakland or thereabouts just broke?
John Neiberger
jneiberger at gmail.com
Wed Oct 2 20:18:28 EDT 2013
I just checked into this. Looks like some sort of transport issue in sfba.
The appropriate teams have been working on it for a little while. I haven't
heard what the expected time to repair is yet.
John
On Oct 2, 2013 5:57 PM, "Jeremy Chadwick" <jdc at koitsu.org> wrote:
> This just happened -- looks like something in Comcast's network within
> Oakland broke, reconverged (quickly and seamless), but issues remain.
>
> Ren Provo et al, if you're lurking, might want to make someone aware of
> this (if they aren't already). :-)
>
> src: 76.102.14.35
> dst: 208.79.90.130
>
> === Wed Oct 2 16:14:00 PDT 2013 (1380755640)
> Start: Wed Oct 2 16:14:00 2013
> HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst
> 1.|-- 192.168.1.1 0.0% 40 40 0.4 0.3 0.2 0.4
> 2.|-- 76.102.12.1 0.0% 40 40 9.6 10.8 8.1 73.5
> 3.|-- 68.86.249.253 0.0% 40 40 8.9 10.2 8.3 62.2
> 4.|-- 69.139.199.98 0.0% 40 40 10.6 13.1 9.8 47.4
> 5.|-- 68.85.155.14 0.0% 40 40 12.0 16.8 11.4 87.7
> 6.|-- 68.86.90.157 0.0% 40 40 14.6 17.8 12.6 87.5
> 7.|-- 68.86.89.210 0.0% 40 40 23.7 24.7 20.6 92.9
> 8.|-- 68.86.87.210 0.0% 40 40 25.9 25.0 20.7 99.0
> 9.|-- 173.167.57.138 0.0% 40 40 22.1 24.7 21.3 89.0
> 10.|-- 69.31.127.141 0.0% 40 40 20.5 22.0 19.8 67.1
> 11.|-- 69.31.124.114 0.0% 40 40 20.6 21.6 20.0 45.9
> 12.|-- 67.199.135.102 0.0% 40 40 39.9 36.5 21.1 231.0
> 13.|-- 208.79.90.130 0.0% 40 40 21.2 20.7 20.2 21.7
> === END
>
> === Wed Oct 2 16:15:00 PDT 2013 (1380755700)
> Start: Wed Oct 2 16:15:00 2013
> HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst
> 1.|-- 192.168.1.1 0.0% 40 40 0.2 0.3 0.2 0.4
> 2.|-- 76.102.12.1 0.0% 40 40 8.1 9.4 8.1 13.6
> 3.|-- 68.86.249.253 0.0% 40 40 8.7 9.7 8.4 14.1
> 4.|-- 68.86.143.98 0.0% 40 40 13.1 13.1 10.2 21.1
> 5.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 6.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 7.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 8.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 9.|-- 173.167.57.138 0.0% 40 40 33.0 33.9 31.6 37.1
> 10.|-- 69.31.127.141 5.0% 40 38 30.2 31.4 29.7 39.3
> 11.|-- 69.31.124.114 7.5% 40 37 31.8 38.5 29.8 104.0
> 12.|-- 67.199.135.102 7.5% 40 37 31.1 36.6 30.6 119.7
> 13.|-- 208.79.90.130 7.5% 40 37 30.9 31.9 30.4 38.9
> === END
>
> 68.86.249.253 = te-0-2-0-5-ur06.santaclara.ca.sfba.comcast.net
> 69.139.199.98 = te-1-1-0-11-ar01.oakland.ca.sfba.comcast.net
> 68.85.155.14 = be-90-ar01.sfsutro.ca.sfba.comcast.net
> 68.86.143.98 = te-1-1-0-5-ar01.oakland.ca.sfba.comcast.net
>
> And packet loss has gradually been getting worse (impacting all forms of
> packets, not just TCP).
>
> === Wed Oct 2 16:45:00 PDT 2013 (1380757500)
> Start: Wed Oct 2 16:45:00 2013
> HOST: icarus.home.lan Loss% Snt Rcv Last Avg Best Wrst
> 1.|-- 192.168.1.1 0.0% 40 40 0.2 0.3 0.2 0.4
> 2.|-- 76.102.12.1 0.0% 40 40 8.6 9.9 8.1 14.3
> 3.|-- 68.86.249.253 0.0% 40 40 9.2 10.2 8.4 16.7
> 4.|-- 68.86.143.98 0.0% 40 40 13.9 13.1 10.6 16.6
> 5.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 6.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 7.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 8.|-- ??? 100.0 40 0 0.0 0.0 0.0 0.0
> 9.|-- 173.167.57.138 15.0% 40 34 34.6 34.8 31.7 41.7
> 10.|-- 69.31.127.141 25.0% 40 30 31.3 32.6 30.0 41.7
> 11.|-- 69.31.124.114 5.0% 40 38 31.8 33.5 30.5 58.6
> 12.|-- 67.199.135.102 12.8% 39 34 30.9 37.1 30.9 77.3
> 13.|-- 208.79.90.130 7.7% 39 36 32.9 32.3 30.8 37.3
> === END
>
> And here's what I see on the return path, for a brief run:
>
> src: 208.79.90.130
> dst: 76.102.14.35
>
> Host Loss% Snt Rcv Last
> Avg Best Wrst
> 1. 208.79.90.129 0.0% 93 93 1.5
> 6.1 0.9 163.0
> 2. s7.lax.arpnetworks.com 0.0% 93 93 0.7
> 11.2 0.5 150.5
> 3. ge0-15.as01.lax07.mzima.net 0.0% 93 93 4.1 2.4 0.4 64.6
> 4. ae3-50g.cr1.lax1.us.nlayer.net 0.0% 93 93 0.6 2.2 0.4 56.0
> 5. ae1-50g.ar1.lax1.us.nlayer.net 1.1% 93 92 3.1 3.3 1.3 6.4
> 6. be-102-pe01.600wseventh.ca.ibone.comcast.ne 0.0% 93 93 3.8 3.1 1.1 5.4
> 7. pos-0-8-0-0-cr01.losangeles.ca.ibone.comcas 0.0% 93 93 3.0
> 2.9 1.5 6.1
> 8. he-2-15-0-0-cr01.sanjose.ca.ibone.comcast.n 0.0% 92 92 8.6
> 10.3 8.1 12.5
> 9. he-0-9-0-0-ar01.sfsutro.ca.sfba.comcast.net 0.0% 92 92 12.5
> 11.6 9.8 13.6
> 10. be-100-ar01.oakland.ca.sfba.comcast.net 3.3% 92 88 22.0
> 25.2 21.9 49.6
> 11. te-0-4-0-2-ur06.santaclara.ca.sfba.comcast. 3.3% 92 89 23.8
> 23.8 23.0 25.4
> 12. te-6-0-acr03.santaclara.ca.sfba.comcast.net 9.8% 92 83 23.8
> 24.4 22.6 46.3
> 13. c-76-102-14-35.hsd1.ca.comcast.net 6.5% 92 86 36.3
> 31.7 30.5 38.3
>
> --
> | Jeremy Chadwick jdc at koitsu.org |
> | UNIX Systems Administrator http://jdc.koitsu.org/ |
> | Making life hard for others since 1977. PGP 4BD6C0CB |
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20131002/6db72942/attachment.htm>
More information about the Outages
mailing list