[outages] comcast chicago blackholing

[John Neiberger]

Thanks, all. Our side should be resolved now. Had a combination of a
linecard fabric failure and a partial RSP failure. We're back in
working shape and BGP from our core to that edge router is back up.
Should be safe for peers to turn sessions back up.

John

On Tue, Nov 17, 2015 at 10:15 AM, Jared Mauch <jared at puck.nether.net> wrote:
>         This is both an issue over V6 and V4, eg:
>
> Jareds-Mac-mini:~% traceroute puck
> traceroute to puck.nether.net (204.42.254.5), 64 hops max, 52 byte packets
>  1  10.0.0.1 (10.0.0.1)  0.603 ms  0.418 ms  0.403 ms
>  2  173-167-0-110-michigan.hfc.comcastbusiness.net (173.167.0.110)  2.044 ms  2.019 ms  2.179 ms
>  3  96.120.40.141 (96.120.40.141)  17.399 ms  12.408 ms  9.639 ms
>  4  68.85.85.37 (68.85.85.37)  14.371 ms  8.378 ms  15.355 ms
>  5  te-0-8-0-6-ar02.pontiac.mi.michigan.comcast.net (68.87.191.153)  14.025 ms
>     te-0-8-0-8-ar02.pontiac.mi.michigan.comcast.net (69.139.255.69)  14.749 ms  18.706 ms
>  6  be-33668-cr02.350ecermak.il.ibone.comcast.net (68.86.90.45)  21.955 ms  20.456 ms  19.259 ms
>  7  he-0-16-0-0-pe03.350ecermak.il.ibone.comcast.net (68.86.88.146)  18.333 ms  17.245 ms  16.952 ms
> ^C
>
> Jareds-Mac-mini:~% traceroute6 puck
> traceroute6 to puck.nether.net (2001:418:3f4::5) from 2601:401:3:6a00:ad42:ce75:8a0:c370, 64 hops max, 12 byte packets
>  1  2601:401:3:6a00:20f:f7ff:feab:1ec0  1.135 ms  0.965 ms  0.948 ms
>  2  2601:401:3:6a00:4694:fcff:fecd:64bc  3.710 ms  3.544 ms  21.221 ms
>  3  2001:558:6007:24::1  30.051 ms  10.909 ms  10.868 ms
>  4  te8-2-ur02.wannarbor.mi.michigan.comcast.net  10.664 ms  10.451 ms  10.570 ms
>  5  xe-11-1-0-0-sur01.rochestrhlls.mi.michigan.comcast.net  14.826 ms
>     te-0-8-0-5-ar02.taylor.mi.michigan.comcast.net  13.890 ms  13.179 ms
>  6  te-0-7-0-13-ar01.pontiac.mi.michigan.comcast.net  18.917 ms  15.503 ms
>     te-0-8-0-11-ar01.pontiac.mi.michigan.comcast.net  13.686 ms
>  7  be-33668-cr01.350ecermak.il.ibone.comcast.net  22.316 ms  22.014 ms  28.086 ms
>  8  hu-0-16-0-1-pe03.350ecermak.il.ibone.comcast.net  20.933 ms  20.740 ms  20.831 ms
> ^C
>
>         - Jared
>
>
> On Tue, Nov 17, 2015 at 10:02:58AM -0700, John Neiberger wrote:
>> Can you give me some more details please? I'll take a look.
>>
>> Thanks,
>> John
>>
>> On Tue, Nov 17, 2015 at 10:00 AM, Jared Mauch via Outages
>> <outages at outages.org> wrote:
>> >
>> >         Anyone know what is going on here?  It seems to have started
>> > around 1600 utc and is still ongoing.
>> >
>> >         - Jared
>> >
>> > --
>> > Jared Mauch  | pgp key available via finger from jared at puck.nether.net
>> > clue++;      | http://puck.nether.net/~jared/  My statements are only mine.
>> > _______________________________________________
>> > Outages mailing list
>> > Outages at outages.org
>> > https://puck.nether.net/mailman/listinfo/outages
>
> --
> Jared Mauch  | pgp key available via finger from jared at puck.nether.net
> clue++;      | http://puck.nether.net/~jared/  My statements are only mine.