[outages] Level3 problems in Chicago?

Jason Baugher jason at thebaughers.com
Thu Dec 13 10:28:50 EST 2012


I just heard back from Level3 confirming that there is an issue affecting
multiple customers being investigated, although I don't have any details.
My last test showed that it would work if I turned it back up, but it would
route from STL to Denver and then to CHI:

Tracing the route to tcsmonitor.trendcs.com (198.17.225.17)

  1 vlan372.car2.StLouis1.Level3.net (4.53.162.241) [AS 3356] 80 msec 216
msec 132 msec
  2 ae-11-11.car1.StLouis1.Level3.net (4.69.132.185) [AS 3356] [MPLS: Label
325 Exp 0] 244 msec 204 msec 116 msec
  3 ae-4-4.ebr1.Denver1.Level3.net (4.69.132.182) [AS 3356] [MPLS: Label
1481 Exp 0] 20 msec 32 msec 32 msec
  4 ae-3-3.ebr1.Chicago2.Level3.net (4.69.132.62) [AS 3356] [MPLS: Label
1173 Exp 0] 52 msec 48 msec 48 msec
  5 ae-6-6.ebr1.Chicago1.Level3.net (4.69.140.189) [AS 3356] [MPLS: Label
1096 Exp 0] 56 msec
    ae-1-100.ebr2.Chicago2.Level3.net (4.69.132.114) [AS 3356] [MPLS: Label
1233 Exp 0] 48 msec
    ae-6-6.ebr1.Chicago1.Level3.net (4.69.140.189) [AS 3356] [MPLS: Label
1096 Exp 0] 56 msec
  6  *  *  *
  7 ae-1-9.bar2.Cleveland1.Level3.net (4.69.136.193) [AS 3356] [MPLS: Label
299776 Exp 0] 52 msec 52 msec 52 msec
  8 ae-0-11.bar1.Cleveland1.Level3.net (4.69.136.185) [AS 3356] 56 msec 56
msec 56 msec
  9 FIDELITY-AC.bar1.Cleveland1.Level3.net (4.53.196.30) [AS 3356] 56 msec
56 msec 56 msec
 10 ge-1-0-1.ar2.kth.cle.oh.fidelityaccess.net (66.94.64.7) [AS 22958] 56
msec 56 msec 56 msec
 11 net-207-58-250-91.arpa.fidelityaccess.net (207.58.250.91) [AS 22958] 52
msec 56 msec 56 msec




On Thu, Dec 13, 2012 at 9:10 AM, Mitch <mitpatterson at gmail.com> wrote:

> Not sure what your issue is. I'm in Ohio so from here I don't hit chicago
> but I did traceroutes from Level3's looking glass in both Chicago and St.
> Louis(since it looks likes thats where you are coming from:
>
> Chicago makes it to the last hop before the last host, not sure if the
> last host blocks it:
>
> *Show Level 3 (Chicago, IL) Traceroute to 198.17.225.17*
>
>   1 ae-32-52.ebr2.Chicago1.Level3.net (4.69.138.62) 0 msec 0 msec 0 msec
>   2 ae-1-9.bar2.Cleveland1.Level3.net (4.69.136.193) 8 msec 12 msec 8 msec
>   3 ae-0-11.bar1.Cleveland1.Level3.net (4.69.136.185) 12 msec 8 msec 8 msec
>   4 FIDELITY-AC.bar1.Cleveland1.Level3.net (4.53.196.30) 12 msec 8 msec 12 msec
>   5 ge-1-0-1.ar2.kth.cle.oh.fidelityaccess.net (66.94.64.7) [AS22958 {FIDELITY-001}] 8 msec 8 msec 12 msec
>   6 net-207-58-250-91.arpa.fidelityaccess.net (207.58.250.91) [AS22958 {FIDELITY-001}] 8 msec 8 msec 12 msec
>   7  *  *  *
>   8  *  *  *
>   9  *  *  *
>  10  *  *  *
>  11  *  *  *
>  12  *  *  *
>  13  *  *  *
>  14  *  *  *
>  15  *  *  *
>  16  *  *  *
>  17  *  *  *
>  18  *  *  *
>  19  *  *  * timeout !
>
> From St. Louis it shows the same thing. It definity makes it though
> chicago:
> *Show Level 3 (Saint Louis, MO) Traceroute to 198.17.225.17*
>
>   1 ae-11-11.car2.StLouis1.Level3.net (4.69.132.186) 192 msec 268 msec 208 msec
>   2 4.69.201.1 100 msec 84 msec 68 msec
>   3 ae-6-6.ebr1.Chicago1.Level3.net (4.69.140.189) 40 msec
>     ae-1-100.ebr2.Chicago2.Level3.net (4.69.132.114) 148 msec 196 msec
>   4 ae-5-5.ebr2.Chicago1.Level3.net (4.69.140.193) 8 msec
>     ae-1-100.ebr2.Chicago1.Level3.net (4.69.151.178) 4 msec 12 msec
>   5 ae-1-9.bar2.Cleveland1.Level3.net (4.69.136.193) 36 msec 16 msec 16 msec
>   6 ae-0-11.bar1.Cleveland1.Level3.net (4.69.136.185) 12 msec 16 msec 12 msec
>   7 FIDELITY-AC.bar1.Cleveland1.Level3.net (4.53.196.30) 16 msec 12 msec 16 msec
>   8 ge-1-0-1.ar2.kth.cle.oh.fidelityaccess.net (66.94.64.7) [AS22958 {FIDELITY-001}] 16 msec 16 msec 16 msec
>   9 net-207-58-250-91.arpa.fidelityaccess.net (207.58.250.91) [AS22958 {FIDELITY-001}] 16 msec 16 msec 16 msec
>  10  *  *  *
>  11  *  *  *
>  12  *  *  *
>  13  *  *  *
>  14  *  *  *
>  15  *  *  *
>  16  *  *  *
>  17  *  *  *
>  18  *  *  *
>  19  *  *  *
>  20  *  *  *
>  21  *  *  *
>  22  *  *  * timeout !
>
> On Thu, Dec 13, 2012 at 9:50 AM, Jason Baugher <jason at thebaughers.com>wrote:
>
>> Starting at 8am Central Time, we're seeing issues with Level3 in Chicago.
>> I don't have access to the far end to do a traceroute from the far side.
>>
>> Traceroute with our Level3 peering up:
>>
>> Tracing the route to tcsmonitor.trendcs.com (198.17.225.17)
>>
>>   1 vlan372.car2.StLouis1.Level3.net (4.53.162.241) [AS 3356] 160 msec 8
>> msec 4 msec
>>   2 4.69.201.1 [AS 3356] [MPLS: Label 1173 Exp 0] 16 msec 8 msec 16 msec
>>   3 ae-1-100.ebr2.Chicago2.Level3.net (4.69.132.114) [AS 3356] [MPLS:
>> Label 1233 Exp 0] 8 msec
>>     ae-6-6.ebr1.Chicago1.Level3.net (4.69.140.189) [AS 3356] [MPLS:
>> Label 1096 Exp 0] 12 msec
>>     ae-1-100.ebr2.Chicago2.Level3.net (4.69.132.114) [AS 3356] [MPLS:
>> Label 1233 Exp 0] 8 msec
>>   4  *  *  *
>>   5  *  *  *
>>   6  *  *  *
>>
>>
>> Traceroute with it down, traffic moved to Sprint:
>>
>> Tracing the route to tcsmonitor.trendcs.com (198.17.225.17)
>>
>>   1 qncy-pc-cat-0001.adams.net (66.226.125.198) 4 msec 0 msec 0 msec
>>   2 sl-st20-chi-9-0-1.sprintlink.net (144.223.60.49) [AS 1239] 8 msec 4
>> msec 8 msec
>>   3 144.232.8.114 [AS 1239] 8 msec 4 msec 4 msec
>>   4 vlan52.ebr2.Chicago2.Level3.net (4.69.138.190) [AS 3356] [MPLS:
>> Label 1233 Exp 0] 8 msec 16 msec 4 msec
>>   5  *  *  *
>>   6 ae-1-9.bar2.Cleveland1.Level3.net (4.69.136.193) [AS 3356] [MPLS:
>> Label 299776 Exp 0] 24 msec 64 msec 52 msec
>>   7 ae-0-11.bar1.Cleveland1.Level3.net (4.69.136.185) [AS 3356] 24 msec
>> 24 msec 24 msec
>>   8 FIDELITY-AC.bar1.Cleveland1.Level3.net (4.53.196.30) [AS 3356] 136
>> msec 212 msec 212 msec
>>   9 ge-1-0-1.ar2.kth.cle.oh.fidelityaccess.net (66.94.64.7) [AS 22958]
>> 16 msec 16 msec 16 msec
>>  10 net-207-58-250-91.arpa.fidelityaccess.net (207.58.250.91) [AS 22958]
>> 16 msec 20 msec 16 msec
>>
>>
>> We've opened a ticket with Level3, just wondering if anyone else has
>> noticed problems this morning.
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20121213/a2a4d1f9/attachment.htm>


More information about the Outages mailing list