[outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too?
Braden McGrath
braden.mcgrath at saberhealth.com
Fri May 12 14:38:53 EDT 2017
The circuits bouncing is most likely Comcast doing rolling DOCSIS firmware updates, assuming you all are seeing it on coax circuits. I've seen it as well over in the VA region.
Thank you,
Braden McGrath
IT Systems Engineer
Saber Healthcare Group
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Rob Hall via Outages
Sent: Friday, May 12, 2017 7:05 AM
To: outages at outages.org
Subject: Re: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too?
I've been seeing this same behavior. Connection bouncing, sometimes multiple times, in the middle of the night. Had one branch office in IL bounce 3 times night before last.
Thanks!
--
Rob Hall, CCNP R&S, CCP-V
I.T. Manager
@Work Personnel Service
3815 Hwy 66 S, Suite 4
Rogersville, TN 37857
Office: (423)272-7897, x10113
Mobile: (423)312-6003
[cisco-certified-ccnp][CCP-V]
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Dave Wagner via Outages
Sent: Thursday, May 11, 2017 2:22 PM
To: outages at outages.org<mailto:outages at outages.org>
Subject: Re: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too?
We have been seeing more issues in the last few weeks with our Comcast Business customers also. Lots of offline alerts at night. Most in the same geographical area and it seems like they are doing different areas each week.
No evidence though from them. When we call them there like what are you talking about?
David Wagner | Director of Technology
Xpert Technologies, Inc.
Office | 248.524.0101 x310
Support | 248.524.0101 Opt 2
Support | support at xxpert.com<mailto:support at xxpert.com>
Web | xxpert.com <https://linkprotect.cudasvc.com/url?a=http://www.xxpert.com/&c=E,1,o_Nqnliwg1g46dbFIlq64oO9mXdW-STJuk-2S3hR8ZsS_ylXZFEWlsHLy8YQaWmBxjpTJfZ4Vic-FcZg7vG3ddGJPNNeg9c1cS23SYNbfcU,&typo=1>
[X_logo_tag (3)]
From: Outages [mailto:outages-bounces at outages.org] On Behalf Of B.J. Ash via Outages
Sent: Thursday, May 11, 2017 10:07 AM
To: Outages Mailing List <outages at outages.org<mailto:outages at outages.org>>
Subject: [outages] Comcast/Level 3 Issues - Detroit/Chicago Areas -- Now CenturyLink today too?
Does anyone have any information on some ongoing issues with Comcast in the Michigan area since end of April that involve intermittent packet loss, latency spikes (from normal 20 ms to 500+ ms), and just overall cruddy paths?
We're seeing issues with 3 of our sites in West/Central Michigan that use Comcast Business for Internet as they route from Comcast to Level 3. Traceroutes seem to die somewhere in Chicago or leaving Detroit either in Level 3 or at the edge of Comcast:
Tracing the route to 172.87.34.1
VRF info: (vrf in name/id, vrf out name/id)
1 50-77-204-206-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://50-77-204-206-static.hfc.comcastbusiness.net&c=E,1,MesRXXX-ZpHhLB33CeAxAXaukAm9dMf3Fh0yMfozobuQGQxj71pIOS9sqQQhWWdrwMNdiWO1PJYRrJO1hmyPQlWjZbF1vS0HE5gDskwxNyvFlmTV&typo=1> (50.77.204.206) 52 msec 40 msec 32 msec
2 96.120.42.37 32 msec 44 msec 40 msec
3 xe-7-0-6-sur02.muskegon.mi.michigan.comcast.net (68.85.189.25) 52 msec 52 msec 56 msec
4 xe-1-3-1-0-sur01.lyndon.mi.michigan.comcast.net (68.87.189.177) 60 msec 68 msec 64 msec
5 ae2.bar1.detroit1.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae2.bar1.detroit1.level3.net&c=E,1,VX1Wtis8mOU_HmkSZzDyqRAzH8VxM31xi8BamAQacC92kIgCHjrUZEqIpVd2L20v8HRTRJ4FQDgFH1AEXGVN3um02afezL5hpr3HkDPQU7e1Ft--&typo=1> (4.68.71.33) 72 msec 64 msec 64 msec
6 * * *
7 lynx-networ.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://lynx-networ.ear3.chicago2.level3.net&c=E,1,mTHzCTVbGRpda9786-dOlc-yLOkIB66HjJCxrLXuEQEdu2xhWIuXr6_SvQCB4kFn1GiRVSEifzVjZsKgz7Xy7RdIeSc6qS_pN_Oi8yk,&typo=1> (4.35.108.250) 136 msec 144 msec 152 msec
8 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,p32JgU9qvfSYIqYng6cxDawsSW4AhPwqOOoN6BSHDrYOQgmtkO34kJVoBBdPP7DCxCyUs-coNMm70xgBJg64M6MJK1ZsgA3Njpkmsj_3V4bzAvIB5gHfEbQ,&typo=1> (172.87.34.1) 148 msec * 220 msec
Tracing the route to 172.87.34.1
VRF info: (vrf in name/id, vrf out name/id)
1 96-95-79-62-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://96-95-79-62-static.hfc.comcastbusiness.net&c=E,1,sNqVce2uAPhqogMSJ6ICjEZ6oKeaT4J5vdHj7qQAhqiwuH6y8-hjv_LWM1O9zW4ueqVKkOIs8FtVEky5IOTiu0A0aCG86dQWZoRZKTYP6LqIEiG-r968jDU,&typo=1> (96.95.79.62) 0 msec 4 msec 0 msec
2 96.120.28.97 8 msec 8 msec 8 msec
3 be92-sur02.stjoseph.mi.sbend.comcast.net (68.85.204.29) 8 msec 12 msec 12 msec
4 68.85.183.237 12 msec 8 msec 8 msec
5 be-126-ar01.area4.il.chicago.comcast.net (68.87.230.249) 12 msec 16 msec 16 msec
6 * * *
7 * * *
8 LYNX-NETWOR.ear3.Chicago2.Level3.net<https://linkprotect.cudasvc.com/url?a=https://LYNX-NETWOR.ear3.Chicago2.Level3.net&c=E,1,A9YvfSwg2cc4y0prRHCtTjFkNpW74klxPfBui53WumpjBfHJPRyCj3BJFpyYkNhXPH5QvewGFBrsfhMvJXjiL5IwzP3Ezds5BUuaug,,&typo=1> (4.35.108.250) 20 msec 16 msec 16 msec
9 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,kXLeVSEgqyqPak2JP8c8bh3tml2ISK-ir5XBPpebVWMaxSSbAGbxEBhR-9kiXnIVq7Kv0KVdk8VpF3o30p3Cn4yScA2NqVelosXMU4Y,&typo=1> (172.87.35.66) 16 msec 28 msec 20 msec
10 * * *
11 * * *
Tracing the route to 172.87.34.1
VRF info: (vrf in name/id, vrf out name/id)
1 173-15-238-74-static.hfc.comcastbusiness.net<https://linkprotect.cudasvc.com/url?a=https://173-15-238-74-static.hfc.comcastbusiness.net&c=E,1,O_E4PxXmKYuMcAhMXD6gjUWOFtgCNWVrSL-umAYMAurWyzGEdh30knO6P_wUXHaTZj1_x0r-9eaTjlrtbZiHUw30WoVgpFoQp3Fcp1eV95FGt6lcZHlgyJHX&typo=1> (173.15.238.74) 0 msec 4 msec 4 msec
2 96.120.42.165 16 msec 16 msec 16 msec
3 68.85.49.133 16 msec 12 msec 12 msec
4 xe-0-0-0-0-sur01.stclairshors.mi.michigan.comcast.net (68.87.189.38) 16 msec 12 msec 12 msec
5 te-0-3-0-15-ar02.pontiac.mi.michigan.comcast.net (68.85.235.249) 20 msec 20 msec 16 msec
6 ae2.bar1.detroit1.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae2.bar1.detroit1.level3.net&c=E,1,C1WUZjUdFRQMEUCj_27wiWn8zTmrQDk_Iq60VqJSR5AqCFCRkfCCgEEHZ5TYYyBOlxAkjSrLMZgL_a0PNhiP7KMAL_BNin_r49EaA9OW90Vr&typo=1> (4.68.71.33) 44 msec 20 msec 16 msec
7 * * *
8 lynx-networ.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://lynx-networ.ear3.chicago2.level3.net&c=E,1,lqBRUMWD4B3q8U8gCPCJQhLWJbT6xXYNMp1JYoaffXZgtHcoyoGyf0epxcudVIcdv6aapj8mrVyRr2W1jJ2yxlW7A-uIgOYWdYWiHLxxGprwRTXFSE7GGA,,&typo=1> (4.35.108.250) 24 msec 32 msec 28 msec
9 66.35.87.172.in-addr.arpa.lynxnetworkgroup.com<https://linkprotect.cudasvc.com/url?a=https://66.35.87.172.in-addr.arpa.lynxnetworkgroup.com&c=E,1,hmmq0QFsFnVzOxps-7iTgl9pVzFXuFsNpp4zqxoEO7wn1oUWmQJzYR7WRPBOXusFsVW2YBJAuyHg8KnGU-w318ioyuXR8a7WB-UqicpeR7p_WBFq5aZ_xpE,&typo=1> (172.87.34.1) 32 msec * 32 msec
Now, this morning we're seeing an inability to pass two-way traffic over CenturyLink through Chicago:
1 23ms 23ms 24ms 63.149.1.22
2 * * *
3 23ms 23ms 23ms 63.146.27.18
4 * * *
5 28ms 29ms 29ms 4.35.108.250
6 35ms 28ms 29ms 172.87.34.1
Tracing the route to 63.149.1.22
VRF info: (vrf in name/id, vrf out name/id)
1 172.87.35.66 0 msec 0 msec 0 msec
2 6-2-40.ear3.chicago2.level3.net<https://linkprotect.cudasvc.com/url?a=https://6-2-40.ear3.chicago2.level3.net&c=E,1,bQwKL6_R3ZpzT-p5vodvIDshnb4XzpklrQ7IiX-fmze32vrAHYOkbgg_0ELRriLm13c62OP67INYb1H_4Hu-q2o6ZHUUh54IgvFbHEMpL1k,&typo=1> (4.35.108.249) 34 msec 0 msec 0 msec
3 ae-14-3604.edge4.chicago3.level3.net<https://linkprotect.cudasvc.com/url?a=https://ae-14-3604.edge4.chicago3.level3.net&c=E,1,DJtHkO_-u9lL4VH9EiXgJk2mR-OU-2sAxZ_hjy5LAVBAlQ83gahXNjGR5jsLAjdCG9XffqbSvv4DURg83jKb_f2s-vH67btLLI5YvNfwAMUcYRs1zIk1lFU2&typo=1> (4.69.204.166) 8 msec 9 msec 8 msec
4 chp-brdr-03.inet.qwest.net<https://linkprotect.cudasvc.com/url?a=https://chp-brdr-03.inet.qwest.net&c=E,1,1U4xzmJx3Dnr_UZ7SFMLrpNcjToJu0EbPy4QINiRxdVEYNby6-cqx8lYm0c0dgtWt1aLPQLNuWFtCcD5p4qn0iw2534t314Au8CNWTMuhJ2S&typo=1> (63.146.27.17) 9 msec 8 msec 8 msec
5 cer-edge-15.inet.qwest.net<https://linkprotect.cudasvc.com/url?a=https://cer-edge-15.inet.qwest.net&c=E,1,RCbzcVO0dNWlB3zVOEg357IMxGJyB0ZtY8gVoH3_5J1POuIE54yF2qEE3bkmW_SSCihpHQljVsxfr1yveXKR38vuqefruWFYb5l3JP0ZyaFFNpY-xhE,&typo=1> (205.171.93.26) 9 msec 8 msec 9 msec
6 63.149.1.22 33 msec 34 msec *
Comcast appears blind to the issues we've been seeing at their edge. Seems more like a Level 3 issue. What's going on in Chicago lately?
--
B.J. Ash
bjash at kentrecords.com<mailto:bjash at kentrecords.com>
IT Supervisor | Kent Record Management Inc.
1950 Waldorf St. NW, Ste. A | Grand Rapids, MI 49544
W:616-459-6681 | www.kentrecords.com<https://linkprotect.cudasvc.com/url?a=http://www.kentrecords.com/&c=E,1,i-ve63w-IWSVEG5iYFh3J5uOmr1LVSJi2Lz60l-0iRyE-S5x5Db_2xTjfIHFk1mDILTwqJosmdo7IW1Z1794xwzxy1OIXVO312dNPVaUVh22NSmN8MY,&typo=1>
Disclaimer: This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Kent Record Management Inc. If you are not the intended recipient of this email and its attachments, you must take no action based upon them, nor must you copy or show them to anyone. Please contact the sender if you believe you have received this email in error.
________________________________
NOTICE: This email may contain PRIVILEGED and CONFIDENTIAL information and is intended only for the use of the specific individual(s) to which it is addressed. If you are not the intended recipient of this email, you are hereby notified that any unauthorized use, dissemination or copying of this email or the information contained in it or attached to it is strictly prohibited. You may be subject to penalties under law for any improper use or further disclosure of any Protected Health Information in this email. If you have received this email in error, please delete it and immediately notify the sender of this email by reply mail. Thank you. [SHG-PHI-TXT]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20170512/e6c10132/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2614 bytes
Desc: image001.jpg
URL: <https://puck.nether.net/pipermail/outages/attachments/20170512/e6c10132/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 3689 bytes
Desc: image002.png
URL: <https://puck.nether.net/pipermail/outages/attachments/20170512/e6c10132/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 14979 bytes
Desc: image003.png
URL: <https://puck.nether.net/pipermail/outages/attachments/20170512/e6c10132/attachment-0001.png>
More information about the Outages
mailing list