[outages] Level3 high latency and some packet loss Columbia SC

[Berry Mobley]

We are back up as of about 30 minutes ago, here's the last we got from 
Level3:


*** CASCADED EXTERNAL NOTES 13-Sep-2017 20:26:29 GMT From CASE: 13369921 
- Event Outage Start: September 13, 2017 14:28 GMT Outage Stop: 
September 13, 2017 18:15 GMT Root Cause: A link was down impacting IP 
services for multiple markets in North Carolina.

Fix Action: Services were re-routed, thus restoring services to a stable 
state.

Summary: The IP NOC reported a link was down impacting multiple markets 
for North Carolina. Services began to re-route, however, due to an 
overutilization issue a delay was experienced. It was determined the 
service disruption remained ongoing. The IP NOC began attempting to 
shift traffic to resolve the overutilization issue, however many clients 
continued to experience packet loss and latency. The IP NOC completed 
migrating the affected service to adjacent paths and service restored. 
This service impact has concluded; if additional issues are experienced, 
please contact the Level 3 Technical Service Center. There may be 
additional analysis and discovery that occurs as the incident is 
reviewed by NOC management. Any available updates will be relayed upon 
event ticket closure. At that time, a customer satisfaction survey link 
may be available. We strive to provide thorough communications 
containing the available information during a service disruption. Please 
let us know if the updates you received during this event were satisfactory.



On 9/13/2017 3:48 PM, James Cornman via Outages wrote:
> Are there still reports of these issues with Level3? Seeing similar 
> with RR.com which is behind Level3. Any word on the scope of this 
> issue and if its still ongoing?
>
> -James
>
> On Wed, Sep 13, 2017 at 1:44 PM, Mattson, Martin A (Alan) via Outages 
> <outages at outages.org <mailto:outages at outages.org>> wrote:
>
>     We saw another drop of connectivity for a couple minutes around
>     11am today here in Raleigh on our L3 Internet circuit and latency
>     went from 20ms to about 200ms between our NC and PA sites.
>
>     *From:* Outages [mailto:outages-bounces at outages.org
>     <mailto:outages-bounces at outages.org>] *On Behalf Of *Aubrey Wells
>     via Outages
>     *Sent:* Wednesday, September 13, 2017 1:27 PM
>     *To:* Berry Mobley <berry at gadsdenst.org <mailto:berry at gadsdenst.org>>
>     *Cc:* outages at outages.org <mailto:outages at outages.org>
>     *Subject:* Re: [outages] Level3 high latency and some packet loss
>     Columbia SC
>
>     There was a planned maintenance for a router upgrade last night
>     that probably explains your circuit drop. Not seeing any latency
>     on our circuit after the maintenance though.
>
>
>     -----------------------------------------------------------------------
>
>     Aubrey Wells
>
>     Packethead Consulting, LLC
>
>     Helping you get your bits together since 0x7DE
>
>     On Wed, Sep 13, 2017 at 1:24 PM, Berry Mobley via Outages
>     <outages at outages.org <mailto:outages at outages.org>> wrote:
>
>         Good afternoon -
>
>         We've been experiencing high latency (~150ms) and around 8%
>         packet loss to our L3 peer in Atlanta. We noticed the circuit
>         drop overnight for an hour or so, then mid-morning things
>         started to go downhill. We've turned down this peering
>         session, and are opening a ticket, but is anybody else seeing
>         anything weird?
>
>         Thanks,
>
>         Berry
>
>         _______________________________________________
>         Outages mailing list
>         Outages at outages.org <mailto:Outages at outages.org>
>         https://puck.nether.net/mailman/listinfo/outages
>         <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpuck.nether.net%2Fmailman%2Flistinfo%2Foutages&data=02%7C01%7CMartin_mattson%40lord.com%7C9abe2544c9b641976cea08d4faccd9ab%7C239fbe1eac6d458cbe3c6d33d129b080%7C0%7C0%7C636409205110633606&sdata=q7Cdfi8OEJezeA4%2FdFNfzAterLjIovQYfpqCwaxUqNE%3D&reserved=0>
>
>
>     _______________________________________________
>     Outages mailing list
>     Outages at outages.org <mailto:Outages at outages.org>
>     https://puck.nether.net/mailman/listinfo/outages
>     <https://puck.nether.net/mailman/listinfo/outages>
>
>
>
>
> -- 
>
> *James Cornman*
>
> /Chief Technology Officer
> /jcornman at atlanticmetro.net <mailto:jcornman at atlanticmetro.net>
> 212.792.9950 - ext 101
> *
> Atlantic Metro Communications*
>
> /4 Century Drive, Parsippany NJ  07054/
>
> /
> //Cloud Hosting //• //Colocation • Network Connectivity • Managed 
> Services/
>
> Follow us on Twitter: @atlanticmetro 
> <https://twitter.com/atlanticmetro> /• Like us on Facebook 
> <https://www.facebook.com/atlanticmetro>/
> www.atlanticmetro.net <https://www.atlanticmetro.net/>
>
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20170913/774e8435/attachment.htm>