[outages] CenturyLink outage?

[Hugh Smallwood]

Below is what we got.
Last night from around 11:30pm until 2:30am<x-apple-data-detectors://0> there was a widespread outage on the MPLS backbone that impacted customers coast to coast. Tier 3 rebooted 125+ routers throughout the network to restore service, though a handful of customers may have taken slightly longer. The issue has been escalated  and engineering is continuing to work to find out what caused the problem. As of now the network is stable, unfortunately I don't have much more info at this time to share.



From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Bryan Riley
Sent: Friday, January 24, 2014 10:37 AM
To: Outages at outages.org
Subject: Re: [outages] CenturyLink outage?

Anyone see an RFO for this yet?


[OCTI-Logo.gif]
Bryan Riley
Operations Manager
757.802.4991 (Anywhere)
757.351.4115 (fax)
757.802.4955 (main)
www.oncalltelecom.com<http://www.oncalltelecom.com>

From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Karl Putland
Sent: Friday, January 24, 2014 12:58 AM
To: Outages at outages.org<mailto:Outages at outages.org>
Subject: Re: [outages] CenturyLink outage?

We were seeing DIA part of Enhanced ports come back up, but we weren't getting routes from the CUGs/VRFs. Looks like BGP session had gone stale. Kicked them all and we're back.

--Karl


Karl Putland
Senior Engineer
SimpleSignal
Anywhere: 303-242-8608<tel:303-242-8608>
[Image removed by sender.]<http://www.simplesignal.com/explainer_video.php>
[Image removed by sender.]


On Thu, Jan 23, 2014 at 10:46 PM, Steve Dispensa <Steve.Dispensa at microsoft.com<mailto:Steve.Dispensa at microsoft.com>> wrote:
Ditto, recovery in one location at 9:25 PM PT. Still down hard in a few others.

From: Outages [mailto:outages-bounces at outages.org<mailto:outages-bounces at outages.org>] On Behalf Of Mitch Patterson
Sent: Thursday, January 23, 2014 9:40 PM
To: SML
Cc: Outages at outages.org<mailto:Outages at outages.org>

Subject: Re: [outages] CenturyLink outage?

I just had one of 2 of my circuits pop back up.

On Fri, Jan 24, 2014 at 12:38 AM, SML <sml at lordsargon.com<mailto:sml at lordsargon.com>> wrote:
On 23 Jan 2014, at 23:34, Karl Putland wrote:

> Anyone seen ETR yet?

My circuits are coming up.


_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages


_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20140124/a4b25bfa/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2000 bytes
Desc: image001.jpg
URL: <https://puck.nether.net/pipermail/outages/attachments/20140124/a4b25bfa/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 823 bytes
Desc: image002.jpg
URL: <https://puck.nether.net/pipermail/outages/attachments/20140124/a4b25bfa/attachment-0001.jpg>