[outages] Level3/TW Telecom issues?

[Jason Hellenthal]

Rofl

-- 
 Jason Hellenthal
 JJH48-ARIN

On Aug 27, 2015, at 20:08, Ryan Brooks via Outages <outages at outages.org> wrote:

Yesterday, fiber cut.  Today, "Internet Protocol" problems.  Tonight, capacity management?

> On Aug 27, 2015, at 8:01 PM, Ryan Gelobter via Outages <outages at outages.org> wrote:
> 
> 08/27/2015 11:16 PM GMT
> Multiple cards have been ordered, and will be installed on equipment in Chicago, IL in an effort to balance the traffic load through the site and alleviate the issue. Field Services are currently en route to the site with the cards, and are expected to arrive shortly.
> 
> On Thu, Aug 27, 2015 at 5:02 PM, Ryan Gelobter <ryan.g at atwgpc.net> wrote:
>> Latest update.....
>> 08/27/2015 9:49 PM GMT
>> Tier III Technical Support and Operations Engineering teams continue investigating the issue identified with the Internet Protocols. Restoration efforts are ongoing; however, there is currently no estimated restoral time.
>> 
>> 
>>> On Thu, Aug 27, 2015 at 3:14 PM, Ryan Brooks <rbrooks at stack41.com> wrote:
>>> ICMP blazes through at times, but TCP completely falls apart.
>>> 
>>>> On Aug 27, 2015, at 3:13 PM, Ryan Gelobter via Outages <outages at outages.org> wrote:
>>>> 
>>>> 08/27/2015 8:05 PM GMT
>>>> Level 3 Tier III and Operations Engineering teams have identified Internet Protocols dropping, affecting customer services.  Restoration efforts are in progress, but an estimated time of restoral is not available at this time.
>>>> 
>>>>> On Thu, Aug 27, 2015 at 3:06 PM, Ryan Gelobter <ryan.g at atwgpc.net> wrote:
>>>>> If you have access to the Level3 portal you should see ticket #9639047 under Network Events now.
>>>>> 
>>>>> 
>>>>> Event Summary
>>>>> IP Network Event ~ Multiple Markets
>>>>> 
>>>>> 08/27/2015 6:36 PM GMT
>>>>> IP and Transport Tier III, Operations Engineering and Field Services continue collaboratively working the issue.
>>>>> 
>>>>> 08/27/2015 4:59 PM GMT
>>>>> Operations Engineering is engaged and Field Services is on site in Chicago, IL investigating the issue.
>>>>> 
>>>>> 08/27/2015 4:38 PM GMT
>>>>> The engineers are currently migrating traffic in efforts of restoring services while troubleshooting continues. Field Services is being dispatched to a Chicago, IL site to assist.
>>>>> 
>>>>> 08/27/2015 4:21 PM GMT
>>>>> IP services are affected across multiple markets and the root cause is currently under investigation. The IP NOC and IP and Transport Tier III are actively troubleshooting and working to isolate the cause. The engineers have detected peering issues which are resulting in packet loss for customers. Please be advised that updates will be provided at minimum of hourly unless otherwise noted.
>>>>> 
>>>>> 
>>>>>> On Thu, Aug 27, 2015 at 2:59 PM, Eric Henson via Outages <outages at outages.org> wrote:
>>>>>> We had to shut down our Level3 circuit in the DFW area, had issues from 1406CST until we shut down our circuit.
>>>>>> 
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Outages [mailto:outages-bounces at outages.org] On Behalf Of jungle Boogie via Outages
>>>>>> Sent: Thursday, August 27, 2015 2:56 PM
>>>>>> To: Bobin Joseph
>>>>>> Cc: outages at outages.org
>>>>>> Subject: Re: [outages] Level3/TW Telecom issues?
>>>>>> 
>>>>>> On 27 August 2015 at 12:48, Bobin Joseph via Outages <outages at outages.org> wrote:
>>>>>> > Anybody else having complete outage on Level3? We lost complete
>>>>>> > internet in DFW and SAN
>>>>>> >
>>>>>> 
>>>>>> We were not down but significant routing issues in Las Vegas.
>>>>>> 
>>>>>> > Thanks,
>>>>>> > Bobin
>>>>>> >
>>>>>> _______________________________________________
>>>>>> Outages mailing list
>>>>>> Outages at outages.org
>>>>>> https://puck.nether.net/mailman/listinfo/outages
>>>>>> _______________________________________________
>>>>>> Outages mailing list
>>>>>> Outages at outages.org
>>>>>> https://puck.nether.net/mailman/listinfo/outages
>>>> 
>>>> _______________________________________________
>>>> Outages mailing list
>>>> Outages at outages.org
>>>> https://puck.nether.net/mailman/listinfo/outages
> 
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20150827/f832af29/attachment.htm>