[outages] Level3/TW Telecom issues?

Steven Fischer sfischer1967 at gmail.com
Thu Aug 27 15:01:57 EDT 2015 

just got off the phone with our Level3 Service Manager - he indicated all
should be well.  Watching and Waiting...

On Thu, Aug 27, 2015 at 2:33 PM, Jameel Akari via Outages <
outages at outages.org> wrote:

> On Thu, 27 Aug 2015, Matthew Topper via Outages wrote:
>
> Seeing on and off in upstate NY
>>
>>
>> Matthew Topper
>>
>
> Confirmed upstate NY (Albany) as well.  We've rerouted in the meanwhile;
> were seeing lost routes and split routing back to us.
>
> --
> Jameel Akari
>
>
>
>
>> -----Original Message-----
>> From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Jason
>> Hellenthal via Outages
>> Sent: Thursday, August 27, 2015 2:23 PM
>> To: outages at outages.org
>> Subject: Re: [outages] Level3/TW Telecom issues?
>>
>> Retracting my statement from yesterday this is still a solid issue here.
>> Yes absolutely still continuing. Root Cause would be a great explanation at
>> this point.
>>
>> On Aug 27, 2015, at 13:10, Ryan Brooks via Outages <outages at outages.org>
>>> wrote:
>>>
>>> 24 hours of on and off packet loss, more than just Chicago.   More
>>> communication would be helpful.   NOC can't seem to decide if it's
>>> transport or routing.  Any insights?
>>>
>>> On Aug 27, 2015, at 1:08 PM, Bobin Joseph via Outages <
>>>> outages at outages.org> wrote:
>>>>
>>>> We just got another update saying they still have issues.
>>>>
>>>>
>>>> IP services are affected across multiple markets. The IP NOC and IP and
>>>> Transport Tier III are actively troubleshooting the issue. Operations
>>>> Engineering is engaged and Field Services is on site in Chicago, IL
>>>> investigating the issue.
>>>> tw telecom has identified a network failure affecting your services. tw
>>>> telecom tier-2 group is investigating the issue at this time.
>>>>
>>>> Thanks,
>>>> Bobin
>>>>
>>>> From: Outages [mailto:outages-bounces at outages.org] On Behalf Of Aaron
>>>> Gross via Outages
>>>> Sent: Thursday, August 27, 2015 10:55 AM
>>>> To: Bob Hancock
>>>> Cc: outages at outages.org
>>>> Subject: Re: [outages] Level3/TW Telecom issues?
>>>>
>>>> We are seeing the same things that happened yesterday on our circuits
>>>> in Milwaukee and Chicago areas. Lots of packet loss and large latency
>>>> through Chicago exchanges.
>>>>
>>>> On Thu, Aug 27, 2015 at 12:33 PM, Bob Hancock via Outages <
>>>> outages at outages.org> wrote:
>>>> We were seeing substantial loss through Chicago this morning but it
>>>> seems to have subsided at this time.
>>>>
>>>>
>>>> Bob Hancock
>>>>
>>>>
>>>> On 08/27/2015 12:19 PM, Steven Fischer via Outages wrote:
>>>>
>>>>> we are still seeing significant packet loss to 3 of our sites in
>>>>> the Midwest.
>>>>>
>>>>> On Thu, Aug 27, 2015 at 12:47 PM, Bobin Joseph via Outages
>>>>> <outages at outages.org <mailto:outages at outages.org>> wrote:
>>>>>
>>>>>     We got this from Level3____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     Hello,
>>>>>
>>>>>     The network outage has been resolved.
>>>>>
>>>>>     *Event Conclusion Summary*
>>>>>
>>>>>     Start: August 26, 2015 19:15 GMT
>>>>>     Stop: August 27, 2015 00:14 GMT
>>>>>
>>>>>     Root Cause: An unknown issue in Chicago, IL was impacting IP
>>>>> services.
>>>>>     Fix Action: Adjustments were made to resolve the issue and restore
>>>>>     services.
>>>>>
>>>>>     Summary:
>>>>>     The IP NOC responded to an unknown issue in Chicago, IL that was
>>>>>     impacting IP services. Tier III Technical Support were engaged to
>>>>>     assist with investigations, and adjustments were made to resolve
>>>>> the
>>>>>     issue and stabilize traffic. Monitoring continues to ensure
>>>>>     stability. Should further issues be experienced, please contact the
>>>>>     Level 3 Technical Service Center.
>>>>>
>>>>>
>>>>>     ____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     Thanks,____
>>>>>
>>>>>     Bobin____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     *From:*Outages [mailto:outages-bounces at outages.org
>>>>>     <mailto:outages-bounces at outages.org>] *On Behalf Of *Ryan Gelobter
>>>>>     via Outages
>>>>>     *Sent:* Thursday, August 27, 2015 9:44 AM
>>>>>     *To:* Rafael Possamai
>>>>>     *Cc:* outages at outages.org <mailto:outages at outages.org>
>>>>>     *Subject:* Re: [outages] Level3/TW Telecom issues?____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     Was anyone able to get some useful information out of Level3
>>>>>     regarding this yesterday? Also is anyone else seeing similar
>>>>> issues?____
>>>>>
>>>>>     On Aug 26, 2015 2:40 PM, "Rafael Possamai via Outages"
>>>>>     <outages at outages.org <mailto:outages at outages.org>> wrote:____
>>>>>
>>>>>     Here's a graph showing some packet loss: http://imgur.com/KVAzetA
>>>>>
>>>>> <http://cp.mcafee.com/d/FZsSd38w83hJ5xyXxEVsKCMrKrhKCy-qen4TXCQrFEL
>>>>> CzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LO
>>>>> VJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjr1J7HF
>>>>> kr8kJ6h3PtPo0eiF5KfbU1gp41jVy1KHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQz
>>>>> h0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrehG_98MC>____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     __ __
>>>>>
>>>>>     On Wed, Aug 26, 2015 at 3:16 PM, Matt Elliott via Outages
>>>>>     <outages at outages.org <mailto:outages at outages.org>> wrote:____
>>>>>
>>>>>     Anyone else having issues on Level3?  We’re seeing packet loss with
>>>>>     anything that traverses Chicago.____
>>>>>
>>>>>     __ __
>>>>>
>>>>>     Thanks,____
>>>>>
>>>>>     Matt____
>>>>>
>>>>>     This message and attachment(s), if any, is intended for the sole
>>>>> use
>>>>>     of the individual and/or entity for which it is addressed, and may
>>>>>     contain information that is privileged, confidential and prohibited
>>>>>     from disclosure under applicable law. If you are not the addressee,
>>>>>     or authorized to receive this on behalf of the addressee, you are
>>>>>     hereby notified that you may not use, copy, disclose or distribute
>>>>>     to anyone this message or any part thereof. If you have received
>>>>>     this in error, please immediately advise the sender by e-mail and
>>>>>     delete this information and all attachments from your computer and
>>>>>     network. Thank you. ____
>>>>>
>>>>>
>>>>>     _______________________________________________
>>>>>     Outages mailing list
>>>>>     Outages at outages.org <mailto:Outages at outages.org>
>>>>>     https://puck.nether.net/mailman/listinfo/outages
>>>>>
>>>>> <http://cp.mcafee.com/d/avndy0QrhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVs
>>>>> ppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrug
>>>>> sotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJcSMrhWWl6O5
>>>>> bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_j
>>>>> w09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdBPdH>
>>>>> ____
>>>>>
>>>>>     __ __
>>>>>
>>>>>
>>>>>     _______________________________________________
>>>>>     Outages mailing list
>>>>>     Outages at outages.org <mailto:Outages at outages.org>
>>>>>     https://puck.nether.net/mailman/listinfo/outages
>>>>>
>>>>> <http://cp.mcafee.com/d/2DRPoQ938ArhooKUqenbFI6XCQrFELCzBNd-VJ6WqbV
>>>>> EVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYK
>>>>> rugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJASMrhWWl
>>>>> 6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ
>>>>> 8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdWo
>>>>> Uq>____
>>>>>
>>>>>
>>>>>     _______________________________________________
>>>>>     Outages mailing list
>>>>>     Outages at outages.org <mailto:Outages at outages.org>
>>>>>     https://puck.nether.net/mailman/listinfo/outages
>>>>>
>>>>> <http://cp.mcafee.com/d/k-Kr41ESyMNtMQsKnjodTdETjhvd7byrZPqdQQnPhOU
>>>>> OOYrhKCy-qen4TQjqdQTbLfzDSjp0DIblIl1iJlvxdJmvD-LOVJmGLMCSHfP_nVsSYw
>>>>> UMWw-_R-uoovu7ctuVt5dMQsYyUeod7bdTeEyCJtdmXX_axVZicHs3jr1JwSzRQGdAa
>>>>> mz8xVKVIDeqR4INpKNnwqj-f0T1dnoovaAVgtHBmVWh-xKHsZ8_gKgGT2TQ1lmVWh-D
>>>>> 00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrVbh4>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> To him who is able to keep you from falling and to present you
>>>>> before his glorious presence without fault and with great joy
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Outages mailing list
>>>>> Outages at outages.org
>>>>> http://cp.mcafee.com/d/avndzhJ5xyXxEVsKC-rKrhKCy-qen4TXCQrFELCzBNBB
>>>>> USztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1Nx
>>>>> R1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjq9J7HFkr8kJ6
>>>>> h3PtPpesRG9pyPtyL0QDYu1K2qKMM-l9OwXnaJPQzZ3tmVWh-xsxlK5LE2GJPQzZe00
>>>>> CQm4PhOUr1vF6y0QJRrDF7W6BQQg1nPfDIU2g5d409Y_RPh1aB6_EidICzASH16Ohmp
>>>>>
>>>>> The information contained in this E-mail message is privileged,
>>>> confidential, and may be protected from disclosure; please be aware that
>>>> any unauthorized use, printing, copying, disclosure or dissemination of
>>>> this communication may be subject to legal restriction or sanction. If you
>>>> think that you have received this E-mail message in error, please reply to
>>>> the sender. This E-mail message and any attachments have been scanned for
>>>> viruses and are believed to be free of any virus or other defect that might
>>>> affect any computer system into which it is received and opened. It is the
>>>> responsibility, however, of the recipient to ensure that it is virus free.
>>>> _______________________________________________
>>>> Outages mailing list
>>>> Outages at outages.org
>>>> https://puck.nether.net/mailman/listinfo/outages
>>>>
>>>> _______________________________________________
>>>> Outages mailing list
>>>> Outages at outages.org
>>>> https://puck.nether.net/mailman/listinfo/outages
>>>>
>>>
>>> _______________________________________________
>>> Outages mailing list
>>> Outages at outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>>
>>
>>
>> --
>> Jason Hellenthal
>> JJH48-ARIN
>>
>>
>>
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>>
> --
> Jameel Akari
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
>


-- 
To him who is able to keep you from falling and to present you before his
glorious presence without fault and with great joy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20150827/d7d842a2/attachment.htm>

More information about the Outages mailing list