<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">24 hours of on and off packet loss, more than just Chicago. More communication would be helpful. NOC can't seem to decide if it's transport or routing. Any insights?<div class=""><br class=""><div class=""><div><blockquote type="cite" class=""><div class="">On Aug 27, 2015, at 1:08 PM, Bobin Joseph via Outages <<a href="mailto:outages@outages.org" class="">outages@outages.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-family: Arial, sans-serif;" class="">We just got another update saying they still have issues.<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-family: Arial, sans-serif;" class=""> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-family: Arial, sans-serif;" class=""> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-family: Arial, sans-serif;" class="">IP services are affected across multiple markets. The IP NOC and IP and Transport Tier III are actively troubleshooting the issue. Operations Engineering is engaged and Field Services is on site in Chicago, IL investigating the issue.<span class="Apple-converted-space"> </span><br class="">tw telecom has identified a network failure affecting your services. tw telecom tier-2 group is investigating the issue at this time.</span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Thanks,<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class="">Bobin<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);" class=""> </span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Tahoma, sans-serif;" class="">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;" class=""><span class="Apple-converted-space"> </span>Outages [<a href="mailto:outages-bounces@outages.org" class="">mailto:outages-bounces@outages.org</a>]<span class="Apple-converted-space"> </span><b class="">On Behalf Of<span class="Apple-converted-space"> </span></b>Aaron Gross via Outages<br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Thursday, August 27, 2015 10:55 AM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Bob Hancock<br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:outages@outages.org" class="">outages@outages.org</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [outages] Level3/TW Telecom issues?<o:p class=""></o:p></span></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">We are seeing the same things that happened yesterday on our circuits in Milwaukee and Chicago areas. Lots of packet loss and large latency through Chicago exchanges.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">On Thu, Aug 27, 2015 at 12:33 PM, Bob Hancock via Outages <<a href="mailto:outages@outages.org" target="_blank" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a>> wrote:<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">We were seeing substantial loss through Chicago this morning but it<br class="">seems to have subsided at this time.<br class=""><br class=""><br class="">Bob Hancock<br class=""><br class=""><br class="">On 08/27/2015 12:19 PM, Steven Fischer via Outages wrote:<br class="">> we are still seeing significant packet loss to 3 of our sites in the<br class="">> Midwest.<br class="">><br class="">> On Thu, Aug 27, 2015 at 12:47 PM, Bobin Joseph via Outages<br class="">> <<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a>>> wrote:<br class="">><br class="">> We got this from Level3____<br class="">><br class="">> __ __<br class="">><br class="">> Hello,<br class="">><br class="">> The network outage has been resolved.<br class="">><br class="">> *Event Conclusion Summary*<br class="">><br class="">> Start: August 26, 2015 19:15 GMT<br class="">> Stop: August 27, 2015 00:14 GMT<br class="">><br class="">> Root Cause: An unknown issue in Chicago, IL was impacting IP services.<br class="">> Fix Action: Adjustments were made to resolve the issue and restore<br class="">> services.<br class="">><br class="">> Summary:<br class="">> The IP NOC responded to an unknown issue in Chicago, IL that was<br class="">> impacting IP services. Tier III Technical Support were engaged to<br class="">> assist with investigations, and adjustments were made to resolve the<br class="">> issue and stabilize traffic. Monitoring continues to ensure<br class="">> stability. Should further issues be experienced, please contact the<br class="">> Level 3 Technical Service Center.<br class="">><br class="">><br class="">> ____<br class="">><br class="">> __ __<br class="">><br class="">> Thanks,____<br class="">><br class="">> Bobin____<br class="">><br class="">> __ __<br class="">><br class="">> *From:*Outages [mailto:<a href="mailto:outages-bounces@outages.org" style="color: purple; text-decoration: underline;" class="">outages-bounces@outages.org</a><br class="">> <mailto:<a href="mailto:outages-bounces@outages.org" style="color: purple; text-decoration: underline;" class="">outages-bounces@outages.org</a>>] *On Behalf Of *Ryan Gelobter<br class="">> via Outages<br class="">> *Sent:* Thursday, August 27, 2015 9:44 AM<br class="">> *To:* Rafael Possamai<br class="">> *Cc:*<span class="Apple-converted-space"> </span><a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a>><br class="">> *Subject:* Re: [outages] Level3/TW Telecom issues?____<br class="">><br class="">> __ __<br class="">><br class="">> Was anyone able to get some useful information out of Level3<br class="">> regarding this yesterday? Also is anyone else seeing similar issues?____<br class="">><br class="">> On Aug 26, 2015 2:40 PM, "Rafael Possamai via Outages"<br class="">> <<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a>>> wrote:____<br class="">><br class="">> Here's a graph showing some packet loss:<span class="Apple-converted-space"> </span><a href="http://imgur.com/KVAzetA" target="_blank" style="color: purple; text-decoration: underline;" class="">http://imgur.com/KVAzetA</a><br class="">> <<a href="http://cp.mcafee.com/d/FZsSd38w83hJ5xyXxEVsKCMrKrhKCy-qen4TXCQrFELCzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjr1J7HFkr8kJ6h3PtPo0eiF5KfbU1gp41jVy1KHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrehG_98MC" target="_blank" style="color: purple; text-decoration: underline;" class="">http://cp.mcafee.com/d/FZsSd38w83hJ5xyXxEVsKCMrKrhKCy-qen4TXCQrFELCzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjr1J7HFkr8kJ6h3PtPo0eiF5KfbU1gp41jVy1KHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrehG_98MC</a>>____<br class="">><br class="">> __ __<br class="">><br class="">> __ __<br class="">><br class="">> On Wed, Aug 26, 2015 at 3:16 PM, Matt Elliott via Outages<br class="">> <<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:outages@outages.org" style="color: purple; text-decoration: underline;" class="">outages@outages.org</a>>> wrote:____<br class="">><br class="">> Anyone else having issues on Level3? We’re seeing packet loss with<br class="">> anything that traverses Chicago.____<br class="">><br class="">> __ __<br class="">><br class="">> Thanks,____<br class="">><br class="">> Matt____<br class="">><br class="">> This message and attachment(s), if any, is intended for the sole use<br class="">> of the individual and/or entity for which it is addressed, and may<br class="">> contain information that is privileged, confidential and prohibited<br class="">> from disclosure under applicable law. If you are not the addressee,<br class="">> or authorized to receive this on behalf of the addressee, you are<br class="">> hereby notified that you may not use, copy, disclose or distribute<br class="">> to anyone this message or any part thereof. If you have received<br class="">> this in error, please immediately advise the sender by e-mail and<br class="">> delete this information and all attachments from your computer and<br class="">> network. Thank you. ____<br class="">><br class="">><br class="">> _______________________________________________<br class="">> Outages mailing list<br class="">> <a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a>><br class="">> <a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/outages</a><br class="">> <<a href="http://cp.mcafee.com/d/avndy0QrhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJcSMrhWWl6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdBPdH" target="_blank" style="color: purple; text-decoration: underline;" class="">http://cp.mcafee.com/d/avndy0QrhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJcSMrhWWl6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdBPdH</a>>____<br class="">><br class="">> __ __<br class="">><br class="">><br class="">> _______________________________________________<br class="">> Outages mailing list<br class="">> <a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a>><br class="">> <a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/outages</a><br class="">> <<a href="http://cp.mcafee.com/d/2DRPoQ938ArhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJASMrhWWl6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdWoUq" target="_blank" style="color: purple; text-decoration: underline;" class="">http://cp.mcafee.com/d/2DRPoQ938ArhooKUqenbFI6XCQrFELCzBNd-VJ6WqbVEVsppudETjhvd7byrW9J6WrBTDNPX9IwjS5GSawFmGLMCSHfP_nVsSHlnUjrlDV_HYKrugsotgvvW_fccfL3CeLsKyCUqeuhs7c6zBCXDkhjmKCHtZ_BgY-F6lK1FJASMrhWWl6O5bhAgYTsSjDdqymoIToHMd9_7wrwCHIcfBisEeROHsZ8_gTlKuAvEn8lrxrW0GHsZ8_jw09J5xcQsK6MnWhEwdbtmVWh-xFtd40lYPVXe0A1jh02vfZsQgiFhLW4zr9EVdWoUq</a>>____<br class="">><br class="">><br class="">> _______________________________________________<br class="">> Outages mailing list<br class="">> <a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a><span class="Apple-converted-space"> </span><mailto:<a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a>><br class="">> <a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/outages</a><br class="">> <<a href="http://cp.mcafee.com/d/k-Kr41ESyMNtMQsKnjodTdETjhvd7byrZPqdQQnPhOUOOYrhKCy-qen4TQjqdQTbLfzDSjp0DIblIl1iJlvxdJmvD-LOVJmGLMCSHfP_nVsSYwUMWw-_R-uoovu7ctuVt5dMQsYyUeod7bdTeEyCJtdmXX_axVZicHs3jr1JwSzRQGdAamz8xVKVIDeqR4INpKNnwqj-f0T1dnoovaAVgtHBmVWh-xKHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrVbh4" target="_blank" style="color: purple; text-decoration: underline;" class="">http://cp.mcafee.com/d/k-Kr41ESyMNtMQsKnjodTdETjhvd7byrZPqdQQnPhOUOOYrhKCy-qen4TQjqdQTbLfzDSjp0DIblIl1iJlvxdJmvD-LOVJmGLMCSHfP_nVsSYwUMWw-_R-uoovu7ctuVt5dMQsYyUeod7bdTeEyCJtdmXX_axVZicHs3jr1JwSzRQGdAamz8xVKVIDeqR4INpKNnwqj-f0T1dnoovaAVgtHBmVWh-xKHsZ8_gKgGT2TQ1lmVWh-D00jqb2pEVsdwLQzh0qmWJPQzZ3iWq80HVDPSs182Cy04-vWVEwBizvQ96SjhOrVbh4</a>><br class="">><br class="">><br class="">><br class="">><br class="">> --<br class="">> To him who is able to keep you from falling and to present you before<br class="">> his glorious presence without fault and with great joy<br class="">><br class="">><br class="">> _______________________________________________<br class="">> Outages mailing list<br class="">><span class="Apple-converted-space"> </span><a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a><br class="">><span class="Apple-converted-space"> </span><a href="http://cp.mcafee.com/d/avndzhJ5xyXxEVsKC-rKrhKCy-qen4TXCQrFELCzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjq9J7HFkr8kJ6h3PtPpesRG9pyPtyL0QDYu1K2qKMM-l9OwXnaJPQzZ3tmVWh-xsxlK5LE2GJPQzZe00CQm4PhOUr1vF6y0QJRrDF7W6BQQg1nPfDIU2g5d409Y_RPh1aB6_EidICzASH16Ohmp" target="_blank" style="color: purple; text-decoration: underline;" class="">http://cp.mcafee.com/d/avndzhJ5xyXxEVsKC-rKrhKCy-qen4TXCQrFELCzBNBBUSztd5YQsK9LECQrFKnuv7fICO1fomHoG2BqG_2rqI_fZvBPqJlvxdJmvD-LOVJV1NxR1Z_HYYMM-YeoWZOWarxEVV5MsMqemrKth5dqWqJTT-l3PWApmU6CQjq9J7HFkr8kJ6h3PtPpesRG9pyPtyL0QDYu1K2qKMM-l9OwXnaJPQzZ3tmVWh-xsxlK5LE2GJPQzZe00CQm4PhOUr1vF6y0QJRrDF7W6BQQg1nPfDIU2g5d409Y_RPh1aB6_EidICzASH16Ohmp</a><br class=""><span class="im">></span><br class=""><span class="im">The information contained in this E-mail message is privileged, confidential, and may be protected from disclosure; please be aware that any unauthorized use, printing, copying, disclosure or dissemination of this communication may be subject to legal restriction or sanction. If you think that you have received this E-mail message in error, please reply to the sender. This E-mail message and any attachments have been scanned for viruses and are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened. It is the responsibility, however, of the recipient to ensure that it is virus free.</span><o:p class=""></o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class="">_______________________________________________<br class="">Outages mailing list<br class=""><a href="mailto:Outages@outages.org" style="color: purple; text-decoration: underline;" class="">Outages@outages.org</a><br class=""><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/outages</a><o:p class=""></o:p></div></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;" class=""><o:p class=""> </o:p></div></div></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Outages mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="mailto:Outages@outages.org" class="">Outages@outages.org</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="https://puck.nether.net/mailman/listinfo/outages" class="">https://puck.nether.net/mailman/listinfo/outages</a></span></div></blockquote></div><br class=""></div></div></body></html>