<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I’ve been going through our updates and it looks like another issue is that they (Crown Castle) were being denied access to POP(s) to do repairs. I’ve seen some fiber rings where the go into a downtown area where the full fiber route
enters Building A then goes back out to the street then Building B and so on down the line. So the ring is snaking in and out of every building on the route. Incomes a new building owner of several buildings with no real agreement in place and they
get into a dispute over payment and then they get pissed and cut you out of the building taking out anyone along the route.
<br>
<br>
Who knows what’s really going on. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-richey<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Jason Grider <jgrider@asc.edu><br>
<b>Date: </b>Friday, April 29, 2022 at 1:26 PM<br>
<b>To: </b>richey goldberg <richey.goldberg@gmail.com>, outages@outages.org <outages@outages.org><br>
<b>Subject: </b>Re: Cogent Fiber Cut Birmingham?<o:p></o:p></span></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black">My understanding is that Crown Castle was doing maintenance. Questions remain about what companies were notified about said maintenance and questions
<b>abound</b> about why it is taking 36 hours now to repair. I know we were not notified by cogent or any other providers. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black">Our cogent connection is up and has routes but we are located in the same building as Cogent in that case so Cogent just needed to reestablish connections to their router and we got routes
again. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black">I have other down connections that leave the same building that are dark fiber or long haul lit services. The dark fiber connections we have did not drop till 0200 or 0300 local. I was
literally logging into a device that I had been in less than 3minutes before when it dropped. The long haul services were cut at around 12 with every thing else. I got a message from a path monitoring system at 00:04 local on Thursday and then a whole bunch
of connection disconnects at 01:21 from logs. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black">The fiber network gremlins are in running rampant in central Alabama the last few days. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="color:black">As I type this another engineer said cogent just went down again. #*^%*^#%!<o:p></o:p></span></p>
</div>
</div>
<div id="ms-outlook-mobile-signature">
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jason Grider<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Network Engineer<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Alabama Supercomputer Authority<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Alabama Research and Education Network<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">800.338.8320<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">jgrider@asc.edu<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> richey goldberg <richey.goldberg@gmail.com><br>
<b>Sent:</b> Friday, April 29, 2022 12:00:42 PM<br>
<b>To:</b> Jason Grider <jgrider@asc.edu>; outages@outages.org <outages@outages.org><br>
<b>Subject:</b> Re: Cogent Fiber Cut Birmingham?</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:11.0pt">According to our guy who was on call at the time that time seems about right. I’m wondering what’s going on that they would have several cuts on the same ring at the same time that’s taken this long to
get repaired. We did hear in the last update that they had engineers from other nearby markets on site as part of the restoration. I’m wondering if this was something that was malicious or just really bad timing.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">-richey</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Jason Grider <jgrider@asc.edu><br>
<b>Date: </b>Friday, April 29, 2022 at 12:08 PM<br>
<b>To: </b>richey goldberg <richey.goldberg@gmail.com>, outages@outages.org <outages@outages.org><br>
<b>Subject: </b>Re: Cogent Fiber Cut Birmingham?</span><o:p></o:p></p>
</div>
<p class="xmsonormal"><span style="font-size:11.0pt">Yeah I got you, there are cut(s) in and around the Alabama Power HQ/6<sup>th</sup> Ave N in downtown that were affecting Cogent. We were still neighbored BGP with Cogent but they were not adverting any routes.
Started at about midnight central on Wed night Thursday morning for us. </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">We do currently have routes in BGP from Cogent. Not sure if that was resolved yesterday or this morning for us.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div>
<div>
<p class="xmsonormal"><span style="font-size:11.0pt">Jason Grider</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Network Engineer – ASA (AS 3464)</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">Alabama Supercomputer Authority</span><o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Outages <outages-bounces@outages.org> on behalf of richey goldberg via Outages <outages@outages.org><br>
<b>Date: </b>Friday, April 29, 2022 at 8:20 AM<br>
<b>To: </b>outages@outages.org <outages@outages.org><br>
<b>Subject: </b>[outages] Cogent Fiber Cut Birmingham?</span><o:p></o:p></p>
</div>
<p class="xmsonormal"><span style="font-size:11.0pt">Does anyone know about a Cogent fiber cut in Birmingham, AL? We have a customer who’s been down for 30+ hours and Cogent has given us a run around. Their updates have been conflicting. We have someone
asking for updates every 4-5 hours and every update is different from the last. Sometimes they tell us it’s fixed (when it’s not) then they tell us it’s being spliced, then they tell us a crew is in route. They are all just random responses. After
30+ hours I can’t imagine what they are doing that they have been on site splicing this long. They can’t tell us where it is other than it’s a a 3<sup>rd</sup> party vendor they are relying on.
<br>
<br>
Does anyone have some insight on what’s really going on?</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt">-richey </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:12.0pt"> </span><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>