<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Century Schoolbook";
panose-1:2 4 6 4 5 5 5 2 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Century Schoolbook","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Century Schoolbook","serif";
color:#1F497D;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Century Schoolbook","serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'>Team CYMRU is having issues with their BGP Bogon Service. <br><br><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'>Below is an email I received from them a little bit ago.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoPlainText>To all,<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>We have identified a hardware problem associated with the fullbogons peering sessions that were taken offline roughly 12 hours ago. The service has been disabled and at this time we do not have an ETA on when service will be restored.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>We will issue updates at approximately 8am, 12pm, and 5pm EST until this outage is resolved.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>For the finer details please read below.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>8pm EST 7/15/13, Team Cymru was notified of a couple problems with our Bogon service. At this time we did not know the severity and scope of the larger problem.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>11pm EST 7/15/13 the issue was escalated from Operations to Engineering.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>1am EST 7/16/13, we confirmed the propagation of incorrect prefixes.<o:p></o:p></p><p class=MsoPlainText>Workarounds failed to fully mitigate the issue, so all sessions were disabled. This ceased the propagation of incorrect prefixes.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>~2am EST 7/16/13, we sent out notices to everyone we have current contact details on regarding the outage.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>12:15pm EST 7/16/13, we are working on replacing / reconfiguring hardware to resolve this outage.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Regards,<o:p></o:p></p><p class=MsoPlainText>Dave<o:p></o:p></p><p class=MsoPlainText>Team Cymru<o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Century Schoolbook","serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Outages [mailto:outages-bounces@outages.org] <b>On Behalf Of </b>Paul Stewart<br><b>Sent:</b> Tuesday, July 16, 2013 1:06 PM<br><b>To:</b> Ben Bartsch; outages@outages.org<br><b>Subject:</b> Re: [outages] 216.115.160.0/20 being blackholed<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>They were experiencing some issues and sent out an update today – you may wish to reach out to them if you haven't heard anything. The details did mention something about incorrect prefixes getting announced…<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>Paul<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>From: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:black'>Ben Bartsch <<a href="mailto:uwcableguy@gmail.com">uwcableguy@gmail.com</a>><br><b>Date: </b>Tuesday, 16 July, 2013 7:29 AM<br><b>To: </b><<a href="mailto:outages@outages.org">outages@outages.org</a>><br><b>Subject: </b>[outages] 216.115.160.0/20 being blackholed<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><blockquote style='border:none;border-left:solid #B5C4DF 4.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-right:0in' id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>FYI - got a call from a customer last night about a Xerox site unreachable through us. Found the prefix coming from Cymru. Instructed our NOC to work with them to resolve. Cymru first claimed they were not sending it because it was not a bogon. Once we proved it was coming from them, they woke somebody up at our request and found it listed as a 'bad route', whatever that means. Their workaround was to shut down the peering to us.<o:p></o:p></span></p><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>Anybody else use Cymru and have this issue? Our 2 peering addresses are 38.229.66.20 and 193.231.140.82<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>Thanks.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>-ben<o:p></o:p></span></p></div></div><p class=MsoNormal><span style='font-size:10.5pt;font-family:"Calibri","sans-serif";color:black'>_______________________________________________ Outages mailing list <a href="mailto:Outages@outages.org">Outages@outages.org</a> <a href="https://puck.nether.net/mailman/listinfo/outages">https://puck.nether.net/mailman/listinfo/outages</a> <o:p></o:p></span></p></blockquote></div></body></html>