<div dir="ltr"><div><div>This is just a horribly painful example of why we/they should all follow the route of NetFlix ( <a href="http://techblog.netflix.com/2013/05/denominating-multi-region-sites.html">http://techblog.netflix.com/2013/05/denominating-multi-region-sites.html</a> ) and use multiple DNS providers. Denominator ( <a href="https://github.com/Netflix/denominator">https://github.com/Netflix/denominator</a> ) the tool NetFlix uses for this and open-sourced, currently supports managing (mirrored) DNS records across AWS Route53, RackSpace CloudDNS, DynECT and UltraDNS.<br></div><br></div>-Jason<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 21, 2016 at 1:09 PM, Patrick W. Gilmore via Outages <span dir="ltr"><<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">It is a shame in that PagerDuty is affected.<div><br></div><div>However, Dyn is far, far better positioned to withstand attacks than a company like PagerDuty could possibly be on their own. So I think PagerDuty did the right thing in using Dyn.</div><div><br></div><div>Remember, Twitter, Pingdom, github, and lots of other people use Dyn. And this is the first major outage I have heard of their service.</div><div><br></div><div>Let’s all be clear that Dyn has to be in the top 10 DNS infrastructures on the planet. Possibly the largest that sells DNS as a service. This is a serious attack, and the entire community should help track this miscreant down, then crush them like a bug.</div><div><span class="HOEnZb"><font color="#888888"><br><div>
<div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><div style="color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word"><font face="Inconsolata"><span style="color:rgb(0,0,0);font-variant:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important">-- </span></font><div style="color:rgb(0,0,0);font-variant:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><font face="Inconsolata">TTFN,</font></div><div style="color:rgb(0,0,0);font-variant:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><font face="Inconsolata">patrick</font></div></div></div></div></div></div></div></div>
</div></font></span><div><div class="h5">
<br><div><blockquote type="cite"><div>On Oct 21, 2016, at 1:00 PM, Terry Hardie via Outages <<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:</div><br class="m_-8970225396739631865Apple-interchange-newline"><div><div>It's a shame services like PagerDuty use Dyn. Now they're down too...<br><br>On Fri, Oct 21, 2016 at 9:57 AM, Justin Krejci via Outages<br><<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:<br><blockquote type="cite">Previously it was not affecting upper midwest for me, traces went to<br>Chicago.<br><br>Now traces to twitter DNS servers in Chicago are failing.<br><br><br>______________________________<wbr>__<br>From: Neil Hanlon via Outages [<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>]<br>Sent: Friday, October 21, 2016 11:16 AM<br>To: Terry Hardie; <a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a><br>Subject: Re: [outages] Dyn outage continuing<br><br>The attacks appear to be continuing again. We are experiencing issues in<br>Europe/Asia.<br><br>On Fri, Oct 21, 2016 at 12:13 PM Terry Hardie via Outages<br><<a href="mailto:outages@outages.org" target="_blank">outages@outages.org</a>> wrote:<br><blockquote type="cite"><br>Even though their page says the outage is resolved, I'm still getting<br>DNS failures (return SERVFAIL, not no answer) to their anycast<br>networks from California:<br><br>$ dig @<a href="http://ns2.p34.dynect.net" target="_blank">ns2.p34.dynect.net</a> <a href="http://video.twimg.com" target="_blank">video.twimg.com</a><br><br>; <<>> DiG 9.3.2 <<>> @<a href="http://ns2.p34.dynect.net" target="_blank">ns2.p34.dynect.net</a> <a href="http://video.twimg.com" target="_blank">video.twimg.com</a><br>; (1 server found)<br>;; global options: printcmd<br>;; Got answer:<br>;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 170<br>;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0<br><br>;; QUESTION SECTION:<br>;<a href="http://video.twimg.com" target="_blank">video.twimg.com</a>. IN A<br><br>;; Query time: 3103 msec<br>;; SERVER: 204.13.250.34#53(204.13.250.<wbr>34)<br>;; WHEN: Fri Oct 21 17:11:36 2016<br>;; MSG SIZE rcvd: 33<br>______________________________<wbr>_________________<br>Outages mailing list<br><a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/<wbr>mailman/listinfo/outages</a><br></blockquote><br>--<br><br>KAYAK<br><br>Neil Hanlon<br><br>Devops Engineer<br><br><br><a href="tel:%2B1%20978%20902%208171" value="+19789028171" target="_blank">+1 978 902 8171</a><br><br>______________________________<wbr>_________________<br>Outages mailing list<br><a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/<wbr>mailman/listinfo/outages</a><br><br></blockquote>______________________________<wbr>_________________<br>Outages mailing list<br><a href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/<wbr>mailman/listinfo/outages</a><br></div></div></blockquote></div><br></div></div></div></div><br>______________________________<wbr>_________________<br>
Outages mailing list<br>
<a href="mailto:Outages@outages.org">Outages@outages.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/outages" rel="noreferrer" target="_blank">https://puck.nether.net/<wbr>mailman/listinfo/outages</a><br>
<br></blockquote></div><br></div>