<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><title>Re: [Outages-discussion] [outages] Fwd: RE: FYI Netflix is down</title><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Tahoma","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'>According to a Minnesota Public Radio this morning, “…Amazon worked to restore both its primary power supply and backup generators at its Virginia facility.”<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'><a href="http://www.marketplace.org/topics/tech/storms-hit-amazon-data-center-netflix-and-instagram">http://www.marketplace.org/topics/tech/storms-hit-amazon-data-center-netflix-and-instagram</a><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'>I have not seen this documented elsewhere, so I wonder if it’s speculation or based on a statement that Amazon provided NPR/MPR or was communicated elsewhere.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'>Frank<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> outages-discussion-bounces@outages.org [mailto:outages-discussion-bounces@outages.org] <b>On Behalf Of </b>Bill Wichers<br><b>Sent:</b> Saturday, June 30, 2012 9:46 AM<br><b>To:</b> blackham@gmail.com; fergdawgster@gmail.com<br><b>Cc:</b> outages-discussion@outages.org<br><b>Subject:</b> Re: [Outages-discussion] [outages] Fwd: RE: FYI Netflix is down<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p><span style='font-size:10.0pt'>Does anyone know what actually caused the failure? I know there was a "power outage" due to weather, but presumably the physical site would have also had generator backup so that must have also failed?<br><br>[Sent using Blackberry Messaging]<br><br>----- Original Message -----<br>From: <a href="mailto:outages-discussion-bounces@outages.org">outages-discussion-bounces@outages.org</a> <<a href="mailto:outages-discussion-bounces@outages.org">outages-discussion-bounces@outages.org</a>><br>To: Paul Ferguson <<a href="mailto:fergdawgster@gmail.com">fergdawgster@gmail.com</a>><br>Cc: <a href="mailto:outages-discussion@outages.org">outages-discussion@outages.org</a> <<a href="mailto:outages-discussion@outages.org">outages-discussion@outages.org</a>><br>Sent: Sat Jun 30 01:46:05 2012<br>Subject: Re: [Outages-discussion] [outages] Fwd: RE: FYI Netflix is down<br><br>That is a good point. Why didn't they survive in an adjacent AZ in<br>Ashburn? Reports say only a single AZ lost power.<br><br>On Fri, Jun 29, 2012 at 10:15 PM, Paul Ferguson <<a href="mailto:fergdawgster@gmail.com">fergdawgster@gmail.com</a>> wrote:<br>> I'm looking forward to additional details on this outage -- Netflix<br>> has previously avoided EC2 outages due to their creative use of Chaos<br>> Monkey:<br>><br>> <a href="http://techblog.netflix.com/2011/07/netflix-simian-army.html">http://techblog.netflix.com/2011/07/netflix-simian-army.html</a><br>><br>> FYI,<br>><br>> - ferg<br>><br>> On Fri, Jun 29, 2012 at 10:07 PM, Kevin Blackham <<a href="mailto:blackham@gmail.com">blackham@gmail.com</a>> wrote:<br>><br>>> On Fri, Jun 29, 2012 at 9:40 PM, Paul Ferguson <<a href="mailto:fergdawgster@gmail.com">fergdawgster@gmail.com</a>> wrote:<br>>>> <a href="http://venturebeat.com/2012/06/29/amazon-outage-netflix-instagram-pinterest/">http://venturebeat.com/2012/06/29/amazon-outage-netflix-instagram-pinterest/</a><br>>><br>>> "The outage underscores the vulnerabilities of depending on the public<br>>> cloud versus using your own data centers."<br>>><br>>> Let me fix that: "The outage underscores the lack of attention to<br>>> failure tolerance by many affected services." Building my own sites<br>>> isn't going to keep them from failing. Every datacenter will fail<br>>> eventually.<br>>><br>>> Granted, when an Amazon AZ fails, history has shown the thrash induced<br>>> by recovery mechanisms can also take out adjacent AZs (e.g. network<br>>> overloads vs. EBS), but that doesn't stop an architect from<br>>> considering regional redundancy.<br>>> _______________________________________________<br>>> Outages-discussion mailing list<br>>> <a href="mailto:Outages-discussion@outages.org">Outages-discussion@outages.org</a><br>>> <a href="https://puck.nether.net/mailman/listinfo/outages-discussion">https://puck.nether.net/mailman/listinfo/outages-discussion</a><br>><br>><br>><br>> --<br>> "Fergie", a.k.a. Paul Ferguson<br>> fergdawgster(at)gmail.com<br>_______________________________________________<br>Outages-discussion mailing list<br><a href="mailto:Outages-discussion@outages.org">Outages-discussion@outages.org</a><br><a href="https://puck.nether.net/mailman/listinfo/outages-discussion">https://puck.nether.net/mailman/listinfo/outages-discussion</a></span><o:p></o:p></p></div></body></html>