[outages] Linode Fremont outage

[Mark Keymer]

So, As a reminder a few years back Amazon had a similar situation back 
east. (I forget which location) The utility power went off and one of 
the Generators was not working correctly. Before they were able to get 
it fixed the UPS Juice went out and a section of servers went off-line. 
A few minutes later power was returned. (Either utility or generator I 
don't recall) But it took apx 1 day to get most the servers (And Virtual 
servers on them) back online. However there was some servers / virtual 
servers that were not recoverable.

 From my experience loss of power on a single machine generally doesn't 
hurt the machine to badly. However when you have Hundreds go off-line 
there are bound to be a few with issues. It just seems to be that way.

Also I do not Fault HE or Linode for only have Grid + Generator. However 
I bet there are some UPS in there too.

People tend to believe that there is just so much redundancy that how 
could a thing happen. People need to see reality and as mentioned before 
you do get what you pay for. However most people do not want to pay for 
tons of redundancy. And even when you do I foresee there are things that 
can still take you down.

If there is more to the story sure I would like to hear as well. But 
honestly I am guessing it was a failure with no foul play happening.

Sincerely,

Mark

On 5/31/2015 7:44 AM, Greg Ihnen via Outages wrote:
>
> There must be more to the story because the equipment on a particular 
> generator seems to have gotten damaged, no?
>
> Greg
>
> On May 31, 2015 9:39 AM, "Alex Forster via Outages" 
> <outages at outages.org <mailto:outages at outages.org>> wrote:
>
>     The redundancy was grid + generator. Unfortunately both failed.
>
>     (Logo) 	
>     	
>     Alex Forster
>     Network Engineer | Linode
>
>
>     *From:* Scott Howard via Outages <outages at outages.org>
>     <mailto:outages at outages.org>
>     *Sent:* Sunday, May 31, 2015 4:37AM
>     *To:* Frank Bulk <frnkblk at iname.com> <mailto:frnkblk at iname.com>
>     *Cc:* Outages Mailing List <outages at outages.org>
>     <mailto:outages at outages.org>
>     *Subject:* Re: [outages] Linode Fremont outage
>
>>     On Sat, May 30, 2015 at 8:04 PM, Frank Bulk via Outages
>>     <outages at outages.org <mailto:outages at outages.org>> wrote:
>>
>>         Are suites tied to just one generator?
>>
>>
>>     Unless things have changed since I was last there, yes.  HE do
>>     not offer redundant power (at least, not at FMT2).
>>
>>     HE offer a great price point, but as the saying goes, you get
>>     what you pay for!
>>
>>       Scott
>>
>>
>>     _______________________________________________
>>     Outages mailing list
>>     Outages at outages.org  <mailto:Outages at outages.org>
>>     https://puck.nether.net/mailman/listinfo/outages
>
>
>     _______________________________________________
>     Outages mailing list
>     Outages at outages.org <mailto:Outages at outages.org>
>     https://puck.nether.net/mailman/listinfo/outages
>
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20150531/3d04bb70/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 3418 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/outages/attachments/20150531/3d04bb70/attachment.png>