[outages] Linode Fremont outage

[Brendan Mannella]

Sounds like they have a block design, with one generator feeding a specific
bus which is designed for capacity not redundancy. As they need more
capacity they add another bus, but don't offer the ability to connect to
multiple buses for a 2N type configuration.

On Sunday, May 31, 2015, DTNX Postmaster via Outages <outages at outages.org>
wrote:

> On 31 May 2015, at 18:47, ryanL via Outages <outages at outages.org
> <javascript:;>> wrote:
>
> > for all we know the generator was proactively tested on a regular basis.
> testing doesn't preclude failure on a subsequent run.
>
> And until they release a thorough post-incident report, there's no more
> value in your answer than in Jim's. Like, 'for all we know', their systems
> are oversubscribed, they're behind on maintenance and planned upgrades, and
> their tests aren't reflective of the actual load during an outage.
>
> Eight generators total, and the failure of a single one takes out an
> entire suite. If I were Linode, I'd want a damn good explanation for that
> one.
>
> --
>
> > On Sun, May 31, 2015 at 9:23 AM Jim Popovitch via Outages <
> outages at outages.org <javascript:;>> wrote:
> > On Sun, May 31, 2015 at 12:16 PM, Chris Vervais <cvervais at me.com
> <javascript:;>> wrote:
> > >
> > >> On May 31, 2015, at 08:06, Jim Popovitch via Outages <
> outages at outages.org <javascript:;>> wrote:
> > >>
> > >> On Sun, May 31, 2015 at 11:00 AM, Mark Keymer via Outages
> > >> <outages at outages.org <javascript:;>> wrote:
> > >>> But honestly I am guessing it was a failure with no foul play
> happening.
> > >>
> > >> Apparently no solid proactive testing plan either.
> > >
> > > For all we know the generator was checked last week and was fine.
> > >
> > > In the absence of all the facts we should probably restrain from
> assuming one thing or another and give them the benefit of the doubt. And,
> that they're taking this is a learning experience and delving into where
> gaps exist and how to close them.
> >
> > Of course, for a new DC or relatively new DC operator.  But come on,
> > HE @ FMT2 are where I would expect perfection.
> >
> > -Jim P.
>
>
> _______________________________________________
> Outages mailing list
> Outages at outages.org <javascript:;>
> https://puck.nether.net/mailman/listinfo/outages
>


-- 
Brendan Mannella

TeraSwitch Inc.
Main - 1.412.945.7045
Direct - 1.412.945.7049
eFax - 1.412.945.7049
Colocation . Cloud . Connectivity
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20150531/cbabc54b/attachment.htm>