[outages] Linode outage in Newark, NJ starting at 3:30 pm

[Blake Pfankuch]

Alex,
	Thanks for the information!  Good luck with those relays!

Blake Pfankuch
Cisco, Microsoft, Adtran and VMware Certification Information available upon request.

-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of Alex Rubenstein
Sent: Friday, August 31, 2012 5:34 PM
To: 'Lonnie.Bozeman at coresite.com'; 'reza at lethalnetworks.com'; 'outages at outages.org'
Subject: Re: [outages] Linode outage in Newark, NJ starting at 3:30 pm

The fault was related to an automatically operated circuit breaker on one of several generators. It would not close automatically, and had to be persuaded, which took some time. Further details will be made available to customers as soon as the post-event investigation is complete. 



----- Original Message -----
From: outages-bounces at outages.org <outages-bounces at outages.org>
To: reza a <reza at lethalnetworks.com>; outages at outages.org <outages at outages.org>
Sent: Fri Aug 31 18:52:35 2012
Subject: Re: [outages] Linode outage in Newark, NJ starting at 3:30 pm

It is common to have not only strings of batteries but an ATS, or automatic transfer switch. In this case, it sounds like the fault was on the ATS side of the plant based on the IR below.

- LB

-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of reza a
Sent: Friday, August 31, 2012 3:29 PM
To: outages at outages.org
Subject: Re: [outages] Linode outage in Newark, NJ starting at 3:30 pm

Is it common practice to have an array of batteries in between the colo and the generators to give you time to start the generators?

----- Original Message -----
From: "Sadiq Saif" <sadiq at asininetech.com>
To: "Jack Carrozzo" <jack at crepinc.com>
Cc: outages at outages.org
Sent: Friday, August 31, 2012 2:59:16 PM
Subject: Re: [outages] Linode outage in Newark, NJ starting at 3:30 pm

This is the incident report from NAC:
Subsequent to the utility power failure at our Cedar Knolls facility the generator that powers systems A and D did not start automatically.
Manual intervention was required to get generator power running.
Unfortunately before the generator was manually engaged customers on these systems experienced a complete power loss. We are still investigating why the generator did not engage automatically.Utility power has been restored, and transfer back to utility was successful.

On Fri, Aug 31, 2012 at 3:59 PM, Jack Carrozzo <jack at crepinc.com> wrote:
> 3:52pm (EDT): NAC has informed us of a power issue affecting at least 
> some portion of the datacenter. As soon as power is restored we are 
> poised to execute our recovery procedures to all affected systems.
>
> On Fri, Aug 31, 2012 at 3:46 PM, Smith, Kyle <ksmith at litle.com> wrote:
>> FYI,
>>
>>
>>
>> http://status.linode.com/
>>
>>
>>
>> We started getting alerts at roughly ~3:30
>>
>>
>>
>> - Kyle
>>
>>
>>
>> The information in this message is for the intended recipient(s) only 
>> and may be the proprietary and/or confidential property of Litle & 
>> Co., LLC, and thus protected from disclosure. If you are not the 
>> intended recipient(s), or an employee or agent responsible for 
>> delivering this message to the intended recipient, you are hereby 
>> notified that any use, dissemination, distribution or copying of this 
>> communication is prohibited. If you have received this communication 
>> in error, please notify Litle & Co. immediately by replying to this 
>> message and then promptly deleting it and your reply permanently from your computer.
>>
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
> _______________________________________________
> Outages mailing list
> Outages at outages.org
> https://puck.nether.net/mailman/listinfo/outages



--
Sadiq S
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org _______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages
_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages

_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages