[outages] Power outage - Cologix 3 - Montreal, QC Canada

[Eric Dugas]

The official report:

" Dear Valued Customers, 
 
During annual systems inspections an emergency power off (EPO) switch was inadvertently triggered.  
This switch, when triggered, shut down power to bus duct F.  (EPOs are designed to shut down all power, including utility and generator power, and, as a result, all power to bus duct F was shut down.)  The de-energizing of this bus duct removed power from multiple pieces of mechanical and electrical equipment, including a chiller and UPS2.  Customers supported by UPS2 experienced a 14 minute outage, after the UPS depleted its batteries (which provided 11 minutes and 57 seconds of run time).   Additionally, some customers saw higher temperatures in the data center. All power has been restored and all temperatures have returned to normal. 
 
Immediate corrective actions:
All further maintenance and inspection activity has been halted until such time as systems, policies and procedures are ensured.
We will be providing further information as it becomes available, including a formal root cause document with corrective actions.
We sincerely apologize for the impact this had to your business and that of your customers.  Cologix will act quickly and diligently to ensure that all corrective actions needed are taken."

Eric

-----Original Message-----
From: Paul Stewart [mailto:pstewart726 at gmail.com] On Behalf Of Paul Stewart
Sent: January 24, 2014 3:07 PM
To: Jason Lixfeld; Eric Dugas
Cc: outages at outages.org
Subject: Re: [outages] Power outage - Cologix 3 - Montreal, QC Canada

We experienced no power loss at that location - will be curious to see the report come out.

Paul


On 1/24/2014, 11:02 AM, "Jason Lixfeld" <jason at lixfeld.ca> wrote:

>We're in C3 with a couple of cabinets and aren't seeing any issues with 
>our equipment or our waves out of 1250.  We're on their DC grid though, 
>not sure exactly where my wave providers are located, however.
>
>FYI: My QIX link is still up too.
>
>On Jan 24, 2014, at 10:47 AM, Eric Dugas <EDugas at zerofail.com> wrote:
>
>> We lost a full cabinet (with dual electric feed) at 1250 
>>Rene-Levesque, Montreal, QC Canada. A lot of carriers are present at 
>>that datacenter, Level3, Cogent, Hibernia, etc..
>> 
>> Just received this email from Cologix support:
>> 
>> "Hello,
>> 
>> Please be advised that Cologix is experiencing a commercial power 
>>issue at our Cologix 3, 1250 René Levesque datacenter.
>> Our technicians are informed and working on repairing the problem. An 
>>update will be provided to you in the next 30 minutes.
>> 
>> Regards,"
>> 
>> Eric Dugas
>> edugas at zerofail.com
>> Network Management Group
>> ZEROFAIL / AS40191
>> Montreal (514) 848-9198 x310
>> New York (212) 796-5853 x310
>> Toronto (416) 800-4209 x310
>> 1-888-GO-0FAIL (460-3245)
>> Facebook  |  Twitter  | LinkedIn  | Google+ www.zerofail.com
>> 
>> 
>> _______________________________________________
>> Outages mailing list
>> Outages at outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>
>
>_______________________________________________
>Outages mailing list
>Outages at outages.org
>https://puck.nether.net/mailman/listinfo/outages