[Outages-discussion] [outages] Knology email servers

[Frank Bulk]

Moving this over to outages-discussion: 

Via twitter:

~ 3pm Central: 
	We have started the restoration process for emails that were stored
on our server. This process will take a few days.
	We understand the importance of email that was stored on our server
and it is a very high priority to get these messages restored ASAP.
	We apologize for the inconvenience caused by the failure of our
email storage hardware. Service has been restored using new hardware.

For those who don't have a complete DR plan, perhaps think about looking at
your email system!  If *days* isn't acceptable to your CEO, this event may
get you the resources to build and execute on such a plan.

Frank

-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On
Behalf Of Frank Bulk
Sent: Sunday, December 18, 2011 12:12 PM
To: mailop at mailop.org; outages at outages.org
Subject: Re: [outages] Knology email servers

Latest news from twitter regarding Knology:

11:55 am (Central): Email service will begin being restored in the next
hour. To prevent a possible future disruption, we are using a new email
system. 
11:57 am: Any emails sent since Saturday at 1:30 eastern should be in queue
and will start populating email boxes upon restoration.
11:58 am: Once this step is completed, we will begin to restore old emails
stored on our server.

****
Knology confirmed Saturday afternoon that they had server failure.  They
have multiple MX records, so it appears that they have email filtering
redundancy, but all their eggs in one email server basket.  Not sure if
that's really the case, or if their supporting infrastructure (SAN?
Virtualization?) failed.  

Around 9 pm (Central) Saturday night they reported that they were restoring
from backup, but based on these latest tweets, I'd have to guess that that
didn't go as planned, so they're starting fresh to at least let the pending
email get delivered while they repopulate the mailboxes from a backup.

Frank

-----Original Message-----
From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On
Behalf Of Frank Bulk
Sent: Saturday, December 17, 2011 4:05 PM
To: mailop at mailop.org; outages at outages.org
Subject: [outages] Knology email servers

Looks like Knology's email servers have been down for a few hours now.
Twitter reports confirm the same.

Frank

_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages


_______________________________________________
Outages mailing list
Outages at outages.org
https://puck.nether.net/mailman/listinfo/outages