[outages] Mimecast SMTP outage
Casimir Couvillion
ccouvillion at texonlp.com
Tue May 3 12:53:57 EDT 2022
>From status.mimecast.com:
May 3, 2022 4:28PM UTC
[Monitoring]
We can confirm that email processing and all Mimecast services have been restored for our US-hosted customers. All services should now be functioning normally and processing queues are approaching normal levels as email is retried on the normal SMTP schedules. We continue to monitor and will provide updates here.
May 3, 2022 3:58PM UTC
[Identified]
We can confirm we experienced a major power outage in one of our US data centers that impacted all power supply sources, including the backup generators, which in turn caused a cascading issue and resulted in degraded performance. Midway through our Disaster Recovery Process, the power was restored which complicated our ability to recover. As of now, services are available and we are processing email but customers may still experience delays until the backlog clears. We sincerely apologize for the extended outage and the disruption caused.
-
Casimir Couvillion - Texon LP - Manager, IT Operations and Helpdesk - ccouvillion at texonlp.com
-----Original Message-----
From: Outages <outages-bounces at outages.org> On Behalf Of Mike Lewinski via Outages
Sent: Tuesday, May 3, 2022 11:49 AM
To: outages (outages at outages.org) <outages at outages.org>
Subject: [outages] Mimecast SMTP outage
I'm unable to reach the SMTP service on us-smtp-inbound-1.mimecast.com or us-smtp-inbound-2.mimecast.com from multiple markets: Dallas, Denver, Phoenix. I can reach them from Atlanta and Seattle. They are reachable on at least one address in Los Angeles, unreachable on others.
Dallas:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 205.139.110.221...
Trying 207.211.30.221...
Trying 207.211.30.141...
Trying 205.139.110.141...
Trying 205.139.110.242...
Trying 207.211.30.242...
telnet: Unable to connect to remote host: Connection timed out
Denver:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.221...
Trying 207.211.30.242...
Trying 207.211.30.141...
Trying 205.139.110.242...
Trying 205.139.110.221...
Trying 205.139.110.141...
telnet: Unable to connect to remote host: Connection timed out
Finally found one host in LA that worked after two failures:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 205.139.110.221...
Trying 205.139.110.141...
Trying 207.211.30.242...
Connected to us-smtp-inbound-1.mimecast.com.
Escape character is '^]'.
220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:38:11 -0400 QUIT
221 Service closing transmission channel [uajtgwEROGqfo99gP4hQyg.us397] Connection closed by foreign host.
$ mtr -r -c3 -w 205.139.110.221
Start: 2022-05-03T10:43:06-0600
HOST: los-angeles Loss% Snt Last Avg Best Wrst StDev
1.|-- LA624-C6504-1-VL624.massivenetworks.net 0.0% 3 0.6 0.5 0.5 0.6 0.1
2.|-- 199.45.152.18 0.0% 3 0.4 0.4 0.4 0.4 0.0
3.|-- xe-10-0-0.edge1.LosAngeles9.Level3.net 0.0% 3 4.7 2.0 0.6 4.7 2.4
4.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
5.|-- MIMECAST-SE.ear1.NewYork6.Level3.net 0.0% 3 68.8 68.8 68.7 68.8 0.1
6.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
$ mtr -r -c3 -w 207.211.30.242
Start: 2022-05-03T10:47:11-0600
HOST: los-angeles Loss% Snt Last Avg Best Wrst StDev
1.|-- LA624-C6504-1-VL624.massivenetworks.net 0.0% 3 0.6 0.5 0.4 0.6 0.1
2.|-- 199.45.152.18 0.0% 3 0.4 0.4 0.4 0.5 0.1
3.|-- et-0-0-6-2.cr4-lax2.ip4.gtt.net 0.0% 3 0.4 0.5 0.4 0.6 0.1
4.|-- ae23.cr3-lax1.ip4.gtt.net 0.0% 3 1.3 1.3 1.1 1.3 0.1
5.|-- ae-3.r01.lsanca07.us.bb.gin.ntt.net 0.0% 3 1.3 1.2 1.2 1.3 0.1
6.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
7.|-- ae-3.r24.snjsca04.us.bb.gin.ntt.net 33.3% 3 15.4 13.7 12.1 15.4 2.4
8.|-- ae-8.r23.chcgil09.us.bb.gin.ntt.net 0.0% 3 57.3 58.0 57.3 58.9 0.8
9.|-- ae-0.r22.chcgil09.us.bb.gin.ntt.net 0.0% 3 64.4 61.5 58.9 64.4 2.8
10.|-- ae-6.r21.nwrknj03.us.bb.gin.ntt.net 0.0% 3 69.7 71.2 69.4 74.6 2.9
11.|-- ae-11.r01.nycmny17.us.bb.gin.ntt.net 0.0% 3 70.2 70.0 69.8 70.2 0.2
12.|-- ce-0-8-0-3-200.r01.nycmny17.us.ce.gin.ntt.net 0.0% 3 74.9 87.4 74.9 112.2 21.5
13.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
$
Phoenix:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.141...
Trying 207.211.30.221...
Trying 205.139.110.141...
Trying 205.139.110.242...
Trying 207.211.30.242...
Trying 205.139.110.221...
telnet: Unable to connect to remote host: Connection timed out $
Seattle:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.141...
Connected to us-smtp-inbound-1.mimecast.com.
Escape character is '^]'.
220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:34:04 -0400 QUIT
221 Service closing transmission channel [0EM41sKhMWizUnOQs3cnYQ.us479] Connection closed by foreign host.
$ mtr -r -c3 -w us-smtp-inbound-1.mimecast.com
Start: 2022-05-03T10:37:23-0600
HOST: seattle Loss% Snt Last Avg Best Wrst StDev
1.|-- 204.144.179.129 0.0% 3 0.4 0.4 0.4 0.5 0.0
2.|-- 9-1-4.ear1.Seattle3.Level3.net 0.0% 3 0.5 0.6 0.5 0.7 0.1
3.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
4.|-- MIMECAST-SE.ear1.NewYork6.Level3.net 0.0% 3 76.6 76.6 76.5 76.7 0.1
5.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
$
Atlanta:
$ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.242...
Connected to us-smtp-inbound-1.mimecast.com.
Escape character is '^]'.
220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:08:11 -0400 QUIT
221 Service closing transmission channel [M0wFRdZ-NhONlAOBhS22Sg.us227] Connection closed by foreign host.
$ mtr 207.211.30.242 -w -r -c 3
Start: Tue May 3 10:17:58 2022
HOST: atlanta Loss% Snt Last Avg Best Wrst StDev
1.|-- Atl55-C6504-1-VL412.massivenetworks.net 0.0% 3 0.3 0.4 0.3 0.7 0.0
2.|-- 6-2-2.ear3.Atlanta2.Level3.net 0.0% 3 0.3 0.3 0.3 0.3 0.0
3.|-- 4.69.217.206 0.0% 3 17.9 7.3 1.5 17.9 9.1
4.|-- NTT-level3-100G.Atlanta2.Level3.net 0.0% 3 1.1 1.0 0.8 1.1 0.0
5.|-- ae-4.r23.atlnga05.us.bb.gin.ntt.net 0.0% 3 1.3 2.6 0.5 6.0 2.9
6.|-- ae-4.r25.asbnva02.us.bb.gin.ntt.net 0.0% 3 13.0 13.6 13.0 14.2 0.0
7.|-- ae-4.r21.nwrknj03.us.bb.gin.ntt.net 33.3% 3 18.9 18.8 18.7 18.9 0.0
8.|-- ae-11.r01.nycmny17.us.bb.gin.ntt.net 0.0% 3 19.3 19.2 19.0 19.3 0.0
9.|-- ce-0-8-0-3-200.r01.nycmny17.us.ce.gin.ntt.net 0.0% 3 24.8 24.7 24.5 24.8 0.0
10.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20220503/b80973b3/attachment-0001.htm>
More information about the Outages
mailing list