<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
That post is from an outage back in 2010, not the outage from
yesterday...<br>
<br>
<br>
<div class="moz-signature">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<title>Signature</title>
<style>
/* Style Definitions */
p.Normal
{mso-style-parent:"";
margin:0in;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:8.5pt;
font-family:"Arial";
mso-fareast-font-family:"MS Mincho";
mso-fareast-language:JA;}
/**/
span.nobr {white-space}
a {color:0000FF;}
a:visited {color:800080;}
a:active {color:0000FF;}
a:hover {color:00D040;}
span.text {font-size:8.5pt;font-family:Arial;color:gray;}
</style>
<div>
<br>
<p class="Normal"> <span class="text"> <span class="nobr">To
ensure prompt service, please use "Reply All" when
responding </span> <br>
<br>
<span class="nobr">Robert Colon | </span> <span
class="nobr">TCSA Tier 3 | </span> <span class="nobr">EthoStream
LLC | </span> <span class="nobr">P: (877) 282-2519 x2114
| </span> <span class="nobr">F: (414) 258-8307 | </span>
<span class="nobr">20800 Swenson Drive, Suite 175 Waukesha,
WI 53186 | </span> <span class="nobr"><a
href="http://www.telkonet.com/"><a class="moz-txt-link-abbreviated" href="http://www.telkonet.com">www.telkonet.com</a></a> | </span>
<span class="nobr"><a href="http://www.ethostream.com/">www.ethostream.com</a>
| </span> <span class="nobr"><a
href="https://twitter.com/telkonet">@Telkonet</a> </span>
</span> </p>
<br>
</div>
</div>
<div class="moz-cite-prefix">On 9/29/2015 12:44 PM, Scott Voll via
Outages wrote:<br>
</div>
<blockquote
cite="mid:CAHgd+38s5Sc-ysfejonJpOLrLYSD7x89E+wrR=i4HXn17GQPTQ@mail.gmail.com"
type="cite">
<div dir="ltr"><a moz-do-not-send="true"
href="https://www.facebook.com/notes/facebook-engineering/more-details-on-todays-outage/431441338919">https://www.facebook.com/notes/facebook-engineering/more-details-on-todays-outage/431441338919</a><br>
<div><br>
</div>
<div>Guess it's already out there.</div>
<div><br>
</div>
<div>Scott</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Sep 29, 2015 at 9:36 AM,
William Byrd via Outages <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:outages@outages.org"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:outages@outages.org">outages@outages.org</a></a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hey Rafael -
<div><br>
</div>
<div>I am sure it would make a very interesting post
mortem post, and I will ask if anyone would be willing
to put something together to share externally. Right now
I can't share any details, so I'll just have to
apologize (again) for the inconvenience and see if we
can get something posted.</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div>William Collier-Byrd<br>
<a moz-do-not-send="true"
href="mailto:will@collier-byrd.net" target="_blank">will@collier-byrd.net</a></div>
</div>
<br>
<div class="gmail_quote">On Tue, Sep 29, 2015 at 8:25 AM,
Rafael Possamai <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:rafael@e2wsolutions.com"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:rafael@e2wsolutions.com">rafael@e2wsolutions.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Will,
<div><br>
</div>
<div>Are you able to disclose any high level details
of what happened? It's just interesting to know
how such a large infrastructure, with assumed
redundancies, can go down completely. I know AWS
had a similar outage, which lasted even longer,
but it was a series of cascading problems due to
human error. Was it an SDN bad config push?</div>
<div>
<div class="h5">
<div class="gmail_extra"><br clear="all">
<div>
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div
style="font-size:12.8000001907349px"><br>
</div>
<div
style="font-size:12.8000001907349px"><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Mon, Sep 28, 2015
at 2:59 PM, William Collier-Byrd via Outages
<span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:outages@outages.org"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:outages@outages.org">outages@outages.org</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div>
<div>I am starting to feel like a broken
record, but we're aware and working on
it. <br>
<br>
<div>--<br>
Will Collier-Byrd</div>
<br>
</div>
<div>
<div><br>
<br>
<br>
<div class="gmail_quote">On Mon, Sep
28, 2015 at 12:56 PM -0700, "Mike
Phipps via Outages" <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:outages@outages.org"
target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:outages@outages.org">outages@outages.org</a></a>></span>
wrote:<br>
<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="3D"ltr"">
<div dir="ltr">Confirmed on
T-Mobile LTE and Comcast
Business Class in metro
Detroit. It is that same
message.<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Mon, Sep 28, 2015 at 3:27
PM, Bob Colon via Outages
<span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:outages@outages.org" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:outages@outages.org">outages@outages.org</a></a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div bgcolor="#FFFFFF"
text="#000000">
Yup....getting the
same here, <br>
<div>
<div> <br>
<p> <span> <span>To
ensure prompt
service,
please use
"Reply All"
when
responding </span>
<br>
<br>
<span>Robert
Colon | </span>
<span>TCSA
Tier 3 | </span>
<span>EthoStream
LLC | </span>
<span>P: <a
moz-do-not-send="true"
href="tel:%28877%29%20282-2519%20x2114" value="+18772822519"
target="_blank">(877)
282-2519 x2114</a>
| </span> <span>F:
<a
moz-do-not-send="true"
href="tel:%28414%29%20258-8307" value="+14142588307" target="_blank">(414)
258-8307</a> |
</span> <span>20800
Swenson Drive,
Suite 175
Waukesha, WI
53186 | </span>
<span><a
moz-do-not-send="true"
href="http://www.telkonet.com" target="_blank"><a class="moz-txt-link-abbreviated" href="http://www.telkonet.com">www.telkonet.com</a></a> | </span>
<span><a
moz-do-not-send="true"
href="http://www.ethostream.com/" target="_blank"><a class="moz-txt-link-abbreviated" href="http://www.ethostream.com">www.ethostream.com</a></a>
| </span> <span><a
moz-do-not-send="true" href="https://twitter.com/telkonet"
target="_blank">@Telkonet</a>
</span> </span>
</p>
<br>
</div>
</div>
<span>
<div>On 9/28/2015
2:23 PM, James
Hartig via Outages
wrote:<br>
</div>
</span><span>
<blockquote
type="cite">
<div dir="ltr">Anyone
else getting:
<div><br>
<div>
<div>Sorry,
something went
wrong.</div>
<div>We're
working on it
and we'll get
it fixed as
soon as we
can.</div>
<div><br>
</div>
<div>when you
go to
Facebook?</div>
<div>--</div>
<div>James
Hartig</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Outages mailing list
<a moz-do-not-send="true" href="mailto:Outages@outages.org" target="_blank">Outages@outages.org</a>
<a moz-do-not-send="true" href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a>
</pre>
</blockquote>
<br>
</span></div>
<br>
<hr>
<p
style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif"><a
moz-do-not-send="true" href="http://www.telkonet.com" target="_blank"><img
moz-do-not-send="true"
src="http://www.telkonet.com/images/signature/telkonetlogo.gif"><img
moz-do-not-send="true"
src="http://www.telkonet.com/images/signature/telkosig.gif"
style="font-size:1.3em"></a><a moz-do-not-send="true"
href="http://www.ethostream.com"
target="_blank"><img
moz-do-not-send="true"
src="http://www.ethostream.com/images/signature/ethosig.gif"
style="font-size:1.3em"></a></p>
<p><font color="#808080"
size="1"
face="Arial">This
e-mail and any files
transmitted with it
are private and
confidential and are
solely for the use
of the addressee. It
may contain material
which is legally
privileged. If you
are not the
addressee or the
person responsible
for delivering to
the addressee, be
advised that you
have received this
e-mail in error and
that any use of it
is strictly
prohibited. If you
have received this
e-mail in error
please contact us at
<a
moz-do-not-send="true"
href="tel:%28414%29%20223-0473" value="+14142230473" target="_blank">(414)
223-0473</a>.</font></p>
<br>
_______________________________________________<br>
Outages mailing list<br>
<a
moz-do-not-send="true"
href="mailto:Outages@outages.org" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:Outages@outages.org">Outages@outages.org</a></a><br>
<a
moz-do-not-send="true"
href="https://puck.nether.net/mailman/listinfo/outages" rel="noreferrer"
target="_blank"><a class="moz-txt-link-freetext" href="https://puck.nether.net/mailman/listinfo/outages">https://puck.nether.net/mailman/listinfo/outages</a></a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Outages mailing list<br>
<a moz-do-not-send="true"
href="mailto:Outages@outages.org"
target="_blank">Outages@outages.org</a><br>
<a moz-do-not-send="true"
href="https://puck.nether.net/mailman/listinfo/outages"
rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
_______________________________________________<br>
Outages mailing list<br>
<a moz-do-not-send="true" href="mailto:Outages@outages.org">Outages@outages.org</a><br>
<a moz-do-not-send="true"
href="https://puck.nether.net/mailman/listinfo/outages"
rel="noreferrer" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Outages mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Outages@outages.org">Outages@outages.org</a>
<a class="moz-txt-link-freetext" href="https://puck.nether.net/mailman/listinfo/outages">https://puck.nether.net/mailman/listinfo/outages</a>
</pre>
</blockquote>
<br>
</body>
</html>
<br>
<hr><p style="font-size:1.3em;font-family:Arial,Helvetica,sans-serif"><a href="http://www.telkonet.com" target="_blank"><img src="http://www.telkonet.com/images/signature/telkonetlogo.gif"><img src="http://www.telkonet.com/images/signature/telkosig.gif" style="font-size:1.3em"></a><a href="http://www.ethostream.com" target="_blank"><img src="http://www.ethostream.com/images/signature/ethosig.gif" style="font-size:1.3em"></a></p>
<p><font face="Arial" size="1" color="#808080">This e-mail and any files transmitted with it are private and confidential and are solely for the use of the addressee. It may contain material which is legally privileged. If you are not the addressee or the person responsible for delivering to the addressee, be advised that you have received this e-mail in error and that any use of it is strictly prohibited. If you have received this e-mail in error please contact us at (414) 223-0473.</font></p>