<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
@font-face
{font-family:Tahoma}
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif"}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline}
span.EmailStyle17
{font-family:"Calibri","sans-serif";
color:#1F497D}
span.EmailStyle18
{font-family:"Calibri","sans-serif";
color:#1F497D}
.MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.WordSection1
{}
-->
</style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Work begins NOW.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<div>
<div style="border:none; border-top:solid #B5C4DF 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt; font-family:"Tahoma","sans-serif""> outages-bounces@outages.org [mailto:outages-bounces@outages.org]
<b>On Behalf Of </b>Frank Bulk<br>
<b>Sent:</b> Monday, January 28, 2013 7:09 PM<br>
<b>To:</b> 'Damian Guppy'; Jeremy L. Gaddis<br>
<b>Cc:</b> outages@outages.org<br>
<b>Subject:</b> Re: [outages] Facebook is down</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">I saw the same thing here after flushing one DNS server’s facebook.com entry. I’ve now flushed that entry on all our caching DNS resolvers so that our end
users will get the new info (caveat: local and application DNS caching).</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Anyone have a historical list of facebook.com’s A records to see if it’s always been just 173.252.100.16?</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D">Frank</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"; color:#1F497D"> </span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt; font-family:"Calibri","sans-serif""> outages-bounces@outages.org [mailto:outages-bounces@outages.org]
<b>On Behalf Of </b>Damian Guppy<br>
<b>Sent:</b> Monday, January 28, 2013 8:50 PM<br>
<b>To:</b> Jeremy L. Gaddis<br>
<b>Cc:</b> outages@outages.org<br>
<b>Subject:</b> Re: [outages] Facebook is down</span></p>
<p class="MsoNormal"> </p>
<div>
<p class="MsoNormal">Port 80 on <a href="http://facebook.com">facebook.com</a> is now working for me, redirecting to
<a href="http://www.facebook.com">www.facebook.com</a> as is normal operation. Looks like they have updated their DNS zone as i now get the following response (before i only got a single IPv4 address)</p>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<div>
<p class="MsoNormal">Name: <a href="http://facebook.com">facebook.com</a></p>
</div>
<div>
<p class="MsoNormal">Addresses: 2a03:2880:10:8f01:face:b00c:0:1</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:10:6f01:face:b00c:0:8</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:2030:2f01:face:b00c:0:8</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:10:cf01:face:b00c:0:5</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:10:cf02:face:b00c:0:5</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:2110:cf01:face:b00c:0:9</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:10:1f02:face:b00c:0:8</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:10:8f02:face:b00c:0:1</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:2040:1f01:face:b00c:0:8</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:2110:df01:face:b00c:0:8</p>
</div>
<div>
<p class="MsoNormal"> 2a03:2880:2050:1f01:face:b00c:0:9</p>
</div>
<div>
<p class="MsoNormal"> 69.171.242.27</p>
</div>
<div>
<p class="MsoNormal"> 69.171.228.24</p>
</div>
<div>
<p class="MsoNormal"> 69.171.234.18</p>
</div>
<div>
<p class="MsoNormal"> 173.252.110.27</p>
</div>
<div>
<p class="MsoNormal"> 173.252.101.26</p>
</div>
<div>
<p class="MsoNormal"> 173.252.100.27</p>
</div>
<div>
<p class="MsoNormal"> 69.171.237.24</p>
</div>
<div>
<p class="MsoNormal"> 69.171.237.40</p>
</div>
<div>
<p class="MsoNormal"> 66.220.152.19</p>
</div>
<div>
<p class="MsoNormal"> 69.171.224.42</p>
</div>
<div>
<p class="MsoNormal"> 69.171.234.34</p>
</div>
<div>
<p class="MsoNormal"> 66.220.158.27</p>
</div>
<div>
<p class="MsoNormal"> 69.171.229.25</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">--Damian</p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> </p>
<div>
<p class="MsoNormal">On Tue, Jan 29, 2013 at 10:37 AM, Jeremy L. Gaddis <<a href="mailto:jeremy@evilrouters.net" target="_blank">jeremy@evilrouters.net</a>> wrote:</p>
<blockquote style="border:none; border-left:solid #CCCCCC 1.0pt; padding:0in 0in 0in 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0in; margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">* Josh Luthman <<a href="mailto:josh@imaginenetworksllc.com">josh@imaginenetworksllc.com</a>> wrote:<br>
> <a href="http://isup.me/facebook.com" target="_blank">http://isup.me/facebook.com</a></p>
</div>
<p class="MsoNormal">Probably no useful details here that haven't already been posted, but<br>
what the hell...<br>
<br>
>From Chicago (Level3):<br>
<br>
IPv4:<br>
<br>
<a href="http://facebook.com" target="_blank">facebook.com</a> - times out (173.252.100.16) - trace below<br>
<a href="http://www.facebook.com" target="_blank">www.facebook.com</a> - OK (CNAME
<a href="http://star.c10r.facebook.com/173.252.100.27" target="_blank">star.c10r.facebook.com/173.252.100.27</a>)<br>
<br>
$ mtr -4rwnc3 173.252.100.16<br>
HOST: bofh Loss% Snt Last Avg Best Wrst StDev<br>
1. 8.30.239.145 0.0% 3 1.4 2.2 1.4 2.7 0.7<br>
2. 8.30.239.161 0.0% 3 5.5 9.5 5.5 17.6 7.0<br>
3. 4.30.13.41 0.0% 3 10.7 16.3 10.6 27.7 9.8<br>
4. 4.69.158.50 0.0% 3 34.7 29.1 26.2 34.7 4.8<br>
5. 4.69.148.145 0.0% 3 26.2 26.3 26.2 26.5 0.1<br>
6. 4.69.143.221 0.0% 3 26.4 26.4 26.3 26.4 0.1<br>
7. 4.69.134.150 0.0% 3 32.2 31.3 27.4 34.2 3.5<br>
8. 4.69.149.82 0.0% 3 26.5 26.3 26.2 26.5 0.2<br>
9. 4.53.116.62 0.0% 3 26.3 26.2 26.2 26.3 0.1<br>
10. 204.15.20.54 0.0% 3 27.0 26.6 26.4 27.0 0.3<br>
11. 31.13.24.24 0.0% 3 37.2 44.4 37.2 56.8 10.8<br>
12. 31.13.24.15 0.0% 3 36.3 36.4 36.3 36.5 0.1<br>
13. ??? 100.0 3 0.0 0.0 0.0 0.0 0.0<br>
<br>
>From Los Angeles (One Wilshire):<br>
<br>
IPv4:<br>
<br>
<a href="http://facebook.com" target="_blank">facebook.com</a> - times out (173.252.100.16) - trace below<br>
<a href="http://www.facebook.com" target="_blank">www.facebook.com</a> - OK (CNAME
<a href="http://star.c10r.facebook.com/69.171.229.25" target="_blank">star.c10r.facebook.com/69.171.229.25</a>)<br>
<br>
IPv6:<br>
<br>
<a href="http://facebook.com" target="_blank">facebook.com</a> - OK (2a03:2880:2040:1f01:face:b00c:0:8)<br>
<a href="http://www.facebook.com" target="_blank">www.facebook.com</a> - OK (2a03:2880:2030:2f01:face:b00c:0:8)<br>
<br>
$ mtr -4rwnc3 173.252.100.16<br>
HOST: deep-thought Loss% Snt Last Avg Best Wrst StDev<br>
1. 206.125.175.17 0.0% 3 6.3 4.4 0.8 6.3 3.1<br>
2. 206.223.143.161 0.0% 3 44.3 25.7 0.5 44.3 22.7<br>
3. 31.13.30.26 0.0% 3 0.7 0.8 0.7 1.1 0.3<br>
4. 31.13.28.111 0.0% 3 50.3 50.6 50.3 50.9 0.3<br>
5. 31.13.27.116 0.0% 3 54.6 55.0 54.5 55.8 0.7<br>
6. 31.13.27.73 0.0% 3 54.3 54.2 54.1 54.3 0.1<br>
7. ??? 100.0 3 0.0 0.0 0.0 0.0 0.0<br>
<br>
(hop #2 == <a href="http://facebook.com.any2ix.coresite.com" target="_blank">facebook.com.any2ix.coresite.com</a>)<br>
<span style="color:#888888"><br>
--<br>
Jeremy L. Gaddis e: <a href="mailto:jeremy@evilrouters.net">jeremy@evilrouters.net</a><br>
Network Engineer m: <a href="tel:%2B1.812.865.0581">+1.812.865.0581</a> (Cell)<br>
PGP: 0x95E2C8FE w: <a href="http://evilrouters.net" target="_blank">http://evilrouters.net</a></span></p>
<div>
<div>
<p class="MsoNormal"><br>
<br>
_______________________________________________<br>
Outages mailing list<br>
<a href="mailto:Outages@outages.org">Outages@outages.org</a><br>
<a href="https://puck.nether.net/mailman/listinfo/outages" target="_blank">https://puck.nether.net/mailman/listinfo/outages</a></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
NOTICE: The information contained in this communication (including attachments) may be confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please re-send this communication to the sender and delete the original
message and any copy of it from your computer system.<br>
</font>
</body>
</html>