[outages] Facebook is down

Mon Jan 28 22:31:28 EST 2013

Work begins NOW.

From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of Frank Bulk
Sent: Monday, January 28, 2013 7:09 PM
To: 'Damian Guppy'; Jeremy L. Gaddis
Cc: outages at outages.org
Subject: Re: [outages] Facebook is down

I saw the same thing here after flushing one DNS server's facebook.com entry.   I've now flushed that entry on all our caching DNS resolvers so that our end users will get the new info (caveat: local and application DNS caching).

Anyone have a historical list of facebook.com's A records to see if it's always been just 173.252.100.16?

Frank

From: outages-bounces at outages.org [mailto:outages-bounces at outages.org] On Behalf Of Damian Guppy
Sent: Monday, January 28, 2013 8:50 PM
To: Jeremy L. Gaddis
Cc: outages at outages.org
Subject: Re: [outages] Facebook is down

Port 80 on facebook.com<http://facebook.com> is now working for me, redirecting to www.facebook.com<http://www.facebook.com> as is normal operation. Looks like they have updated their DNS zone as i now get the following response (before i only got a single IPv4 address)

Name:    facebook.com<http://facebook.com>
Addresses:  2a03:2880:10:8f01:face:b00c:0:1
          2a03:2880:10:6f01:face:b00c:0:8
          2a03:2880:2030:2f01:face:b00c:0:8
          2a03:2880:10:cf01:face:b00c:0:5
          2a03:2880:10:cf02:face:b00c:0:5
          2a03:2880:2110:cf01:face:b00c:0:9
          2a03:2880:10:1f02:face:b00c:0:8
          2a03:2880:10:8f02:face:b00c:0:1
          2a03:2880:2040:1f01:face:b00c:0:8
          2a03:2880:2110:df01:face:b00c:0:8
          2a03:2880:2050:1f01:face:b00c:0:9
          69.171.242.27
          69.171.228.24
          69.171.234.18
          173.252.110.27
          173.252.101.26
          173.252.100.27
          69.171.237.24
          69.171.237.40
          66.220.152.19
          69.171.224.42
          69.171.234.34
          66.220.158.27
          69.171.229.25

--Damian

On Tue, Jan 29, 2013 at 10:37 AM, Jeremy L. Gaddis <jeremy at evilrouters.net<mailto:jeremy at evilrouters.net>> wrote:
* Josh Luthman <josh at imaginenetworksllc.com<mailto:josh at imaginenetworksllc.com>> wrote:
> http://isup.me/facebook.com
Probably no useful details here that haven't already been posted, but
what the hell...

>From Chicago (Level3):

IPv4:

  facebook.com<http://facebook.com> - times out (173.252.100.16) - trace below
  www.facebook.com<http://www.facebook.com> - OK (CNAME star.c10r.facebook.com/173.252.100.27<http://star.c10r.facebook.com/173.252.100.27>)

$ mtr -4rwnc3 173.252.100.16
HOST: bofh         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. 8.30.239.145  0.0%     3    1.4   2.2   1.4   2.7   0.7
  2. 8.30.239.161  0.0%     3    5.5   9.5   5.5  17.6   7.0
  3. 4.30.13.41    0.0%     3   10.7  16.3  10.6  27.7   9.8
  4. 4.69.158.50   0.0%     3   34.7  29.1  26.2  34.7   4.8
  5. 4.69.148.145  0.0%     3   26.2  26.3  26.2  26.5   0.1
  6. 4.69.143.221  0.0%     3   26.4  26.4  26.3  26.4   0.1
  7. 4.69.134.150  0.0%     3   32.2  31.3  27.4  34.2   3.5
  8. 4.69.149.82   0.0%     3   26.5  26.3  26.2  26.5   0.2
  9. 4.53.116.62   0.0%     3   26.3  26.2  26.2  26.3   0.1
 10. 204.15.20.54  0.0%     3   27.0  26.6  26.4  27.0   0.3
 11. 31.13.24.24   0.0%     3   37.2  44.4  37.2  56.8  10.8
 12. 31.13.24.15   0.0%     3   36.3  36.4  36.3  36.5   0.1
 13. ???          100.0     3    0.0   0.0   0.0   0.0   0.0

>From Los Angeles (One Wilshire):

IPv4:

  facebook.com<http://facebook.com> - times out (173.252.100.16) - trace below
  www.facebook.com<http://www.facebook.com> - OK (CNAME star.c10r.facebook.com/69.171.229.25<http://star.c10r.facebook.com/69.171.229.25>)

IPv6:

  facebook.com<http://facebook.com> - OK (2a03:2880:2040:1f01:face:b00c:0:8)
  www.facebook.com<http://www.facebook.com> - OK (2a03:2880:2030:2f01:face:b00c:0:8)

$ mtr -4rwnc3 173.252.100.16
HOST: deep-thought    Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. 206.125.175.17   0.0%     3    6.3   4.4   0.8   6.3   3.1
  2. 206.223.143.161  0.0%     3   44.3  25.7   0.5  44.3  22.7
  3. 31.13.30.26      0.0%     3    0.7   0.8   0.7   1.1   0.3
  4. 31.13.28.111     0.0%     3   50.3  50.6  50.3  50.9   0.3
  5. 31.13.27.116     0.0%     3   54.6  55.0  54.5  55.8   0.7
  6. 31.13.27.73      0.0%     3   54.3  54.2  54.1  54.3   0.1
  7. ???             100.0     3    0.0   0.0   0.0   0.0   0.0

(hop #2 == facebook.com.any2ix.coresite.com<http://facebook.com.any2ix.coresite.com>)

--
Jeremy L. Gaddis        e: jeremy at evilrouters.net<mailto:jeremy at evilrouters.net>
Network Engineer        m: +1.812.865.0581<tel:%2B1.812.865.0581> (Cell)
PGP:  0x95E2C8FE        w: http://evilrouters.net

_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages

________________________________

NOTICE: The information contained in this communication (including attachments) may be confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please re-send this communication to the sender and delete the original message and any copy of it from your computer system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20130129/4a206a99/attachment.htm>