[outages] Facebook is down

[Frank Bulk]

Appears to be inside of fb's network - prn1 = Prineville and San Francisco =
frc1.   Here's some non-working traces:

 

7  sl-st31-chi-bu-20.v6.sprintlink.net (2600:0:2:1239:144:232:1:104)  16.873
ms  16.069 ms  15.925 ms

8  2600:4:2000::f (2600:4:2000::f)  15.715 ms  15.688 ms  15.682 ms

9  ae1.bb02.ord1.tfbnw.net (2620:0:1cff:dead:beee::452)  16.016 ms  15.897
ms  15.957 ms

10  ae12.bb02.atl1.tfbnw.net (2620:0:1cff:dead:beef::1336)  36.603 ms
36.825 ms  36.497 ms

11  ae7.bb02.frc1.tfbnw.net (2620:0:1cff:dead:beef::ce)  40.452 ms  40.345
ms  40.291 ms

12  ae2.dr03.frc1.tfbnw.net (2620:0:1cff:dead:beef::1d6)  40.156 ms  40.093
ms  40.147 ms

13  po1022.csw03c.frc1.tfbnw.net (2620:0:1cff:dead:beef::1631)  40.661 ms
48.299 ms  40.817 ms

14  * * *

 

7  sl-st31-chi-bu-20.v6.sprintlink.net (2600:0:2:1239:144:232:1:104)  15.932
ms  15.758 ms  15.881 ms

8  2600:4:2000::f (2600:4:2000::f)  15.424 ms  15.469 ms  15.494 ms

9  ae1.bb01.ord1.tfbnw.net (2620:0:1cff:dead:beee::450)  15.719 ms  15.754
ms  15.674 ms

10  ae16.bb05.prn1.tfbnw.net (2620:0:1cff:dead:beee::591)  68.011 ms  67.868
ms  67.854 ms

11  ae33.dr06.prn1.tfbnw.net (2620:0:1cff:dead:beee::236)  63.434 ms  63.440
ms  63.383 ms

12  po1021.csw08a.prn1.tfbnw.net (2620:0:1cff:dead:beee::14d)  66.504 ms
64.966 ms  64.429 ms

13  * * *

 

7  sl-st31-chi-bu-20.v6.sprintlink.net (2600:0:2:1239:144:232:1:104)  16.297
ms  16.055 ms  15.914 ms

8  2600:4:2000::f (2600:4:2000::f)  15.614 ms  15.596 ms  15.567 ms

9  ae1.bb01.ord1.tfbnw.net (2620:0:1cff:dead:beee::450)  15.871 ms  15.798
ms  15.695 ms

10  ae22.bb01.prn1.tfbnw.net (2620:0:1cff:dead:beee::593)  68.502 ms  68.354
ms  68.531 ms

11  ae0.dr05.prn1.tfbnw.net (2620:0:1cff:dead:beef::33)  63.433 ms  63.487
ms  63.491 ms

12  po1021.csw08d.prn1.tfbnw.net (2620:0:1cff:dead:beee::149)  72.100 ms
65.004 ms  63.671 ms

13  * * *

 

And a working trace that appears to go through Ashburn:

# tcptraceroute6 www.facebook.com

traceroute to edge-star6-ecmp-02-ash3.facebook.com
(2a03:2880:2030:2f01:face:b00c:0:8) from 2607:fe28:0:1000::5, port 80, from
port 59526, 30 hops max, 60 bytes packets

<snip>

8  sl-crs2-chi-te1-2-0-1.v6.sprintlink.net (2600:4::6)  16.147 ms  16.101 ms
16.032 ms

9  sl-st31-chi-bu-20.v6.sprintlink.net (2600:0:2:1239:144:232:1:104)  17.147
ms  16.948 ms  17.116 ms

10  2600:4:2000::f (2600:4:2000::f)  16.588 ms  35.301 ms  16.540 ms

11  ae1.bb01.ord1.tfbnw.net (2620:0:1cff:dead:beee::450)  59.335 ms  16.766
ms  16.871 ms

12  ae27.bb01.iad1.tfbnw.net (2620:0:1cff:dead:beef::13df)  36.157 ms
40.243 ms  35.998 ms

13  ae0.dr02.ash3.tfbnw.net (2620:0:1cff:dead:beef::2f1)  35.736 ms  35.771
ms  35.755 ms

14  po1020.csw02b.ash3.tfbnw.net (2620:0:1cff:dead:beef::1605)  36.800 ms
40.665 ms  36.404 ms

15  * * *

16  edge-star6-ecmp-02-ash3.facebook.com (2a03:2880:2030:2f01:face:b00c:0:8)
36.043 ms [open]  36.122 ms  36.060 ms

#

 

Frank

 

From: Frank Bulk [mailto:frnkblk at iname.com] 
Sent: Monday, January 28, 2013 7:33 PM
To: 'Josh Luthman'; outages at outages.org
Subject: RE: [outages] Facebook is down

 

I monitor a couple of their IPv6 addresses and
2a03:2880:10:8f01:face:b00c:0:25 and 2a03:2880:10:8f02:face:b00c:0:25 had
HTTP issues at 5:03 pm Central and also had a blip at 4:04 pm Central, and
the hosts 2a03:2880:2110:3f01:face:b00c::, 2a03:2880:10:8f01:face:b00c:0:25,
and 2a03:2880:10:8f02:face:b00c:0:25 have been down since about 6:56 pm
Central.

 

But the AAAA in my local cache for www.facebook.com
<http://www.facebook.com>  resolves to 2a03:2880:2050:1f01:face:b00c:0:9,
and so things work for me.   The joys of GLBDNS (or whatever facebook uses).

 

Frank

 

From: outages-bounces at outages.org <mailto:outages-bounces at outages.org>
[mailto:outages-bounces at outages.org] On Behalf Of Josh Luthman
Sent: Monday, January 28, 2013 7:14 PM
To: outages at outages.org <mailto:outages at outages.org> 
Subject: [outages] Facebook is down

 

http://isup.me/facebook.com




Josh Luthman
Office: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20130128/92138ec3/attachment.htm>