[outages] [EXTERNAL] Re: FAA.gov nameserver outage

[Blue, John W (FAA)]

Initial looks from the firewall team point to an automatic failover event and the secondary failed.

John W Blue
Infrastructure Services - AIF-330
d: 405-954-4177
c: 405-443-8445
john.w.blue at faa.gov<mailto:john.w.blue at faa.gov>

From: virendra rode <virendra.rode at outages.org>
Sent: Saturday, March 25, 2023 11:05 PM
To: Chapman, Brad (NBCUniversal) <Brad.Chapman at nbcuni.com>; Michael B. Williams <Michael.Williams at glexia.com>
Cc: outages <outages at outages.org>; Blue, John W (FAA) <john.w.blue at faa.gov>
Subject: Re: [outages] [EXTERNAL] Re: FAA.gov nameserver outage

+I'm adding John Blue (cc'd) to see if he has any insight into this matter.


/vrode

On Sun, 2023-03-26 at 03:41 +0000, Chapman, Brad (NBCUniversal) wrote:
This screenshot of the FAA status page says otherwise.

[image0.jpeg]

—Sent from my iPhone


On Mar 25, 2023, at 8:29 PM, Michael B. Williams <Michael.Williams at glexia.com<mailto:Michael.Williams at glexia.com>> wrote:


All -

I finally got a hold of someone at FAA and they said they aren’t seeing any issues.

I tried to explain but didn’t get anywhere.

Michael

On Sun, Mar 26, 2023 at 14:21 virendra rode <virendra.rode at outages.org<mailto:virendra.rode at outages.org>> wrote:

On Sat, 2023-03-25 at 20:06 -0700, virendra rode via Outages wrote:
On Sun, 2023-03-26 at 02:00 +0000, Chapman, Brad (NBCUniversal) via Outages wrote:
I don't know how one would contact them, but the list of the senior IT officers are on their memorandum to migrate to IPv6 dated September 2021.  This memo states they would be migrating in 2023.  Perhaps we're seeing evidence of a recent change?

https://www.faa.gov/sites/faa.gov/files/about/office_org/headquarters_offices/afn/transition-ipv6.pdf<https://urldefense.com/v3/__https:/www.faa.gov/sites/faa.gov/files/about/office_org/headquarters_offices/afn/transition-ipv6.pdf__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhWPAxlYWQ$>

—Sent from my iPhone


On Mar 25, 2023, at 6:53 PM, Michael B. Williams via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:

Seeing the same here across all our POPs.

Has anyone notified FAA NOC yet?
---------------
I did notice ipv6 path traversal...but now its totally dead


/vrode
_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
-------------------

More proof, just so I'm not blowing smoke,

traceroute to www.faa.gov<https://urldefense.com/v3/__https:/www.faa.gov__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhWTxg6fRg$> (2600:1406:6c00:192::1256), 30 hops max, 80 byte packets
1 2603-8001-3b00-2dc7-0000-0000-0000-0001.res6.spectrum.com<https://urldefense.com/v3/__http:/2603-8001-3b00-2dc7-0000-0000-0000-0001.res6.spectrum.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhVNye0yxA$> (2603:8001:3b00:2dc7::1) 15.618 ms 15.521 ms 15.473 ms
2 2603-90c5-0001-0927-0000-0000-0000-0001.inf6.spectrum.com<https://urldefense.com/v3/__http:/2603-90c5-0001-0927-0000-0000-0000-0001.inf6.spectrum.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhVOufjceA$> (2603:90c5:1:927::1) 31.726 ms 19.029 ms 20.498 ms
3 lag-61.snaucaoszh1.netops.charter.com<https://urldefense.com/v3/__http:/lag-61.snaucaoszh1.netops.charter.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhXcPtSajw$> (2605:e000:0:4::7:37d) 21.835 ms 31.496 ms 31.462 ms
4 lag-22.cyprcabw02r.netops.charter.com<https://urldefense.com/v3/__http:/lag-22.cyprcabw02r.netops.charter.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhUfU3ZmIQ$> (2605:e000:0:8::6:e) 31.500 ms 31.464 ms 38.891 ms
5 * * *
6 * * *
7 lag-1.pr2.lax10.netops.charter.com<https://urldefense.com/v3/__http:/lag-1.pr2.lax10.netops.charter.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhUMcgbOHg$> (2001:1998:0:4::559) 25.888 ms 24.790 ms lag-2.pr2.lax10.netops.charter.com<https://urldefense.com/v3/__http:/lag-2.pr2.lax10.netops.charter.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhWIIaxxoQ$> (2001:1998:56::27) 32.321 ms
8 2001:1998:0:8::667 (2001:1998:0:8::667) 24.557 ms 32.133 ms 32.091 ms
9 * * ae2.ctl-lax8.netarch.akamai.com<https://urldefense.com/v3/__http:/ae2.ctl-lax8.netarch.akamai.com__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhW5tAaIhg$> (2600:1488:a080:114::b) 49.474 ms


/vrode



https://puck.nether.net/mailman/listinfo/outages<https://urldefense.com/v3/__https:/puck.nether.net/mailman/listinfo/outages__;!!PIZeeW5wscynRQ!qznGJHQ7J3qeiuEgKIkWMakaGx4TN4Cq8BxLL-K0TVQAYbHmNgMKD1DffUYQu7QG6zFyNhv4Fob9UnWNVe0_RhXeGYiCdw$>



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20230326/363a2813/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 164574 bytes
Desc: image001.jpg
URL: <https://puck.nether.net/pipermail/outages/attachments/20230326/363a2813/attachment-0001.jpg>