[outages] DNS SERVFAIL for nist.gov

[Matthew Huff]

Of course.

Like I stated in the original email, we don’t use NIST for time sync. We actually have a GPS and a PTP feed.

WE MUST, HOWEVER, VALIDATE OUR SYSTEM TIMES VERSUS THE INTERNET NIST SERVERS VIA FINRA REGULATIONS

Yes, it is stupid
Yes, it isn’t a good idea
But if we don’t want to get fined and prevented from trading, we have to follow FINRA regulations.


Matthew Huff | Director of Technical Operations | OTA Management LLC

Office: 914-460-4039
mhuff at ox.com<mailto:mhuff at ox.com> | www.ox.com<http://www.ox.com>
...........................................................................................................................................

From: Mel Beckman <mel at beckman.org>
Sent: Monday, June 14, 2021 9:40 AM
To: Matthew Huff <mhuff at ox.com>
Cc: outages at outages.org
Subject: Re: [outages] DNS SERVFAIL for nist.gov

You should consider eliminating dependence on Internet-delivered NIST time and switch to GPS-based time servers. The GPS network has its own airborne atomic clocks that use a well-disciplined protocol to synchronize to NIST reference atomic time without transiting the Internet.

According to NIST’s documentation:


“Currently, the GPS system provides time to the general public with uncertainties measured in nanoseconds. With a well-designed receiver system the user can obtain the time to better than 100 ns in a few minutes, and to about +/- 10 ns with a 24 hour average (and a good local clock).”

All sources of error in GPS time propagation total less than one millisecond, well within your 50ms tolerance.

https://www.nist.gov/pml/time-and-frequency-division/time-services/one-way-gps-time-transfer

NIST maintains publicly-accessible logs of al clock differences to provide documented compliance under FINRA clock synchronization rules.

[cid:image001.jpg at 01D76104.DD1B3430]

The log has a one-hour resolution, satisfying the FINRA requirement to verify synchronization “throughout the day”.

IP-based GPS clocks are widely available with low-drift oven-controlled crystal oscillators (OXCO), or even internal cesium-based atomic clocks, for as little as a few thousand dollars. This lets you ride out time signal outages of days or even weeks.

The US DHS recommends discontinuation of unauthenticated Internet-based reference clocks, owing to their vulnerability to IP address spoofing:

https://www.dhs.gov/sites/default/files/publications/GPS-PNT-Best-Practices-Time-Frequency-Sources-Fixed-Locations-508.pdf


-mel via cell


On Jun 14, 2021, at 3:51 AM, Matthew Huff via Outages <outages at outages.org<mailto:outages at outages.org>> wrote:
We have to query and compare against NIST time servers for FINRA compliance This morning I noticed our systems are unable to DNS query the NIST time servers. Neither our local resolvers or google (8.8.8.8) work.

[root at bacall log]# dig @8.8.8.8 time-a-g.nist.gov

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.5 <<>> @8.8.8.8 time-a-g.nist.gov
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 36018
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;time-a-g.nist.gov.             IN      A

;; Query time: 6 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Jun 14 06:27:45 EDT 2021
;; MSG SIZE  rcvd: 46

[root at bacall log]# dig @8.8.8.8 nist.gov in soa

; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.5 <<>> @8.8.8.8 nist.gov in soa
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 17779
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;nist.gov.                      IN      SOA

;; Query time: 5 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Mon Jun 14 06:31:59 EDT 2021
;; MSG SIZE  rcvd: 37

The time servers are documented here: https://tf.nist.gov/tf-cgi/servers.cgi

Using the IP addresses work, it look like the nist.gov domain is offline.

Matthew Huff | Director of Technical Operations | OTA Management LLC

Office: 914-460-4039
mhuff at ox.com<mailto:mhuff at ox.com> | www.ox.com<http://www.ox.com>
.........................................................................................................................................

_______________________________________________
Outages mailing list
Outages at outages.org<mailto:Outages at outages.org>
https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/outages/attachments/20210614/0f9773d6/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 631916 bytes
Desc: image001.jpg
URL: <https://puck.nether.net/pipermail/outages/attachments/20210614/0f9773d6/attachment-0001.jpg>