<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<font face="monospace">Longtime ntpsec user, run my own raspi
timeserver, works just dandy. <br>
<br>
Except somewhere along the path I apparently strayed, and I'm
flummoxed. My ntp.log shows no errors related to NTS service.
Everything functions seemingly fine - except that the NTS servers
show no traffic from my peers. Cookies arrive, and gradually fade
away. 'Reach' is always zero in ntpq. I've tested my firewall from
outside, port 4460 TCP is open, and a telnet connects. My
certificates are fine, and NTS receives the peer certs fine.<br>
<br>
For a time I thought it was my 'restrict' lines, but NTP peering
is perfectly normal. If other eyes can see where the failure is,
it would be greatly appreciated, I've spent much of the day
futzing with this to no avail. <br>
<br>
NTS adoption doesn't seem to be really taking hold, but I'd still
like to offer it. Diagnostics below. <br>
<br>
root@ 64bit A-NTPsec: ~ # cat /etc/ntp.conf|grep -v ^$|grep -v
"^#"<br>
refclock shm unit 1 refid PPS minpoll 2 maxpoll 3 flag4 1
prefer<br>
server ntp.silvertree.org minpoll 6 maxpoll 10<br>
server clock.fmt.he.net minpoll 6 maxpoll 10<br>
server clock.sjc.he.net minpoll 6 maxpoll 10<br>
server ntp1.net.berkeley.edu minpoll 6 maxpoll 10 <br>
refclock shm unit 0 refid GPS minpoll 3 maxpoll 5 flag4 1<br>
restrict default kod limited nomodify<br>
restrict 0.0.0.0/0 kod limited nomodify<br>
restrict 127.0.0.1<br>
restrict ::1<br>
restrict 192.168.1.0/24<br>
leapfile /var/lib/ntp/leap-seconds.list<br>
logfile /var/log/ntp.log<br>
driftfile /var/lib/ntp/ntp.drift<br>
statsdir /var/log/ntpstats/<br>
statistics loopstats peerstats protostats clockstats<br>
filegen loopstats file loopstats type day enable<br>
filegen peerstats file peerstats type day enable<br>
filegen protostats file protostats type day enable<br>
filegen clockstats file clockstats type day enable<br>
filegen ntsstats file ntsstats type day enable<br>
filegen ntskestats file ntskestats type day enable<br>
nts enable<br>
nts key /etc/letsencrypt/live/nts.anastrophe.com/privkey.pem<br>
nts cert /etc/letsencrypt/live/nts.anastrophe.com/fullchain.pem<br>
nts cookie /var/lib/ntp/nts-keys<br>
server time.cloudflare.com nts<br>
server oregon.time.system76.com nts<br>
server time.cifelli.xyz nts<br>
server time.txryan.com nts<br>
<br>
--------------------------------------------------------------------------------------------------------<br>
<br>
root@ 64bit A-NTPsec: ~ # ntpq -pu<br>
remote refid st t when
poll reach delay offset jitter<br>
=======================================================================================================<br>
*SHM(1) .PPS. 0 l
3 8 377 0ns 474ns 967ns<br>
-ntp.silvertree.org .GPS. 1 u
60 64 377 24.561ms -942.7us 2.5457ms<br>
+clock.fmt.he.net 66.220.9.122 2 u
52 64 377 16.159ms 2.3941ms 3.2085ms<br>
-clock.sjc.he.net 66.220.9.122 2 u
54 64 377 14.788ms 2.6938ms 1.7487ms<br>
+ntp1.net.berkeley.edu .GPS. 1 u
53 64 377 17.434ms 1.5008ms 1.4526ms<br>
-SHM(0) .GPS. 0 l
2 8 377 0ns -134.0ms 339.76us<br>
time.cloudflare.com .NTS4. 16 5
- 64 0 0ns 0ns 1.907us<br>
oregon.time.system76.com .NTS4. 16 7
- 64 0 0ns 0ns 1.907us<br>
time.cifelli.xyz .NTS4. 16 7
- 64 0 0ns 0ns 1.907us<br>
time.txryan.com .NTS4. 16 7
- 64 0 0ns 0ns 1.907us<br>
<br>
</font><font face="monospace">--------------------------------------------------------------------------------------------------------<br>
<br>
From offsite:<br>
root@relay: ~ # ntpq -pu<br>
remote refid st t when
poll reach delay offset jitter<br>
=======================================================================================================<br>
*aws.ntp 169.254.169.122 3 u
240 256 377 168.51us 1.2749ms 195.54us<br>
+ntpsec.anastrophe.com .PPS. 1 u
190 256 377 35.264ms -175.4us 1.5759ms<br>
+clock.fmt.he.net 66.220.9.122 2 u
252 256 377 20.874ms 1.3826ms 126.66us<br>
-tick.eoni.com 208.90.144.52 3 u
201 256 377 13.717ms -3.005ms 258.10us<br>
nts.anastrophe.com .NTS. 16 8 -
2048 0 0ns 0ns 119ns<br>
</font><font face="monospace"><br>
<br>
</font><font face="monospace">--------------------------------------------------------------------------------------------------------</font><br>
<font face="monospace"><br>
</font><font face="monospace">root@ 64bit A-NTPsec: ~ # tail -100
/var/log/ntp.log<br>
2024-04-08T16:14:20 ntpd[65005]: INIT: ntpd
ntpsec-1.2.3+24-gb5252d172: Starting<br>
2024-04-08T16:14:20 ntpd[65005]: INIT: Command line:
/usr/local/sbin/ntpd -p /run/ntpd.pid -g -u ntp:ntp<br>
2024-04-08T16:14:20 ntpd[65005]: CLOCK: leapsecond file
('/var/lib/ntp/leap-seconds.list'): good hash signature<br>
2024-04-08T16:14:20 ntpd[65005]: CLOCK: leapsecond file
('/var/lib/ntp/leap-seconds.list'): loaded,
expire=2024-06-28T00:00Z last=2017-01-01T00:00Z ofs=37<br>
2024-04-08T16:14:20 ntpd[65005]: INIT: Using SO_TIMESTAMPNS(ns)<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen and drop on 0
v6wildcard [::]:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen and drop on 1
v4wildcard 0.0.0.0:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen normally on 2 lo
127.0.0.1:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen normally on 3 wlan0
192.168.1.10:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen normally on 4 lo
[::1]:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listen normally on 5 wlan0
[fe80::ba27:ebff:fec7:9ea9%3]:123<br>
2024-04-08T16:14:20 ntpd[65005]: IO: Listening on routing socket
on fd #22 for interface updates<br>
2024-04-08T16:14:20 ntpd[65005]: SYNC: Found 10 servers, suggest
minsane at least 3<br>
2024-04-08T16:14:20 ntpd[65005]: INIT: MRU 10922 entries, 13 hash
bits, 65536 bytes<br>
2024-04-08T16:14:20 ntpd[65005]: INIT: OpenSSL 1.1.1w 11 Sep
2023, 1010117f<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: starting NTS-KE server
listening on port 4460<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: OpenSSL security level is 2<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: starting NTS-KE server
listening on port 4460<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: listen4 worked<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: listen6 worked<br>
2024-04-08T16:14:20 ntpd[65005]: NTSc: Using system default root
certificates.<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: loaded certificate (chain)
from /etc/letsencrypt/live/nts.anastrophe.com/fullchain.pem<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: loaded private key from
/etc/letsencrypt/live/nts.anastrophe.com/privkey.pem<br>
2024-04-08T16:14:20 ntpd[65005]: NTSs: Private Key OK<br>
2024-04-08T16:14:20 ntpd[65005]: NTS: Read cookie file, 10 keys.<br>
2024-04-08T16:14:22 ntpd[65005]: DNS: dns_probe:
ntp.silvertree.org, cast_flags:1, flags:20801<br>
2024-04-08T16:14:22 ntpd[65005]: DNS: dns_check: processing
ntp.silvertree.org, 1, 20801<br>
2024-04-08T16:14:22 ntpd[65005]: DNS: Server taking:
173.11.101.155<br>
2024-04-08T16:14:22 ntpd[65005]: DNS: dns_take_status:
ntp.silvertree.org=>good, 0<br>
2024-04-08T16:14:23 ntpd[65005]: DNS: dns_probe: clock.fmt.he.net,
cast_flags:1, flags:20801<br>
2024-04-08T16:14:23 ntpd[65005]: DNS: dns_check: processing
clock.fmt.he.net, 1, 20801<br>
2024-04-08T16:14:23 ntpd[65005]: DNS: Server taking:
216.218.192.202<br>
2024-04-08T16:14:23 ntpd[65005]: DNS: dns_take_status:
clock.fmt.he.net=>good, 0<br>
2024-04-08T16:14:24 ntpd[65005]: DNS: dns_probe: clock.sjc.he.net,
cast_flags:1, flags:20801<br>
2024-04-08T16:14:24 ntpd[65005]: DNS: dns_check: processing
clock.sjc.he.net, 1, 20801<br>
2024-04-08T16:14:24 ntpd[65005]: DNS: Server taking:
216.218.254.202<br>
2024-04-08T16:14:24 ntpd[65005]: DNS: dns_take_status:
clock.sjc.he.net=>good, 0<br>
2024-04-08T16:14:25 ntpd[65005]: DNS: dns_probe:
ntp1.net.berkeley.edu, cast_flags:1, flags:20801<br>
2024-04-08T16:14:25 ntpd[65005]: DNS: dns_check: processing
ntp1.net.berkeley.edu, 1, 20801<br>
2024-04-08T16:14:25 ntpd[65005]: DNS: Server taking:
169.229.128.134<br>
2024-04-08T16:14:25 ntpd[65005]: DNS: dns_take_status:
ntp1.net.berkeley.edu=>good, 0<br>
2024-04-08T16:14:27 ntpd[65005]: DNS: dns_probe:
time.cloudflare.com, cast_flags:1, flags:21801<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: DNS lookup of
time.cloudflare.com took 0.001 sec<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: connecting to
time.cloudflare.com:4460 => 162.159.200.1:4460<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: set cert host:
time.cloudflare.com<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: Using TLSv1.3,
TLS_AES_256_GCM_SHA384 (256)<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: certificate subject name:
/CN=time.cloudflare.com<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: certificate issuer name:
/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=GeoTrust TLS ECC CA G1<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: SAN:DNS time.cloudflare.com<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: certificate is valid.<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: Good ALPN from
time.cloudflare.com<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: read 750 bytes<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: Using port 123<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: Got 7 cookies, length 100,
aead=15.<br>
2024-04-08T16:14:27 ntpd[65005]: NTSc: NTS-KE req to
time.cloudflare.com took 0.189 sec, OK<br>
2024-04-08T16:14:27 ntpd[65005]: DNS: dns_check: processing
time.cloudflare.com, 1, 21801<br>
2024-04-08T16:14:27 ntpd[65005]: DNS: Server taking: 162.159.200.1<br>
2024-04-08T16:14:27 ntpd[65005]: DNS: dns_take_status:
time.cloudflare.com=>good, 0<br>
2024-04-08T16:14:28 ntpd[65005]: DNS: dns_probe:
oregon.time.system76.com, cast_flags:1, flags:21801<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: DNS lookup of
oregon.time.system76.com took 0.000 sec<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: connecting to
oregon.time.system76.com:4460 => 52.10.183.132:4460<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: set cert host:
oregon.time.system76.com<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: Using TLSv1.3,
TLS_AES_256_GCM_SHA384 (256)<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: certificate subject name:
/CN=oregon.time.system76.com<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: certificate issuer name:
/C=US/O=Let's Encrypt/CN=R3<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: SAN:DNS
oregon.time.system76.com<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: certificate is valid.<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: Good ALPN from
oregon.time.system76.com<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: read 848 bytes<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: Got 8 cookies, length 100,
aead=15.<br>
2024-04-08T16:14:28 ntpd[65005]: NTSc: NTS-KE req to
oregon.time.system76.com took 0.198 sec, OK<br>
2024-04-08T16:14:28 ntpd[65005]: DNS: dns_check: processing
oregon.time.system76.com, 1, 21801<br>
2024-04-08T16:14:28 ntpd[65005]: DNS: Server taking: 52.10.183.132<br>
2024-04-08T16:14:28 ntpd[65005]: DNS: dns_take_status:
oregon.time.system76.com=>good, 0<br>
2024-04-08T16:14:29 ntpd[65005]: DNS: dns_probe: time.cifelli.xyz,
cast_flags:1, flags:21801<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: DNS lookup of
time.cifelli.xyz took 0.000 sec<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: connecting to
time.cifelli.xyz:4460 => 50.116.42.84:4460<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: set cert host:
time.cifelli.xyz<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: Using TLSv1.3,
TLS_AES_256_GCM_SHA384 (256)<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: certificate subject name:
/CN=time.cifelli.xyz<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: certificate issuer name:
/C=US/O=Let's Encrypt/CN=R3<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: SAN:DNS time.cifelli.xyz<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: certificate is valid.<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: Good ALPN from
time.cifelli.xyz<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: read 816 bytes<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: Got 8 cookies, length 96,
aead=15.<br>
2024-04-08T16:14:29 ntpd[65005]: NTSc: NTS-KE req to
time.cifelli.xyz took 0.387 sec, OK<br>
2024-04-08T16:14:29 ntpd[65005]: DNS: dns_check: processing
time.cifelli.xyz, 1, 21801<br>
2024-04-08T16:14:29 ntpd[65005]: DNS: Server taking: 50.116.42.84<br>
2024-04-08T16:14:29 ntpd[65005]: DNS: dns_take_status:
time.cifelli.xyz=>good, 0<br>
2024-04-08T16:14:30 ntpd[65005]: DNS: dns_probe: time.txryan.com,
cast_flags:1, flags:21801<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: DNS lookup of
time.txryan.com took 0.000 sec<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: connecting to
time.txryan.com:4460 => 5.161.184.148:4460<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: set cert host:
time.txryan.com<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: Using TLSv1.3,
TLS_AES_256_GCM_SHA384 (256)<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: certificate subject name:
/CN=time.txryan.com<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: certificate issuer name:
/C=US/O=Let's Encrypt/CN=R3<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: SAN:DNS time.txryan.com<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: certificate is valid.<br>
2024-04-08T16:14:30 ntpd[65005]: NTSc: Good ALPN from
time.txryan.com<br>
2024-04-08T16:14:31 ntpd[65005]: NTSc: read 816 bytes<br>
2024-04-08T16:14:31 ntpd[65005]: NTSc: Got 8 cookies, length 96,
aead=15.<br>
2024-04-08T16:14:31 ntpd[65005]: NTSc: NTS-KE req to
time.txryan.com took 0.429 sec, OK<br>
2024-04-08T16:14:31 ntpd[65005]: DNS: dns_check: processing
time.txryan.com, 1, 21801<br>
2024-04-08T16:14:31 ntpd[65005]: DNS: Server taking: 5.161.184.148<br>
2024-04-08T16:14:31 ntpd[65005]: DNS: dns_take_status:
time.txryan.com=>good, 0<br>
2024-04-08T16:19:53 ntpd[65005]: NTSs: NTS-KE from
35.208.122.240:48376, OK, Using TLSv1.3, TLS_AES_256_GCM_SHA384
(256), took 0.239 sec, CPU: 32.438+0.000 ms<br>
2024-04-08T16:19:54 ntpd[65005]: NTSs: NTS-KE from
107.174.212.113:39628, OK, Using TLSv1.3, TLS_AES_256_GCM_SHA384
(256), took 1.117 sec, CPU: 26.350+0.000 ms<br>
<br>
</font><font face="monospace">--------------------------------------------------------------------------------------------------------<br>
<br>
</font><font face="monospace">root@ 64bit A-NTPsec: ~ # ntpq -c nts<br>
NTS client sends: 48<br>
NTS client recvs good: 0<br>
NTS client recvs w error: 0<br>
NTS server recvs good: 1<br>
NTS server recvs w error: 0<br>
NTS server sends: 1<br>
NTS make cookies: 17<br>
NTS cookies not server: 0<br>
NTS decode cookies total: 1<br>
NTS decode cookies current: 1<br>
NTS decode cookies old: 0<br>
NTS decode cookies old2: 0<br>
NTS decode cookies older: 0<br>
NTS decode cookies too old: 0<br>
NTS decode cookies error: 0<br>
NTS KE serves good: 2<br>
NTS KE serves good wall: 1.356<br>
NTS KE serves good CPU: 0.059<br>
NTS KE serves no-TLS: 0<br>
NTS KE serves no-TLS wall: 0.0<br>
NTS KE serves no-TLS CPU: 0.0<br>
NTS KE serves bad: 0<br>
NTS KE serves bad wall: 0.0<br>
NTS KE serves bad CPU: 0.0<br>
NTS KE client probes good: 8<br>
NTS KE client probes bad: 0<br>
</font>
<pre class="moz-signature" cols="74">--
Paul Theodoropoulos
<a class="moz-txt-link-abbreviated" href="http://www.anastrophe.com">www.anastrophe.com</a></pre>
</body>
</html>