Configure a Stratum 1 NTP time server with Chrony and gpsd on Manjaro

· 8 min read

Background

A GPS satellite contains multiple atomic clocks, so it is considered a “stratum zero” time source. Any equipment that gets its time from GPS satellites is therefore “stratum 1” time sourcej. And gear that gets time from a “stratum 1” time source (for example, networked servers) are “stratum 2”. And so on.

At each level there is redundancy: A single satellite has several clocks. A single GPS receiver listens to several satellites. A single stratum 2 device listens to several stratum 1 devices. And so, we get some semblance of accurate time on our devices.

And that’s important, because computer clocks are notoriously bad at keeping time. They drift, often by a few seconds a day or more. And if we don’t all know exactly what time it is, then we barely have a functioning civilization.

On this page I’ve documented the setup for the NTP server in my homelab.

I’m using the following hardware:

And the following software:

Research Notes

Helpful Refs

Common NTP metrics:

Transcript of server setup

The basics

I started by flashing Manjaro Raspberry Pi 4 minimal image and running through the installer, which was really simple.

Then, once it rebooted I signed in with SSH. Once on the device, I updated and installed needed packages:

sudo pacman -Syu
sudo pacman -Syu vim ack ethtool git jq pps-tools chrony gpsd

Weirdly, I noticed that I noticed that localhost doesn’t resolve, so I had to add an /etc/hosts entry for it:

sudo echo "127.0.0.1 localhost" >> /etc/hosts

Hardware & kernel modules

Now I’m on to the gps setup. First, we need the right hardware config and kernel modules:

cat <<EOF | sudo tee -a /boot/config.txt
# disable bluetooth & wifi
dtoverlay=disable-bt
dtoverlay=disable-wifi

# Get 1PPS from HAT pin
dtoverlay=pps-gpio,gpiopin=18
EOF

sudo vim /boot/cmdline.txt
# remove console=serial0,115200 and kgdboc=serial0,115200
# Turn off power saving because it causes the system clock to skew even more:
# add nohz=off

Also, there are some services that we need to turn off, because they conflict with resources we’re trying to use:

sudo systemctl disable attach-bluetooth.service
sudo systemctl mask serial-getty@ttyS0.service
sudo systemctl mask serial-getty@ttyAMA0.service
sudo systemctl stop systemd-timesyncd.service
sudo systemctl disable systemd-timesyncd.service

sudo reboot

Now, the Uputronics board connects in two ways to the Pi. It uses a serial port for the GPS signal, and it uses a GPIO pin for the PPS signal. The GPS signal provides complete time information along with a lot of data about the quality of signal, satellite connections, etc. The PPS (pulse per second) signal is a very accurate, very plain electronic puls on a GPIO pin that marks when each second starts.

GPS is the clock, and PPS is used to discipline the clock.

After reboot, PPS is working right away:

$ sudo dmesg | grep pps
[    4.653434] pps_core: LinuxPPS API ver. 1 registered
[    4.653446] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    4.664103] pps pps0: new PPS source pps@12.-1
[    4.664160] pps pps0: Registered IRQ 49 as PPS source

And ppstest /dev/pps0 gave me some good output.

The GPS serial device was found, too:

$ sudo dmesg | grep AMA
[    2.141700] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 21, base_baud = 0) is a PL011 rev2

The basic hardware is working. Now let’s compile gpsctl and configure the Uputronics board:

pacman -Syu gcc make glibc
curl -LO https://github.com/philrandal/gpsctl/archive/refs/tags/v1.16.tar.gz
tar xvzf v1.16.tar.gz
mv gpsctl-1.16 gpsctl
pushd gpsctl
./build.sh
sudo cp gpsctl /usr/local/bin

Now that I have gpsctl, I fiddled with the Uputronics board config for a while, before finally resetting the board and saving my own settings to the board:

gpsctl -p /dev/ttyAMA0 -a --reset
gpsctl -p /dev/ttyAMA0 -a -B 115200 --configure_for_timing --save_config -v

It takes 20 minutes or so for the GPS device to get a strong signal, so you have to wait. Eventually, you’ll see something like:

$ gpsctl -Q fix
 Time (UTC): 2021-10-23 01:01:31 (yyyy-mm-dd hh:mm:ss)
   Latitude:  37.75479700 N
  Longitude: 122.40819180 W
   Altitude: 19.738 feet
     Motion: 0.025 mph at 66.924 degrees heading
 Satellites: 7 used for computing this fix
   Accuracy: time (31 ns), height (+/-39.915 feet), position (+/-29.354 feet), heading(+/-29.597 degrees), speed(+/-0.067 mph)

I installed the accompanying systemd service unit, so that gpsctl will run every time the machine starts:

sudo cp etc/systemd/system/ublox-init.service /etc/systemd/system
cat <<EOF | sudo tee /etc/default/gpsctl
PARAMS="-p /dev/ttyAMA0 -q -a -B 115200 --configure_for_timing"
EOF
systemctl enable ublox-init.service
systemctl daemon-reload

gpsd and Chrony

With any luck, the hardware works great and now we just have to get the software talking to it.

Testing gpsd

This will give you enough debug information to see that everything’s working.

$ sudo gpsd -N -n -s 115200 -D 4 /dev/ttyAMA0 /dev/pps0
gpsd:INFO: launching (Version 3.23.1, revision 3.23.1)
gpsd:INFO: listening on port gpsd
gpsd:INFO: stashing device /dev/ttyAMA0 at slot 0
gpsd:INFO: SER: opening GPS data source type 2 at '/dev/ttyAMA0'
gpsd:INFO: SER: fd 6 current speed 115200, 8N1
gpsd:INFO: SER: fd 6 current speed 115200, 8O1
gpsd:INFO: SER: fd 6 current speed 115200, 8N1
gpsd:INFO: SER: fd 6 current speed 115200, 8N1
gpsd:INFO: SER: fd 6 current speed 115200, 8N1
gpsd:INFO: gpsd_activate(2): activated GPS (fd 6)
gpsd:INFO: KPPS:/dev/pps0 RFC2783 path:/dev/pps0, fd is 7
gpsd:INFO: KPPS:/dev/pps0 pps_caps 0x1151
gpsd:INFO: KPPS:/dev/pps0 have PPS_CANWAIT
gpsd:WARN: KPPS:/dev/pps0 missing PPS_CAPTURECLEAR, pulse may be offset
gpsd:INFO: KPPS:/dev/pps0 kernel PPS will be used

You’ll start seeing gpsd get both GPS and PPS signals.

Alternatively you can run cgps and watch your GPS in real time, which is more fun. At least, that’s my idea of fun.

Once you know that these parameters work reliably for you, put them into /etc/default/gpsd and the systemd unit will pick them up:

cat <<EOF | sudo tee /etc/default/gpsd
# Devices gpsd should collect to at boot time.
# They need to be read/writeable, either by user gpsd or the group dialout.
DEVICES="/dev/ttyAMA0 /dev/pps0"

# Other options you want to pass to gpsd
GPSD_OPTIONS="-n -s 115200"

# Automatically hot add/remove USB GPS devices via gpsdctl
USBAUTO="true"
EOF
systemctl restart gpsd

And gpsd is running:

$ sudo systemctl status gpsd
● gpsd.service - GPS (Global Positioning System) Daemon
     Loaded: loaded (/usr/lib/systemd/system/gpsd.service; enabled; vendor preset: disabled)
     Active: active (running) since Fri 2021-10-22 18:01:36 PDT; 5min ago
TriggeredBy: ● gpsd.socket
    Process: 2053 ExecStart=/usr/bin/gpsd $GPSD_OPTIONS $OPTIONS $DEVICES (code=exited, status=0/SUCCESS)
   Main PID: 2054 (gpsd)
      Tasks: 3 (limit: 1820)
        CPU: 1.510s
     CGroup: /system.slice/gpsd.service
             └─2054 /usr/bin/gpsd -n -s 115200 /dev/ttyAMA0 /dev/pps0

Oct 22 18:01:36 gps systemd[1]: Starting GPS (Global Positioning System) Daemon...
Oct 22 18:01:36 gps systemd[1]: Started GPS (Global Positioning System) Daemon.

Chrony

I found that the chrony package in Arch/Manjaro didn’t have PPS support compiled into it. So, even though I’d installed it, I needed to recompile the binaries. I used the same build flags as the Arch package spec for chrony, but I got a build that has PPS support because I’d installed pps-tools previously.

curl -LO https://download.tuxfamily.org/chrony/chrony-4.2.tar.gz
tar xvzf chrony-4.2.tar.gz
cd chrony-4.2
./configure \
    --prefix=/usr \
    --enable-scfilter \
    --enable-ntp-signd \
    --with-user=chrony \
    --with-sendmail=/usr/bin/sendmail \
    --with-hwclockfile=/etc/adjtime \
    --sbindir=/usr/bin \
    --with-pidfile=/run/chrony/chronyd.pid
make
make install

For my setup, Chrony connects to gpsd via shared memory. Here’s the config file I ended up with. The last three lines are the most important. The allow line starts the NTP server (Chrony is a client otherwise), and the two refclock lines connect gpsd to Chrony as sources:

cat <<EOF | sudo tee /etc/chrony.conf
driftfile /var/lib/chrony/drift
ntsdumpdir /var/lib/chrony
leapsectz right/UTC
makestep 1.0 3
rtcsync

allow
# The offset of 0.0424 is specific to my Uputronics GPS
refclock SHM 0 refid GPS precision 1e-1 offset 0.0424 delay 0.2
refclock SHM 1 refid PPS precision 1e-7
pool north-america.pool.ntp.org
EOF

sudo systemctl restart chronyd.service

And that’s it, Chrony is up and running.

But are the GPS and PPS signals being used by chrony? If you run chronyc sources, you should see a #- by GPS and a #* next to PPS.

$ chronyc sources
# chronyc sources
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
#- GPS                           0   4   375    18  +8362us[+8362us] +/-  200ms
#* PPS                           0   4   364    63   -147ns[  -65ns] +/-  180ns
^- ntp2.wiktel.com               1   6   377    27  +2107us[+2107us] +/-   30ms
^- briareus.schulte.org          3   6   377    31   -737us[ -737us] +/-   70ms
^- ftp8.ofertadasorte.com.br     2   6   377    27  -1755us[-1755us] +/- 7632us
^- 44.190.40.123                 2   6   377    28    +30us[  +30us] +/-   45ms

Helpful chrony commands:

Configuring client machines

If clients on a LAN run chrony also, you can take advantage of the local directive so that they sync with each other in case your NTP server goes down.

I’m mostly running systemd-timesyncd, though:

# timedatectl timesync-status
       Server: 162.159.200.1 (162.159.200.1)
Poll interval: 34min 8s (min: 32s; max 34min 8s)
         Leap: normal
      Version: 4
      Stratum: 3
    Reference: A0401C7
    Precision: 1us (-25)
Root distance: 13.930ms (max: 5s)
       Offset: +175us
        Delay: 4.480ms
       Jitter: 344us
 Packet count: 105
    Frequency: +13.141ppm
# timedatectl
               Local time: Fri 2021-10-22 17:24:06 PDT
           Universal time: Sat 2021-10-23 00:24:06 UTC
                 RTC time: Sat 2021-10-23 00:24:06
                Time zone: America/Los_Angeles (PDT, -0700)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

/etc/systemd/timesyncd.conf:

[Time]
NTP=ntp
FallbackNTP=time.cloudflare.com 0.us.pool.ntp.org 1.us.pool.ntp.org 2.us.pool.ntp.org 3.us.pool.ntp.org

Future Ideas

I’d like to track and improve the accuracy of the clock. I’d also like to put some basic monitoring in place.

Add PTP

Following the GPSD Time Service HOWTO

$ ethtool --show-eee eth0
EEE settings for eth0:
 EEE status: enabled - inactive
 ...
$ ethtool --set-eee eth0 eee off

It needs to be built from source:

git clone git://git.code.sf.net/p/linuxptp/code linuxptp
cd linuxptp
make
sudo make install

cat <<EOF | sudo tee /usr/local/etc/ptp4l.conf
> [global]
# only syslog every 1024 seconds
summary_interval 10

# send to chronyd/ntpd using SHM 2
clock_servo ntpshm
ntpshm_segment 2

# set our priority high since we have PPS
priority1 10
priority2 10

[eth0]
EOF

echo "refclock SHM 2 refid PTP precision 1e-7" >> /etc/chrony.conf