Arduino NTP Server

Andrew Rodland

YAPC::NA 2012

Madison, WI

June 12, 2012

Here It Is…

So what is it?

• A really accurate clock
• That gets its time from GPS
• And makes it available using NTP and Ethernet
• And runs on an Arduino

Why did I do it?

• I had a server room full of machines with different times
• Solution: sync them all to a local NTP pool, and sync that to the internet
• But I wanted more — no reliance on the internet, and more accuracy
• Thus began my hobby of building stratum-1 NTP servers

GPS Is a Time Source?

• Every GPS Satellite contains a precision atomic clock
• GPS positioning is based on the difference in arrival times from satellites at different distances
• But you can use it for time too…

Uses of Precision Time

• Log Auditing / Digital Forensics
• Finance
• Very-Long Baseline Interferometry

The Main Board

Freetronics EtherMega

• 8-bit ATmega2560 CPU
  • 16MHz
  • 256kB Flash
  • 8kB RAM
• Wiznet W5100 Ethernet
• Like an Arduino Mega 2560 and an Arduino Ethernet Shield in one

The GPS Module

USGlobalSat EM-318-02

• Fairly average SiRF-III GPS receiver module
• External antenna
• Pulse-per-second signal
• Runs on 3.3V

The LCD

CrystalFontz CFA-634

• Big
• Easy to read
• 1 serial wire + power
• Expensive
• Can be replaced with a cheaper LCD
  • Or no LCD at all

Timers

AVR timer/counter registers can be configured to:

• Count up and down automatically at various rates
• Trigger interrupts when they reach certain values
• Toggle outputs when they reach certain values
• Capture their values when events occur

Input Capture is Awesome

• GPS pulse-per-second signal (PPS) is wired to an input capture pin
• Timer value is instantly copied to the capture register
• Interrupt is triggered
• Exact moment is recorded, even if it takes a little while to handle the interrupt.

Timekeeping

• 16MHz CPU clock
• 1:8 timer prescaler = 2MHz
• 16-bit timer has a resolution of 500ns, and overflows within 32.768ms

Timekeeping (cont.)

• Configure upper limit to 62,500
• Overflow every 31.25ms = 32Hz timer interrupt
• Every 32nd timer interrupt: 1Hz interrupt
• Current time: seconds + interrupts / 32 + timer / 2,000,000

Digital Frequency Synthesis

• The clock crystal isn’t perfect
• So the timer doesn’t run at exactly 2MHz
• In the long run, it will gain or lose time
• Need a way to correct its speed

Digital Frequency Synthesis

• Add and subtract ticks
• We can adjust the timer limit up and down
• 62,500 ticks = 31.25ms
• 62,501 ticks = 31.2505ms
• 62,499 ticks = 31.2495ms

Digital Frequency Synthesis

If we add or subtract 1 tick to every timer interrupt, that’s:

• 500ns per interrupt
• 16 parts per million
• 16 microseconds per second
• 1.38 seconds per day

Not a fine enough adjustment!

Digital Frequency Synthesis

Instead, adjust the timer in units of 1/2048 tick per interrupt

• 1 tick (500ns) per 64 seconds
• 1/128 part per million
• 675 microseconds per day
• But we update much more often than once per day

Digital Frequency Synthesis

• Divide the adjustment by 2,048 to get the number of ticks to add every interrupt
• Add the remainder to an accumulator every interrupt
• When the accumulator is ≥ 2,048:
  • Subtract 2,048
  • Add one extra tick this interrupt
• Bresenham’s line-drawing algorithm, but with time.

Digital Frequency Synthesis (Example)

Digital Frequency Synthesis

So how do we figure out how fast to run the timer to keep it in sync with the signal from the GPS?

• Frequency Locked Loop (FLL)
• Phase Locked Loop (PLL)

Frequency Locked Loop (FLL)

• Count the number of ticks from one PPS to another one 64 seconds later
• At 2MHz this should be 128,000,000
• The difference is how fast/slow the clock runs, in units of 1 tick / 64 sec
  • Exactly the units the timekeeping loop uses

Phase Locked Loop (PLL)

• Measure how early or late the PPS is, compared to when we think the second is
• Feed it into a PI controller
  • Proportional – Integral, a PID controller without the D
• Clock behind GPS: run fast to catch up
• Clock ahead of GPS: run slow to fall back

NTP

• Send a packet from client to server and back
• Each machine timestamps the packet on transmit and receive
• By assuming network delay is symmetrical we can transfer a time offset

NTP (cont.)

Delay = (t₄ – t₁) – (t₃ – t₂)

• t₄ – t₁ is the roundtrip time seen by the client
• t₃ – t₂ is the processing time on the server
• The remainder is network delay

NTP (cont.)

• Offset = ½ [ (t₂ – t₁ – Delay_out) – (t₄ – t₃ – Delay_back) ]
• Delay_out = Delay_back = Delay / 2
• Offset = ½ [ (t₂ – t₁) – (t₄ – t₃) ]

The Code

• Written in C-style C++
• Some use of Arduino libraries, some bare-metal code
• Builds with gcc and make outside of the IDE, using Arduino.mk by Martin Oldfield et al. with some custom hacks
• Interrupt-driven with “bottom halves”
  • Interrupt handlers do minimal housekeeping, set a flag, and return
  • Main loop runs slower routines when their flags are set
  • Infrequent tasks (DHCP, temp probe) run every N seconds from a once-per-second task

The Simulator

• Allows debugging various parts of the time-keeping code
• Runs natively on Linux
• Contains mocked-up versions of the GPS, the timer, etc.
• Makefile builds same code twice with different defines

The Modules

• ntpserver.pde: Arduino-only main file
• simmain.cpp: Simulator-only main file
• config.h: #defines for tuning parameters and enabling other modules
• hwdep.cpp, hwdep.h: Low-level hardware access
• timing.cpp, timing.h: Time-keeping code
• gps.cpp, gps.h: GPS serial / SiRF protocol
• ethernet.cpp, ethernet.h: Ethernet, DHCP, NTP
• lcd.cpp, lcd.h: Serial LCD
• tempprobe.cpp, tempprobe.h: 1-wire Temperature Probe

GPS Notes

• YYYY-MM-DD HH:MM:SS from the GPS is used to drive the LCD display
• Week number + TOW + UTC-GPS offset from the GPS is used for NTP
• Custom SiRF binary protocol decoder

Ethernet Notes

• Onboard Wiznet W5100 is equivalent to Arduino Ethernet Shield
• Uses Arduino Ethernet library
• Really dirty hack to enable interrupt-driven Ethernet
• Order of magnitude timing improvement

Temperature Probe Notes

• Code and hardware is present but not used
• Measure the temperature-sensitivity of the crystal and compensate it
• Probe and crystal respond to temperature changes at different rates
• Makes things worse instead of better
• Could be improved and brought back?

Performance

• Verified against a Spectracom Netclock 9183
• Timekeeping: 95% within 1.5 microseconds
• NTP: within 10-20 microseconds
• Ethernet is the weak link
• But still outperforms the Spectracom’s ethernet

Performance (cont.)

Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std Dev
==============================================================================
spectre                     5   4   86m     -0.013      0.063  -2091ns    23us
arduino                    42  24   972     -0.712      0.012    -25ms  6302ns
goron                       5   4   452     -0.545      1.109   +545us    26us

Crystal Methods

• Arduino Mega no. 1
  • 300ppm frequency error @ room temp
  • 15ppm/°C temperature coefficient
• Arduino Mega no. 2
  • 800ppm frequency error @ room temp
  • 15ppm/°C temperature coefficient
• Freetronics EtherMega
  • 25ppm frequency error @ room temp
  • 0.1ppm/°C temperature coefficient

The Next Version

• Replace the quartz crystal with a Rubidium oscillator (LPRO-101)
• Replace the Arduino with an ARM chip with builtin Ethernet and a higher clock speed
• Instead of timing the PPS with input capture, generate our own PPS and compare it to GPS’s PPS using a time interval counter (PICTIC II)
• Digital Frequency synthesis is only used for startup / very large adjustments
• DAC adjusts the Rubidium C-field for fine adjustments
• Timestamping in Ethernet driver

The Next Version

Wrap-up

Questions / Code Tour

http://git.cleverdomain.org/arduino/