Electronic Oscillator: Computer itself to Dervie its Notion of Time

Computer’s clock are based on a 1 MHz electronic oscillator circuit, that is oscillating at some frequency, and driving an interrupt. So the operating system can use it to derive its notion of time. It helps computer to keep the time correct.

But a bad oscillator could be influenced by the heat of CPU, like compiling Linux kernel, etc. A really high-quality oscillator would be really expensive. So most computers come with fairly bad oscillators.

GPS: External Reference

Buy a GPS appliance to receive time from the signal of the GPS satellites.

From the book DDIA, nearly every large data center have one in their room to synchronize time from GPS.

The accuracy there is something like maybe 100 nanoseconds.

Why Clock Synchronization is so Important?

Total ordering of events in a distributed system.

NTP: Synchronize Clock Inside of a Computer Network

NTP stands for Network Time Protocol.

There are four timestamps, as the core:

  1. client send timestamp: before client send a packet to server, client record a timestamp as it.
  2. server receive timestamp: when the server received the packet.
  3. server send timestamp: when the server reply to the client.
  4. client receive timestamp: when the client recived the reply packet from the server.

With those four timestamps, we can compute two important numbers:

  1. The offset, how far my clock is off from yours.
  2. The delay, how long it took those packets to traverse the network.

The Weakness of NTP

  • Too much assumption.
  • Adjust time abruptly.
  • May change time in the negative direction.

chrony: A More Customizable Implementation of NTP

NTP can do the Slewing in an appropariate way: keep the rates pretty close to the real-time rates, and it’s slowly, for chrony, the default is 12 seconds to remove one second of error.

PTP: Precision Time Protocol

A successor to NTP that uses multicast, hardware timestamping, networking switches, and faster default update rates to achieve better synchronization. Which meets MiFID II1.

NTP is good to milliseconds. PTP is good to microseconds.

Three things made PTP much more accurate than NTP:

  • Hardware Timestamping: many hardware vendors support it with PTP. So PTP get a timestamp when the packet arrives at the network card.
  • Frequency of Updates.
  • Lots of Switches, multicast by switches, involved and help you move time across the network while understanding their own delays that they’re adding to it.

Multicast: Best Master Clock Algorithm

A process for determining the best time source available, which PTP then uses as its grandmaster2.

With multicast, as a client, you don’t need to configured which server to talk, just run the client, and start receiving from grandmaster. It also good to grandmaster: NO connection need to maintain.

Also with the multicast, PTP should be used:

  1. In a small area.
  2. Hybrid mode, sync message comes in multicast, but the delay request and the delay response wind up being unicast.

Final Solution

  1. Some GPS appliances get time from GPS.
  2. Use PTP to move time from the GPS appliances to a set of Linux machines.
  3. The, NTP interleaved mode to move the time from those machines onto machines further downstream.

Short Overview of NTP Interleaved Mode

the interleaved mode is sort of a special way in which you can run NTP and that when you transmit your next NTP packet you send that hardware timestamp that you got for the previous transmission, and then the other side, each side can use those. I don’t want to get into too much of the details of how that works, but it allows you to get more accuracy and to leverage those hardware timestamps on transmission.

Results

We are absolutely no worse error than about 35 microseconds.

  1. A set of European financial regulations that required, among a number of other changes, that some market participants, including Jane Street, be able to show that their clocks were synchronized, within 100 Microseconds, to UTC. ↩︎

  2. The clock used by PTP as its time reference. ↩︎