How to Set Up a PTP Network Using Linux PTP and SUSE® Linux Enterprise ====================================================================== March 2014 Scope ----- This how-to guide provides a conceptual overview of Linux PTP (Precision Time Protocol) and explains the basic setup of a PTP network consisting of a single Ethernet segment using Linux PTP and SUSE Linux Enterprise. Why Precision Time Protocol? ---------------------------- Precision Time Protocol (PTP) provides accurate clock synchronization across a local group of hosts. As opposed to Network Time Protocol (NTP), it does not aim at minimizing a host's difference from official world time but tries to minimize the difference from other hosts in the group. Traditionally, the task of precise synchronization in a group has been achieved with GPS time synchronization at each host or a central clock synchronizing the hosts via specialized hardware. PTP offers comparable accuracy using Ethernet based hardware, assuming stable network latencies and symmetric transport delays. About the Linux Implementation of PTP ------------------------------------- Linux PTP takes advantage of the hardware support implemented in some network interface cards (NICs). In addition, this implementation can work with some NICs that do not provide hardware PTP facilities. However, the necessary functionality is implemented in their hardware drivers. There are three components of the PTP software: * The in-kernel PTP support, consisting of the kernel PTP subsystem, pulse-per-second subsystem and NIC driver code changes necessary for time stamping passing packets * Daemon for synchronizing the NIC's clock and system clock needed in hardware time stamping setup * Daemon for implementing the protocol itself, which does the time calculations and responds to the inquiries from other hosts The Beginning ------------- First, we need to check what type of time stamping the NIC in question does. That can be determined using ethtool utility with option -T. Example output is as follows: # ethtool -T enp4s0f1 Time stamping parameters for enp4s0f1: Capabilities: hardware-transmit (SOF_TIMESTAMPING_TX_HARDWARE) software-transmit (SOF_TIMESTAMPING_TX_SOFTWARE) hardware-receive (SOF_TIMESTAMPING_RX_HARDWARE) software-receive (SOF_TIMESTAMPING_RX_SOFTWARE) software-system-clock (SOF_TIMESTAMPING_SOFTWARE) hardware-raw-clock (SOF_TIMESTAMPING_RAW_HARDWARE) PTP Hardware Clock: 3 Hardware Transmit Timestamp Modes: off (HWTSTAMP_TX_OFF) on (HWTSTAMP_TX_ON) Hardware Receive Filter Modes: none (HWTSTAMP_FILTER_NONE) ptpv1-l4-sync (HWTSTAMP_FILTER_PTP_V1_L4_SYNC) ptpv1-l4-delay-req (HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ) ptpv2-l4-sync (HWTSTAMP_FILTER_PTP_V2_L4_SYNC) ptpv2-l4-delay-req (HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ) ptpv2-l2-sync (HWTSTAMP_FILTER_PTP_V2_L2_SYNC) ptpv2-l2-delay-req (HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ) ptpv2-event (HWTSTAMP_FILTER_PTP_V2_EVENT) For us the most important section is Capabilities. That shows the card in the example can do both hardware time stamping (“hardware-transmit” and “hardware-receive”) using hardware clock (“hardware-raw-clock“) and software time stamping using kernel system clock (“software-transmit“, “software-receive“ and “software-system-clock“). Cards that support only software time stamping display only the “software-“ capabilities and none of the “hardware-“ ones. The time stamping mode will later be needed for configuring the user space daemon. The User Space Daemons ---------------------- The daemons are primarily configured via options in /etc/sysconfig/ptp4l and /etc/sysconfig/phc2sys. These files control what options will be provided to the two daemons: the protocol daemon, called ptp4l and the synchronization daemon, called phc2sys. Apart from /etc/sysconfig/ there is the /etc/ptp4l.conf configuration file controlling further behavior of the protocol daemon. Configuring Linux PTP with Hardware Time Stamping ------------------------------------------------- The hardware time stamping setup includes an independent clock running on the network card. This independent clock is used for timing calculations in PTP. The NIC PTP clock is synchronized to the system clock by the phc2sys daemon. Therefore, in this setup you always need to enable the phc2sys service. The NIC clock synchronization daemon phc2sys always synchronizes clocks in one direction, for example, slave clock to master clock. This is important when configuring a group master. For a group master, you usually need to synchronize the group to its system clock, which itself is synchronized to a precise time using NTP, for example. In this scenario the options for phc2sys will be “-s CLOCK_REALTIME -c ethX -w” where CLOCK_REALTIME is a special string recognized by phc2sys corresponding to the system clock, and ethX corresponds to the network interface that is connected with the PTP clock. For a PTP slave host the configuration is simpler and involves only telling phc2sys which network interface is associated with the slave PTP clock. Having the system clock as master is the default here, so the options are like “-s ethX -w” You might have noticed “-w” in the example options above. That is connected with the fact that the PTP clock is running in International Atomic Time (TAI), which is not the Coordinated Universal Time (UTC) used by system clock. There is a certain offset between these time scales. The “-w” option ensures the offset between TAI and UTC is read from the ptp4l daemon. See the phc2sys(8) manual page for more details about time scale usage. No extra options need to be passed to ptp4l as hardware time stamping is the default. The option string in /etc/sysconfig/ptp4l will typically look like “-i ethX -f /etc/ptp4l.conf” where ethX is a placeholder for the actual interface name. Configuring Linux PTP with Software Time Stamping ------------------------------------------------- With software time stamping the situation is simpler. There is no independent PTP clock to be synchronized, so phc2sys is not in use and the phc2sys service can be disabled. The only change that must be made is telling ptp4l that the NIC in question uses software time stamping mode. That can be accomplished by using “-S” in OPTIONS in /etc/sysconfig/ptp4l. Therefore, the typical option string will look like “-i ethX -f /etc/ptp4l.conf -S” where ethX is again a placeholder for the actual interface name. When there is a PTP group master using software time stamping, the time announced to the network is not using native PTP time scale. That may cause interoperability issues with third-party PTP solutions and should be taken into account when designing the system. Configuring the Group Master ---------------------------- In every group there is one master clock, and the other hosts run slave clocks. The election of group master is done automatically based on the host's clock parameters. Most important is the host priority value in the PTP domain. As there is a difference in the system setup for slave clocks and master clocks, the group master should be selected by the system administrator, and its priority value should prioritize it over other hosts in the group. That should be done by setting the lower priority1 value in /etc/ptp4l.conf. Domains and Transport Mechanisms -------------------------------- It is possible for multiple PTP host groups to share a single Ethernet segment. These groups can be separated by setting different values for the domainNumber in /etc/ptp4l.conf. Hosts belonging to one group share the same value of domainNumber. There are also different transport options for PTP packets over Ethernet. PTP messages can be encapsulated either in UDP4 (which is the default), UDP6 or directly into Ethernet frames. The transport mechanism can be selected either via command line option to ptp4l or in the /etc/ptp4l.conf configuration file. See the ptp4l(8) manual page for more details. Examples of Configurations -------------------------- Configuration examples in this section are not full configuration files but rather minimal list of changes that should be done to the indicated files. The string ethX always stands for the actual network interface name in your setup. Slave clock using software time stamping: /etc/sysconfig/ptp4l: OPTIONS=”-f /etc/ptp4l.conf -i ethX” No changes made to the distribution /etc/ptp4l.conf. Slave clock using hardware time stamping: /etc/sysconfig/ptp4l: OPTIONS=”-f /etc/ptp4l.conf -i ethX” /etc/sysconfig/phc2sys: OPTIONS=”-s ethX -w” No changes made to the distribution /etc/ptp4l.conf. Master clock using hardware time stamping: /etc/sysconfig/ptp4l: OPTIONS=”-f /etc/ptp4l.conf -i ethX” /etc/sysconfig/phc2sys: OPTIONS=”-s CLOCK_REALTIME -c ethX -w” /etc/ptp4l.conf: priority1 127 Master clock using software time stamping, not generally recommended: /etc/sysconfig/ptp4l: OPTIONS=”-f /etc/ptp4l.conf -i ethX” /etc/ptp4l.conf: priority1 127 (C) 2014 SUSE LLC. All Rights Reserved. SUSE and the SUSE logo are registered trademarks of SUSE LLC in the United States and other countries. All third-party trademarks are the property of their respective owners.