About PTP
PTP is a time synchronization protocol defined in IEEE 1588 for nodes distributed across a network. With PTP, it is possible to synchronize distributed clocks with an accuracy of less than 1 microsecond via Ethernet networks. In addition, PTP's hardware timestamping feature provides timestamp information in the ERSPAN Type III header that can be used to calculate packet latency among edge, aggregate, and core switches.
A PTP system can consist of a combination of PTP and non-PTP devices. PTP devices include ordinary clocks, boundary clocks, and transparent clocks. Non-PTP devices include ordinary network switches, routers, and other infrastructure devices.
PTP is a distributed protocol that specifies how real-time PTP clocks in the system synchronize with each other. These clocks are organized into a master-slave synchronization hierarchy with the grandmaster clock, which is the clock at the top of the hierarchy, determining the reference time for the entire system. Synchronization is achieved by exchanging PTP timing messages, with the members using the timing information to adjust their clocks to the time of their master in the hierarchy. PTP operates within a logical scope called a PTP domain.
PTP supports the following functionality:
-
Multicast and unicast PTP transport—In the multicast transport mode, PTP uses multicast destination IP address 224.0.1.129 as per IEEE 1588 standards for communication between devices. For the source IP address, it uses the user configurable global IP address under the PTP domain. In the unicast transport mode, PTP uses configurable unicast source and destination IP addresses that can be configured under an interface. In both, the unicast and the multicast modes, PTP uses UDP ports, 319 for event messages and 320 for general messages communication between devices.
-
PTP multicast configuration is supported only under physical interface for L2 or L3. Unicast PTP configuration supported only under L3 physical interface. PTP is not supported for virtual interfaces such as Port-channel, SVI, and tunnel.
-
PTP encapsulation over UDP over IP—PTP uses UDP as the transport protocol over IP. In both, the unicast and multicast modes, PTP uses UDP ports 319 for event messages and 320 for general messages communication between devices. L2 encapsulation mode is not supported.
-
PTP profiles—PTP supports default (1588) , AES67, and SMPTE 2059-2 profiles. They all have different ranges of sync and delay request intervals. For information on the default profile, refer to IEEE 1588. For more information on AES67 and SMPTE 2059-2, refer to the respective specifications.
-
Path delay measurement—We support delay request and response mechanism to measure the delay between the master and slave devices. Peer delay request and response mechanism is not supported.
-
Message intervals—You can configure the interval at which the announce, sync, and delay request messages needs to be sent between devices.
-
Best master clock (BMC) selection—BMC algorithm is used to select master, slave, and passive states of the PTP enabled interfaces based on the Announce message received as per 1588 specification.
PTP Offload
This feature distributes the PTP functionality to the line cards and allows scaling of the number of PTP sessions that are supported on the system. This feature is available for Cisco Nexus 9500 platform switches with 9700-EX, 9700-FX, 9636C-R, 9636Q-R, 9624D-R2, and 9636C-RX line cards.
Dynamic Unicast
This feature allows BMCA to select roles dynamically as follows, rather than assigning static roles.
-
Use the ptp peer <ipv4>/<ipv6> command to configure Peer IPs.
-
Port remains in a Listening state until the Peer IP is reachable, when it transitions to Master state.
-
Announce packets are sent as soon as the peer is reachable.
-
Based on Announce packet (using BMCA), Role will be decided. Port state transitions accordingly.
PTP Time Distribution Hold
In a properly synchronized PTP network, when any PTP node goes down and comes up, the PTP clock is synchronized to its primary time source (GM). During this process, the local node has significant correction and it tries to correct its local clock. At that time, the node can send incorrect time to the downstream nodes and cause issues for all downstream nodes. The Time Distribution (TD) hold feature, introduced in Cisco NX-OS Release 10.5(1)F, resolves this issue by ensuring that the node is properly synchronized to its primary source and distributes time to the downstream nodes during boot up.
The TD hold feature holds the time distribution until a Boundary Clock (BC) node locks to the primary time source and settles down to the target correction value. The TD hold enabled node receives all PTP packets, does the normal state change, and synchronizes time, but it does not send any PTP packets out.
Note |
If all nodes reboot at the same time (with a difference of few seconds), each node will be in active hold time, which sometimes results in no nodes having secondary port. This leads to the BMC taking a long time to find the best clock. Hence, the user needs to take this into account when enabling this feature. |
PTP Device Types
The PTP device type is configurable and can be used to set the clock type.
Clocks
The following clocks are common PTP devices:
- Ordinary clock
-
Communicates with the network based on a single physical port, similar to an end host. An ordinary clock can function as a grandmaster clock.
- Boundary clock
-
Typically has several physical ports, with each port behaving like a port of an ordinary clock. However, each port shares the local clock, and the clock data sets are common to all ports. Each port decides its individual state, either master (synchronizing other ports connected to it) or slave (synchronizing to a downstream port), based on the best clock available to it through all of the other ports on the boundary clock. Messages related to synchronization and establishing the master-slave hierarchy terminate in the protocol engine of a boundary clock and are not forwarded.
- Transparent clock
-
Forwards all PTP messages like an ordinary switch or router but measures the residence time of a packet in the switch (the time that the packet takes to traverse the transparent clock) and in some cases the link delay of the ingress port for the packet. The ports have no state because the transparent clock does not need to synchronize to the grandmaster clock.
There are two kinds of transparent clocks:
- End-to-end transparent clock
-
Measures the residence time of a PTP message and accumulates the times in the correction field of the PTP message or an associated follow-up message.
- Peer-to-peer transparent clock
-
Measures the residence time of a PTP message and computes the link delay between each port and a similarly equipped port on another node that shares the link. For a packet, this incoming link delay is added to the residence time in the correction field of the PTP message or an associated follow-up message.
Note |
PTP operates only in boundary clock mode. Cisco recommends deployment of a Grand Master Clock (10 MHz) upstream, with servers containing clocks requiring synchronization connected to the switch. End-to-end transparent clock and peer-to-peer transparent clock modes are not supported. |
- Grandmaster clock
-
In a single PTP domain, the Grandmaster (GM) node acts as a primary clock source for the entire PTP network. The primary source for GM node will be taken either from internal GNSS system or external GNSS system. GM node can’t synchronize time or frequency from other PTP node i.e. GM node can’t have any slave ports, all ports act as only Master role.
Clock Modes
The IEEE 1588 standard specifies two clock modes for the PTP supporting devices to operate in: one-step and two-step.
One-Step Mode:
In one-step mode the clock synchronization messages include the time at which the master port sends the message. The ASIC adds the timestamp to the synchronization message as it leaves the port. The master port operating in one-step mode is available for Cisco Nexus 9508-FM-R and 9504-FM-R fabric modules and Cisco Nexus 9636C-R, 9636Q-R, 9624D-R2, and 9636C-RX line cards.
The slave port uses the timestamp that comes as part of the synchronization messages.
Two-Step Mode:
In two-step mode the time at which the synchronization message leaves the port is sent in a subsequent follow-up message. This is the default mode.
PTP Process
The PTP process consists of two phases: establishing the master-slave hierarchy and synchronizing the clocks.
Within a PTP domain, each port of an ordinary or boundary clock follows this process to determine its state:
-
Examines the contents of all received announce messages (issued by ports in the master state)
-
Compares the data sets of the foreign master (in the announce message) and the local clock for priority, clock class, accuracy, and so on
-
Determines its own state as either master or slave
The ordinary and boundary clocks use Sync , Delay_Req , Follow_Up , Delay_Resp event messages to generate and communicate timing information.
These messages are sent in the following sequence:
-
The master sends a Sync message to the slave and notes the time,
t1
at which it was sent. For one-step Sync message carries the time when the message leaves the master and for two-step this time is sent in the subsequent Follow-Up event message. -
The slave receives the Sync message and notes the time of reception,
t2
. -
The master conveys to the slave the timestamp,
t1
by embedding the timestamp in a Follow_Up event message. -
The slave sends a Delay_Req message to the master and notes the time,
t3
at which it was sent. -
The master receives the Delay_Req message and notes the time of reception,
t4
. -
The master conveys to the slave the timestamp,
t4
by embedding it in a Delay_Resp message. -
After this sequence, the slave possesses all four timestamps. These timestamps can be used to compute the offset of the slave clock relative to the master, and the mean propagation time of messages between the two clocks.
The following figure describes the event messages in the PTP process that generate and communicate timing information.
ITU-T Telecom Profile for PTP
Cisco NX-OS software supports ITU-T Telecom Profiles for PTP as defined in the ITU-T recommendation. A profile consists of PTP configuration options applicable only to a specific application.
Separate profiles can be defined to incorporate PTP in different scenarios based on the IEEE 1588-2008 standard. A telecom profile differs in several ways from the default behavior defined in the IEEE 1588-2008 standard and the key differences are mentioned in the subsequent sections.
The following sections describe the ITU-T Telecom Profiles that are supported for PTP:
Telecom Profile G.8275.1
Cisco's Telecom Profile G.8275.1 feature supports the ITU-T G.8275.1 : Precision time protocol telecom profile for phase/time synchronization with full timing support from the network standard. The G.8275.1 profile fulfills the time-of-day and phase synchronization requirements in telecom networks with all network devices participating in the PTP protocol. The G.8275.1 profile with SyncE provides better frequency stability for the time-of-day and phase synchronization.
Features of the G.8275.1 profile are:
-
Synchronization Model: G.8275.1 profile adopts hop-by-hop synchronization model. Each network device in the path from master to slave synchronizes its local clock to upstream devices and provides synchronization to downstream devices.
-
Clock Selection: G.8275.1 profile also defines an alternate BMCA that selects a clock for synchronization and port states for the local ports of all devices in the network. The parameters defined as a part of the BMCA are:
-
Clock Class
-
Clock Accuracy
-
Offset Scaled Log Variance
-
Priority 2
-
Local Priority
-
Clock Identity
-
Steps Removed
-
Port Identity
-
-
Port State Decision: The port states are selected based on the alternate BMCA.
-
Alternate BMCA: It follows the alternate BMCA dataset comparison algorithm as defined in Rec. ITU-T G.8275.1/Y.1369.1 to select the GM for the node.
-
Packet Rates: The nominal packet rate for Announce packets is 8 packets-per-second and 16 packets-per-second for Sync/Follow-Up and Delay-Request/Delay-Response packets.
-
Transport Mechanism: G.8275.1 profile only supports Ethernet PTP transport mechanism.
-
Mode: G.8275.1 profile supports transport of data packets only in multicast mode. The forwarding is done based on forwardable or non-forwardable multicast MAC address.
-
Clock Type: G.8275.1 profile supports the following clock types:
-
Telecom Grandmaster (T-GM): Provides timing for other network devices and does not synchronize its local clock to other network devices.
-
Telecom Time Slave Clock (T-TSC): A slave clock synchronizes its local clock to another PTP clock, but does not provide PTP synchronization to any other network devices.
-
Telecom Boundary Clock (T-BC): Synchronizes its local clock to a T-GM or an upstream T-BC clock and provides timing information to downstream T-BC or T-TSC clocks.
Note
Telecom Boundary Clock (T-BC) is the only clock type supported in Cisco NX-OS Release 9.3(5).
-
-
Domain Numbers: The domain numbers that can be used in a G.8275.1 profile network ranges from 24 to 43. The default domain number is 24.
Telecom Profile G.8275.2
Cisco's Telecom Profile G.8275.2 feature supports the ITU-T G.8275.2 : Precision time protocol telecom profile for phase/time synchronization with partial timing support from the network standard. The G.8275.2 is a PTP profile for use in telecom networks where phase or time-of-day synchronization is required. It differs from G.8275.1 in that it is not required that each device in the network participates in the PTP protocol. Also, G.8275.2 uses PTP over IPv4 and IPv6 in unicast mode.
The G.8275.2 profile is based on the partial timing support from the network. Hence, nodes using G.8275.2 are not required to be directly connected. The G.8275.2 profile is used in mobile cellular systems that require accurate synchronization of time and phase. For example, the fourth generation (4G) of mobile telecommunications technology.
With upcoming technologies such as LTE-TDD, LTE-A CoMP, LTE MBSFN and Location-based services, eNodeBs (base station devices) are required to be accurately synchronized in phase and time. Having GNSS systems at each node is not only expensive, but also introduces vulnerabilities. The G.8275.2 profile meets the synchronization requirements of these new technologies.
PTP Ports
A port can be configured to change its role dynamically. If no role is assigned to a port, it can dynamically assume a primary, passive, or subordinate role based on the BMCA.
In G.8275.2, PTP ports are not tied to any specific physical interfaces, but are tied to a loopback (virtual) interface. Traffic from a PTP port is routed through any physical interface based on the routing decision. For a dynamic port, only one clock source can be configured.
Alternate BPCA
The BPCA (Best Primary Clock Algorithm, which is also known as Best Master Clock Algorithm (BMCA [RFCÂ 7273]) implementation in G.8275.2 is different from that in the default PTP profile. The G.8275.2 implementation specifies an alternate best primary clock algorithm (ABPCA), which is used by each device to select a clock to synchronize to, and to decide the port states of its local ports.
The following consideration apply to the G.8275.2 implementation of the BPCA:
-
PrimaryOnly—A per port attribute, PrimaryOnly defines the state of the port. If this attribute is true, the port is never placed in the subordinate state.
-
Priority 1—Priority 1 is always static in this profile and is set to 128. Priority 1 is not used in BPCA.
-
Priority 2—Priority 2 is a configurable value and its range if from 0 to 255.
-
Local Priority—Local priority is configured locally on clock ports to set the priority on nominated clocks. The default value is 128 and valid range is from 1 to 255.
Restrictions for Using the G.8275.2 Profile
-
In the G.8275.2 profile, PTP is supported only in the none mode (default).
-
A G.8275.2 PTP clock can have redundant clock sources configured (through multiple PTP ports). However, at any given time, a G.8275.2 PTP clock synchronizes to only one clock source, which is selected by BMCA.
-
The G.8275.2 does not provide any recommendations for performance analysis and network limits for the clocks.
High Availability for PTP
Stateful restarts are not supported for PTP. After a reboot or a supervisor switchover, the running configuration is applied. For more information on high availability, see the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide.