PTP is a time synchronization protocol for nodes distributed across a network. Its hardware timestamp feature provides greater accuracy than other time synchronization protocols such as the Network Time Protocol (NTP).

A PTP system can consist of a combination of PTP and non-PTP devices. PTP devices include ordinary clocks, boundary clocks, and transparent clocks. Non-PTP devices include ordinary network switches, routers, and other infrastructure devices.

PTP is a distributed protocol that specifies how real-time PTP clocks in the system synchronize with each other. These clocks are organized into a master-slave synchronization hierarchy with the grandmaster clock, which is the clock at the top of the hierarchy, determining the reference time for the entire system. Synchronization is achieved by exchanging PTP timing messages, with the members using the timing information to adjust their clocks to the time of their master in the hierarchy. PTP operates within a logical scope called a PTP domain.

Starting from Cisco NXOS Release 6.0(2)A8(3), PTP supports configuring multiple PTP clocking domains, PTP grandmaster capability, PTP cost on interfaces for slave and passive election, and clock identity.

All the switches in a multi-domain environment, belong to one domain. The switches that are the part of boundary clock, must have multi-domain feature enabled on them. Each domain has user configurable parameters such as domain priority, clock class threshold and clock accuracy threshold. The clocks in each domain remain synchronized with the master clock in that domain. If the GPS in a domain fails, the master clock in the domain synchronizes time and data sets associated with the announce messages from the master clock in the domain where the GPS is active. If the master clock from the highest priority domain does not meet the clock quality attributes, a clock in the subsequent domain that match the criteria is selected. The Best Master Clock Algorithm (BMCA) is used to select the master clock if none of the domains has the desired clock quality attributes. If all the domains have equal priority and the threshold values less than master clock attributes or if the threshold values are greater than the master clock attributes, BMCA is used to select the master clock.

Grandmaster capability feature controls the switch’s ability of propagating its clock to other devices that it is connected to. When the switch receives announce messages on an interface, it checks the clock class threshold and clock accuracy threshold values. If the values of these parameters are within the predefined limits, then the switch acts as per PTP standards specified in IEEE 1588v2. If the switch does not receive announce messages from external sources or if the parameters of the announce messages received are not within the predefined limits, the port state will be changed to listening mode. On a switch with no slave ports, the state of all the PTP enabled ports is rendered as listening and on a switch with one slave port, the BMCA is used to determine states on all PTP enabled ports. Convergence time prevents timing loops at the PTP level when grandmaster capability is disabled on a switch. If the slave port is not selected on the switch, all the ports on the switch will be in listening state for a minimum interval specified in the convergence time. The convergence time range is from 3 to 2600 seconds and the default value is 30 seconds.

The interface cost applies to each PTP enabled port if the switch has more than one path to grandmaster clock. The port with the least cost value is elected as slave and the rest of the ports will remain as passive ports.

The clock identity is a unique 8-octet array presented in the form of a character array based on the switch MAC address. The clock identity is determined from MAC according to the IEEE1588v2-2008 specifications. The clock ID is a combination of bytes in a VLAN MAC address as defined in IEEE1588v2.

In a properly synchronized PTP network, when any PTP node goes down and comes up, the PTP clock is synchronized to its primary time source (GM). During this process, the local node has significant correction and it tries to correct its local clock. At that time, the node can send incorrect time to the downstream nodes and cause issues for all downstream nodes. The Time Distribution (TD) hold feature, introduced in Cisco NX-OS Release 10.5(1)F, resolves this issue by ensuring that the node is properly synchronized to its primary source and distributes time to the downstream nodes during boot up.

The TD hold feature holds the time distribution until a Boundary Clock (BC) node locks to the primary time source and settles down to the target correction value. The TD hold enabled node receives all PTP packets, does the normal state change, and synchronizes time, but it does not send any PTP packets out.

Note

If all nodes reboot at the same time (with a difference of few seconds), each node will be in active hold time, which sometimes results in no nodes having secondary port. This leads to the BMC taking a long time to find the best clock. Hence, the user needs to take this into account when enabling this feature.

The following clocks are common PTP devices:

Ordinary clock

Communicates with the network based on a single physical port, similar to an end host. An ordinary clock can function as a grandmaster clock.

Boundary clock

Typically has several physical ports, with each port behaving like a port of an ordinary clock. However, each port shares the local clock, and the clock data sets are common to all ports. Each port decides its individual state, either master (synchronizing other ports connected to it) or slave (synchronizing to a downstream port), based on the best clock available to it through all of the other ports on the boundary clock. Messages that are related to synchronization and establishing the master-slave hierarchy terminate in the protocol engine of a boundary clock and are not forwarded.

Transparent clock

Forwards all PTP messages like an ordinary switch or router but measures the residence time of a packet in the switch (the time that the packet takes to traverse the transparent clock) and in some cases the link delay of the ingress port for the packet. The ports have no state because the transparent clock does not need to synchronize to the grandmaster clock.

There are two kinds of transparent clocks:

End-to-end transparent clock

Measures the residence time of a PTP message and accumulates the times in the correction field of the PTP message or an associated follow-up message.

Peer-to-peer transparent clock

Measures the residence time of a PTP message and computes the link delay between each port and a similarly equipped port on another node that shares the link. For a packet, this incoming link delay is added to the residence time in the correction field of the PTP message or an associated follow-up message.

Note	PTP operates only in boundary clock mode. We recommend that you deploy a Grand Master Clock (10 MHz) upstream. The servers contain clocks that require synchronization and are connected to the switch. End-to-end transparent clock and peer-to-peer transparent clock modes are not supported.

The PTP process consists of two phases: establishing the master-slave hierarchy and synchronizing the clocks.

Within a PTP domain, each port of an ordinary or boundary clock follows this process to determine its state:

• Examines the contents of all received announce messages (issued by ports in the master state)

• Compares the data sets of the foreign master (in the announce message) and the local clock for priority, clock class, accuracy, and so on

• Determines its own state as either master or slave

After the master-slave hierarchy has been established, the clocks are synchronized as follows:

• The master sends a synchronization message to the slave and notes the time it was sent.

• The slave receives the synchronization message and notes the time that it was received. For every synchronization message, there is a follow-up message. The number of sync messages should be equal to the number of follow-up messages.

• The slave sends a delay-request message to the master and notes the time it was sent.

• The master receives the delay-request message and notes the time it was received.

• The master sends a delay-response message to the slave. The number of delay request messages should be equal to the number of delay response messages.

• The slave uses these timestamps to adjust its clock to the time of its master.

Stateful restarts are not supported for PTP

• In a Cisco Nexus 3500 only environment, PTP clock correction is expected to be in the 1- to 2-digit range, from 1 to 99 nanoseconds. However, in a mixed environment, PTP clock correction is expected to be up to 3 digits, from 100 to 999 nanoseconds.

• Cisco Nexus 3500 Series switches support mixed non-negotiated mode of operation on master PTP ports. Meaning that when a slave client sends unicast delay request PTP packet, the Cisco Nexus 3500 responds with an unicast delay response packet. And, if the slave client sends multicast delay request PTP packet, the Cisco Nexus 3500 responds with a multicast delay response packet. For mixed non-negotiated mode to work, the source IP address used in the ptp source <IP address> configuration on the BC device must also be configured on any physical or logical interface of the BC device. The recommended best practice is to use the loopback interface of the device.

• Cisco Nexus 3500 Series switches support a maximum of 48 PTP sessions.

• Cisco Nexus 3500 Series switches do not support PTP on 40G interfaces.

• PTP operates only in boundary clock mode. End-to-end transparent clock and peer-to-peer transparent clock modes are not supported.

• PTP operates when the clock protocol is set to PTP. Configuring PTP and NTP together is not supported.

• PTP supports transport over User Datagram Protocol (UDP). Transport over Ethernet is not supported.

• PTP supports only multicast communication. Negotiated unicast communication is not supported.

• PTP is limited to a single domain per network.

• When ptp acl-redirect is configured, PTP management packets are software forwarded. The management packets are expected to be within 100 packets/second to avoid CoPP drops.

• PTP-capable ports do not identify PTP packets and do not time-stamp or redirect those packets to CPU for processing unless you enable PTP on those ports. This means that if the PTP is disabled on a port, then the device will be capable of routing any multicast PTP packets, regardless of their type, assuming that there is a multicast state present for this. None of these multicast PTP packets from this port will be redirected to CPU for processing, because the exception used to redirect them to the CPU is programmed on a per-port basis, based on whether the PTP is enabled or not on the respective port.

• 1 pulse per second (1 PPS) input is not supported.

• PTP over IPv6 is not supported.

• Cisco Nexus switches should be synchronized from the neighboring master using a synchronization log interval that ranges from –3 to 1.

• All unicast and multicast PTP management messages will be forwarded as per the forwarding rules. All PTP management messages will be treated as regular multicast packets and process these in the same way as the other non-PTP multicast packets are processed by Cisco Nexus 3500 switches.

• You must configure the incoming port as L3/SVI to enable forwarding of the PTP unicast packets.

• We recommend that Cisco Nexus 3500 switches do not participate in unicast negotiation between the unicast master and clients.

• One-step PTP is not supported on Cisco Nexus 3500 series platform switches.

• Beginning with Cisco NX-OS Release 10.5(1)F, the PTP Time Distribution (TD) hold feature is introduced. This feature allows for holding the time distribution until a Boundary Clock node locks to the primary time source and settles down to the target correction value.

PTP supports Mixed mode for delivering PTP messages, which is detected automatically by Cisco Nexus device, based on the type of delay_req message received from connected client and no configuration is required. In this mode when slave sends delay_req in unicast message, master also replies with unicast delay_resp message.

The timestamp tagging feature provides precision time information to track in real time when packets arrive at remote devices. Packets are truncated and timestamped using PTP with nanosecond accuracy. Using the TAP aggregation functionality on the switch, along with the Cisco Nexus Data Broker, you can copy the network traffic using SPAN, filter and timestamp the traffic, and send it for recording and analysis.