Configuring Frequency Synchronization

This chapter describes how to configure Frequency Synchronization on Cisco NX-OS devices.

This chapter includes the following sections:

About Frequency Synchronization

Next generation networks must provide the ability to distribute precision frequency around the network. This is known as frequency synchronization. Precision frequency is required for applications such as circuit emulation and cell tower frequency referring. To achieve compliance to ITU specifications for TDM, differential method circuit emulation must be used, which requires a known, common precision frequency reference at each end of the emulated circuit.

It is also often desirable to precisely synchronize the time-of-day between different network devices, for example in order to accurately calculate the packet delay between two nodes in the network.

As, increasingly, SDH and SONET equipment is replaced by Ethernet equipment, this frequency synchronization ability is required over Ethernet ports. Synchronous Ethernet (SyncE) provides this PHY-level frequency distribution of known common precision frequency references.

To maintain SyncE links, a set of operations messages are required. These messages ensure a node is always deriving timing from the most reliable source, and transfer information about the quality of the timing source being used to clock the SyncE link. A simple protocol providing a transport channel for Synchronization Status Messages (SSMs) over Ethernet is documented in the ITU standard G.8264 and its related recommendations.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) so that devices can know the best available source to use for synchronization. In order to define a preferred network synchronization flow, and to help prevent timing loops, priority values can be assigned to particular timing sources on each switch. The combination of QL information and user-assigned priority levels allows each switch to choose a timing source to use to clock its SyncE as described in the ITU standard G.781.

SyncE does not carry time-of-day information. Time-of-day synchronization is achieved using packet-based technologies, such as PTP. Clock sources such as GNSS/GPS can be used to inject accurate time-of-day, as well as frequency, into the network. Each switch in the network can select a source for time-of-day as well as a source for frequency (or select the same source for both, if possible and desirable), and pass its time-of-day information to its peers using a packet-based protocol. There is no equivalent to QL for time-of-day information, so selection between different sources for time-of-day is achieved using configuration.

Hybrid SyncE-PTP with External PRC Source

Beginning with Cisco NX-OS Release 9.3(5), a hybrid SyncE-PTP topology is supported to achieve the end-to-end network precision frequency required for circuit emulation and cell tower frequency referring.

The following figure shows the external timing source as the Grandmaster/Primary Reference Time Clock (GM/PRTC) providing the timing source for the Telecom Boundary Clocks (T-BCs).

Figure 1. Hybrid SyncE-PTP with External PRC Source

Timing Sources

There are various timing sources that input timing clock signals into the system/network, and outputs of timing clock signals from the system as described below.

Timing Inputs

Input clock signals can be received from the platform hardware either via inputs from timing sources like GPS/GNSS, from internal oscillators, recovered from the line of a SyncE enabled interface, or from timing over packet such as the Precision Time Protocol (PTP).

Platform independent (PI) software keeps a database of all these inputs, including a Quality Level (QL) and priority level associated with each. The priority level is configuration controlled, and the QL values can be obtained in a number of manners:

  • SyncE enabled interfaces receive SSMs via an Ethernet slow protocol (ESMC).

  • GPS and GNSS have fixed QL maintained by platform dependent (PD) software and notified to PI function.

  • PTP communicates its QL to the Frequency Synchronization PI software through the platform APIs.

  • A default QL value may be defined in the PD layer for the timing connectors, and internal oscillators.

  • Configuration may be set defining the QLs of timing sources.

Possible input sources:

  • Internal Oscillator

  • Recovered SyncE Clock

  • External Clock 1588/PTP

  • External Clock (GPS)

  • Internal Clock (GNSS)

Timing Outputs

The platform hardware can have a number of outputs for clock signals like timing clock outputs from SyncE and enabled interfaces for GPS (currently not supported).

The software keeps all these outputs in a database, including QL information associated with the clock signal being used to drive these outputs that may be explicitly configured. The QL information includes a QL value along with steps removed counters, the originator clock ID and a series of flags containing information about the path from the originator clock to the current clock. The QL values are transmitted in the same manners as described for the inputs (i.e. SyncE interfaces send ESMC SSMs).

Possible output sources:

  • SyncE

  • 1588/PTP: packet output is handled separately, in the PTP software.

Timing Source Selection Points

At various stages in syncing timing clocks around the system, the platform has the potential to make a choice over which of the available timing clocks it is to use for further processing. These selection points define the flow of timing clock signals through the system, and eventually lead to the overall decision on which input timing source is to be used for timing outputs.

How these selection points are setup on each platform is hardware dependent, but the platform independent (PI) layer defines a generic selection point abstraction that can flexibly represent any platform selection point hardware, and allows each platform to define which selection points it has, and how they are wired together. The PI code can then control these selection points, tracking and distributing required information about the timing sources, and interacting with the platform dependent (PD) layer to discover what the result of the PD selection is at each stage.

PI timing source selection points:

  • Available Timing Inputs: A number of timing clock inputs are available for the platform selection point hardware to choose between. The availability and associated QL information and priorities are tracked by PI software, which informs the PD layer which inputs are available, ranked in overall order along with their associated quality levels and priorities.

  • Platform Specific Selection: The platform layer makes a decision as to which of the inputs it is using based on the information obtained from PI, and other platform layer decisions (e.g. hardware level qualification of the clock-signals). The actual decision may be made in PD software (and programmed into the hardware), or the decision may be made by the hardware itself and communicated back to the PD software.

  • Selected Timing Source Outputs: The platform passes the selected clock signal(s) through as output(s) from the selection point. The PD layer informs the PI software the status of the available inputs, and which input(s) have been selected.

The platform layer defines what the selection points are, and how they are connected to potential inputs, and to each other, and to potential outputs. At each of the PD defined selection points, the platform can choose how to interact with the PI software to represent its particular hardware to the PI software. The hardware doesn’t have to perform clocking qualification at each selection point. Each selection point simply represents any place where the hardware selects between multiple inputs, passing the clock from one or many inputs forward.

Only one selection point type for SyncE on the switch supervisor is supported. This is named T0 and 1588 selection points. The T0 selection point represents the sources and its selection for the SyncE DPLL. The 1588 selection point represents the sources and its selection for the Assist DPLL for 1588 PLL.

Licensing Requirements for Synchronous Ethernet (SyncE)

Product

License Requirement

Cisco NX-OS

SyncE requires an add-on license. For a complete explanation of the Cisco NX-OS licensing scheme, see the Cisco NX-OS Licensing Guide.

Guidelines and Limitations for Frequency Synchronization

Frequency Synchronization has the following guidelines and limitations:

  • Refer to Nexus Switch Platform Support Matrix to see the list of Cisco Nexus switches that support the Frequency Synchronization (SyncE) feature through Cisco NX-OS releases.

  • SyncE is supported only on physical interfaces.

  • A maximum four ethernet interfaces can be monitored for SyncE selection input at any given instance of time.

  • Each quad port group on the PHY provides one reference clock.

  • Only one Ethernet interface from each quad port group can be configured as a SyncE input (one reference clock for each port group). There is no restriction on SyncE outputs.

  • SyncE must be enabled explicitly on the member interfaces for a port-channel. If a member interface of a port-channel is locked as a SyncE source, the ability to send out DNU on other member interfaces enabled for SyncE is controlled via the global command fsync transmit dnu lag-members .

  • Only G.8275.1 hybrid profile in BC mode is supported.

  • For a list of qualified optics for this release, see the Cisco Optics Compatibility Matrix.


    Note


    SyncE is not supported on 1G when GLC-TE is used as SFP.


  • Beginning with Cisco NX-OS Release 10.3(2)F, GPS and GNSS is supported on the Cisco Nexus 93180YC-FX3 switch.

Configuring Frequency Synchronization

Enabling Frequency Synchronization

Use this procedure to enable frequency synchronization, set the quality level of the switch, identify the clock ID for ESMC extended TLV, and configure the ESMC peer timeout for software upgrades.

SUMMARY STEPS

  1. configure terminal
  2. [ no ] feature frequency-synchronization
  3. [ no ] fsync quality itu-t option{ 1 | 2 generation { 1 | 2 }
  4. fsync clock-identity mac-address | no fsync clock-identity
  5. [ no ] fsync esmc peer receive timeout{ 0 | value}
  6. [ no ] fsync transmit dnu lag-members
  7. (Optional) copy running-config startup-config

DETAILED STEPS

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

[ no ] feature frequency-synchronization

Example:

switch(config)# feature frequency-synchronization
switch(config)#

Enables frequency synchronization on the switch.

Step 3

[ no ] fsync quality itu-t option{ 1 | 2 generation { 1 | 2 }

Example:

switch(config)# fsync quality itu-t option 1
switch(config)#

Specifies the quality level for the switch. The default is option 1 .

  • option 1 - Includes DNU, EEC1, PRC, PRTC, SEC, SSU-A, SSU-B, eEEC and ePRTC.

  • option 2 generation 1 - Includes DUS, EEC2, PRS, PRTC, RES, SMC, ST2, ST3, ST4, STU, eEEC and ePRTC.

  • option 2 generation 2 - Includes DUS, EEC2, PROV, PRS, PRTC, SMC, ST2, ST3, ST3E, ST4, STU, TNC, eEEC and ePRTC.

Note

 

The quality option that is configured here must match the quality option that is specified in the quality receive and quality transmit commands in the interface frequency synchronization configuration mode.

Step 4

fsync clock-identity mac-address | no fsync clock-identity

Example:

switch(config)# fsync clock-identity AB:CD:EF:12:34:56
switch(config)#

Specifies the clock ID to be used for Ethernet Synchronization Message Channel (ESMC) extended TLV. If no clock ID is configured, the system uses the default VDC MAC address.

Step 5

[ no ] fsync esmc peer receive timeout{ 0 | value}

Example:

switch(config)# fsync esmc peer receive timeout 120
switch(config)#

Specifies the ESMC peer receive timeout during ISSU.

0 disables the ESMC peer receive timeout.

value is the ESMC receive timeout in seconds. Enter a value from 120 through 600. Default = 120.

This command ensures that the ESMC control plane, and thus, selection, is not removed during software upgrade for a period of the value .

Step 6

[ no ] fsync transmit dnu lag-members

Example:

switch(config)# fsync transmit dnu lag-members
switch(config)#

SyncE must be enabled explicitly on the member interfaces for a port-channel. If a member interface of a port-channel is locked as a SyncE source, the ability to send out DNU (Do Not Use) QLs on other member interfaces that are enabled for SyncE is controlled by this command.

If enabled and an interface that is driving the clock for the switch is part of a port-channel, then any members of the port-channel will also send out DNU QL if SyncE is enabled on that interface.

If disabled, the system drives the QL of the selected source on all interfaces regardless of whether they are in the same port-channel as the interface driving the clock.

Step 7

(Optional) copy running-config startup-config

Example:

switch(config)# copy running-config startup-config
switch(config)#
(Optional)

Copies the running configuration to the startup configuration.

Configuring Frequency Synchronization on an Interface

Use this procedure to configure frequency synchronization on a specific interface.

Before you begin

This procedure, along with configuring PTP telecom profile on the same interface, constitutes the required interface settings for the "hybrid PTP" platform. For more information about the interface PTP telecom profile configuration, see Configuring PTP Telecom Profile 8275-1 on an Interface.

Make sure that you have globally enabled frequency synchronization on the device (global configuration command feature frequency-synchronization ).

SUMMARY STEPS

  1. configure terminal
  2. [ no ] interface ethernet slot / port
  3. [ no ] frequency synchronization
  4. [no] selection input
  5. [no] ssm disable
  6. [ no ] quality { receive | transmit } { exact | highest | lowest } itu-t option ql-option ql
  7. [no] priority value
  8. [no] wait-to-restore minutes

DETAILED STEPS

  Command or Action Purpose

Step 1

configure terminal

Example:

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

[ no ] interface ethernet slot / port

Example:

switch(config)# interface ethernet 1/5
switch(config-if)#

Specifies the interface on which you are enabling frequency synchronization and enters the interface configuration mode.

Step 3

[ no ] frequency synchronization

Example:

switch(config-if)# frequency synchronization
switch(config-if-freqsync)#

Enables frequency synchronization on the interface and enters the interface frequency synchronization configuration mode. The system selects the frequency signal to be used for clocking transmission, but does not enable the use of the interface as an input.

Note

 

The no form of the command functions only if there is no configuration present under the frequency synchronization configuration mode.

Step 4

[no] selection input

Example:

switch(config-if-freqsync)# selection input
switch(config-if-freqsync)#

Specifies the interface as a timing source to be passed to the selection algorithm.

Step 5

[no] ssm disable

Example:

switch(config-if-freqsync)# ssm disable
switch(config-if-freqsync)#

Disables sending ESMC packets and ignores any received ESMC packets.

Step 6

[ no ] quality { receive | transmit } { exact | highest | lowest } itu-t option ql-option ql

Example:

switch(config-if-freqsync)# quality receive exact itu-t option 1 PRC
switch(config-if-freqsync)#

Adjusts the Quality Level (QL) value that is used in received or transmitted SSMs, before it is used in the selection algorithm. Each timing source has a QL associated with it which provides the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) so that devices can know the best available source to use for synchronization.

  • exact ql : Specifies the exact QL regardless of the value that is received, unless the received value is DNU.

  • highest ql : Specifies an upper limit on the received QL. If the received value is higher than this specified QL, this QL is used instead.

  • lowest ql : Specifies a lower limit on the received QL. If the received value is lower than this specified QL, DNU is used instead.

Note

 

The quality option that is specified in this command must match the globally configured quality option in the quality itu-t option command.

Step 7

[no] priority value

Example:

switch(config-if-freqsync)# priority 100
switch(config-if-freqsync)#

Configures the priority of the frequency source on the interface. This priority is used in the clock-selection algorithm to choose between two sources that have the same QL. Values range from 1 (highest priority) to 254 (lowest priority). The default value is 100.

Note

 

This command is valid only if selection input is configured.

Step 8

[no] wait-to-restore minutes

Example:

switch(config-if-freqsync)# wait-to-restore 0
switch(config-if-freqsync)#

Configures the wait-to-restore time, in minutes, for frequency synchronization on the interface. minutes is the amount of time after the interface initializes before it is used for synchronization. Values range from 0 to 12. The default value is 5.

Note

 

This command is valid only if selection input is configured.

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this reference to check for configuration errors and verify the configuration.

show frequency synchronization configuration errors

The output of this command displays errors in the frequency synchronization configuration.

The following example shows the mismatch between the global quality itu-t option and the interface quality receive itu-t option :

switch# show frequency synchronization configuration errors

Elysian2(config)# show frequency synchronization configuration errors
Ethernet1/9
    quality receive exact itu-t option 1 PRC
* The QL that is configured is from a different QL option set than is
configured globally.


!Command: show running-config fsync_mgr all
!Running configuration last done at: Mon Feb 10 06:06:15 2020
!Time: Mon Feb 10 06:09:18 2020

version 9.3(5) Bios:version 00.04
feature frequency-synchronization

fsync quality itu-t option 2 generation 1 << must be the same as interface
fsync clock-identity 0
fsync esmc peer receive timeout 120

interface Ethernet1/9
  frequency synchronization
    selection input
    ssm disable
    quality receive exact itu-t option 1 PRC << must be the same as global
    priority 100
    wait-to-restore 0

interface Ethernet1/13
  frequency synchronization
    selection input
    ssm disable
    quality receive exact itu-t option 1 PRC
    priority 110
    wait-to-restore 0

show running-config fsync_mgr

The output of this command displays the current frequency synchronization configuration on the device.

The following is an example of the output of the show running-config fsync_mgr command:

switch# show running-config fsync_mgr

!Command: show running-config fsync_mgr
!Running configuration last done at: Mon Jun 29 13:49:34 2020
!Time: Mon Jun 29 13:50:51 2020

version 9.3(5) Bios:version 01.01 
feature frequency-synchronization


interface Ethernet1/9
  frequency synchronization
    selection input
    priority 99
    wait-to-restore 0

interface Ethernet1/13
  frequency synchronization
    selection input
    ssm disable
    quality receive exact itu-t option 1 PRC
    wait-to-restore 0

show frequency synchronization interface brief

The output of this command displays all interfaces that have frequency synchronization configured. Sources that have been nominated as inputs have ‘S’ in the Flags (Fl) column. Sources that have not been nominated as inputs do not have ‘S’ displayed.

The following is an example of the output of the show frequency synchronization interface brief command:

switch# show frequency synchronization interface brief

Flags:  > - Up                D - Down              S - Assigned for selection
        d - SSM Disabled      x - Peer timed out    i - Init state
        e - SSM Enabled       s - Output squelched
Fl    Interface          QLrcv QLuse Pri QLsnd Output driven by
===== ================== ===== ===== === ===== ========================
>S    Eth1/9             PRC   PRC   100 PRC   Eth1/13
>Sds  Eth1/13            n/a   PRC   100 n/a   Eth1/13

show frequency synchronization interface ethernet

The output of this command displays individual (user-selected) interfaces with associated frequency synchronization information.

The following is an example of the output of the show frequency synchronization interface ethernet slot / port command:

switch# show frequency synchronization interface ethernet 1/9

Interface State:UP
Assigned as input for Selection
  Wait-to-restore time 0 minute(s)
  SSM Enabled
    Peer Up for 00:07:01, last SSM received 0.307s ago
    Peer has come up 4 times and timed out 1 times
    ESMC SSMs        Total  Information     Event    DNU/DUS
      Sent:           1097         1088         9         83
      Received:        823          816         7        155
  Input:
    Up
    Last received QL: PRC
    Effective QL: PRC, Priority: 100
    Originator clock ID: fffffffffebfa543
    SyncE steps: 1, eSyncE steps: 1
    Not all steps run eSyncE; Chain of extended ESMC data is broken
    Supports frequency
  Output:
    Selected source: Eth1/13
    Selected source QL: PRC
    Effective QL: PRC
    Originator clock ID: fffffffffebfa863
    SyncE steps: 1, eSyncE steps: 1
    Not all steps run eSyncE; Chain of extended ESMC data is broken
  Next selection points:

show frequency synchronization selection (with PTP Profile 8275-1)

The output of this command displays the detailed view of the different selection points within the system.


Note


This example shows the output when PTP profile 8275-1 is configured.


The following is an example of the output of the show frequency synchronization selection slot / port command:

switch# show frequency synchronization selection
==============
Selection point: System Clock (T0) Selector (3 inputs, 1 selected)
  Last programmed 18.898s ago, and selection made 8.621s ago
  Next selection points
    Node scoped   :
  Uses frequency selection
  Used for local line interface output
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  11 Ethernet1/9               n/a                         PRC   99  Locked     
     Ethernet1/13              n/a                         PRC  100  Available  
     Internal0[1]              n/a                         SEC  255  Available  
==============
Selection point: IEEE 1588 Clock Selector (3 inputs, 1 selected)
  Last programmed 18.898s ago, and selection made 18.626s ago
  Next selection points
    Node scoped   :
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
     Ethernet1/9               n/a                         PRC   99  Unmonitored
     Ethernet1/13              n/a                         PRC  100  Unmonitored
  21 Internal0[1]              n/a                         SEC  255  Freerun    <<

show frequency synchronization selection (without PTP Profile 8275-1)

The output of this command displays the detailed view of the different selection points within the system.


Note


This example shows the output when PTP profile 8275-1 is not configured.


The following is an example of the output of the show frequency synchronization selection slot / port command:

switch# show frequency synchronization selection==============
Selection point: System Clock (T0) Selector (3 inputs, 1 selected)
  Last programmed 00:03:04 ago, and selection made 00:02:54 ago
  Next selection points
    Node scoped   :
  Uses frequency selection
  Used for local line interface output
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  11 Ethernet1/9               n/a                         PRC   99  Locked     
     Ethernet1/13              n/a                         PRC  100  Available  
     Internal0[1]              n/a                         SEC  255  Available  
==============
Selection point: IEEE 1588 Clock Selector (3 inputs, 1 selected)
  Last programmed 00:03:04 ago, and selection made 3.296s ago
  Next selection points
    Node scoped   :
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
     Ethernet1/9               n/a                         PRC   99  Unmonitored
     Ethernet1/13              n/a                         PRC  100  Unmonitored
  21 Internal0[1]              n/a                         SEC  255  Holdover   <<

show esmc counters all

The output of this command displays counters for sent and received ESMC SSMs.

The following is an example of the output of the show esmc counters all command:

ESMC Packet Counters of Interface Ethernet1/1:
    ESMC SSMs        Total  Information     Event    DNU/DUS
      Sent:              0            0         0          0
      Received:          0            0         0          0

ESMC Packet Counters of Interface Ethernet1/5:
    ESMC SSMs        Total  Information     Event    DNU/DUS
      Sent:              0            0         0          0
      Received:          0            0         0          0

ESMC Packet Counters of Interface Ethernet1/9:
    ESMC SSMs        Total  Information     Event    DNU/DUS
      Sent:           7685         7683         2          0
      Received:       7688         7682         6         19

----------------------------------------------------------------

show esmc counters interface ethernet

The output of this command displays counters for sent and received ESMC SSMs on a specific interface.

The following is an example of the output of the show esmc counters interface ethernet slot / port command:

ESMC Packet Counters of Interface Ethernet1/9:
    ESMC SSMs        Total  Information     Event    DNU/DUS
      Sent:           7955         7953         2          0
      Received:       7958         7952         6         19

----------------------------------------------------------------