Synchronous Ethernet ESMC and SSM

Synchronous Ethernet is an extension of Ethernet designed to provide the reliability found in traditional SONET/SDH and T1/E1 networks to Ethernet packet networks by incorporating clock synchronization features. It supports the Synchronization Status Message (SSM) and Ethernet Synchronization Message Channel (ESMC) for synchronous Ethernet clock synchronization.

Synchronous Ethernet incorporates the Synchronization Status Message (SSM) used in Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) networks. While SONET and SDH transmit the SSM in a fixed location within the frame, Ethernet Synchronization Message Channel (ESMC) transmits the SSM using a protocol: the IEEE 802.3 Organization-Specific Slow Protocol (OSSP) standard.

The ESMC carries a Quality Level (QL) value identifying the clock quality of a given synchronous Ethernet timing source. Clock quality values help a synchronous Ethernet node derive timing from the most reliable source and prevent timing loops.

When configured to use synchronous Ethernet, the router synchronizes to the best available clock source. If no better clock sources are available, the router remains synchronized to the current clock source.

The router supports QL-enabled mode.

Frequency Synchronization Timing Concepts

The Cisco IOS XR frequency synchronization infrastructure is used to select between different frequency sources to set the router backplane frequency and time-of-day. There are two important concepts that must be understood with respect to the frequency synchronization implementation.

Sources

A source is a piece of hardware that inputs frequency signals into the system or transmits them out of the system. There are four types of sources:

  • Line interfaces. This includes SyncE interfaces.

  • Clock interfaces. These are external connectors for connecting other timing signals, such as, GPS, BITS.

  • PTP clock. If IEEE 1588 version 2 is configured on the router, a PTP clock may be available to frequency synchronization as a source of the time-of-day and frequency.

  • Internal oscillator. This is a free-running internal oscillator chip.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) or SSMs contained in the SONET/SDH frames so that devices know the best available source to synchronize to. In order to define a preferred network synchronization flow, and to help prevent timing loops, you can assign priority values to particular timing sources on each router. The combination of QL information and user-assigned priority levels allows each router to choose a timing source to use to clock its SyncE and SONET/SDH interfaces, as described in the ITU standard G.781.

Selection Points

A selection point is any point where a choice is made between several frequency signals, and possibly one or more of them are selected. Selection points form a graph representing the flow of timing signals between the different cards in a router running Cisco IOS XR software. For example, one or multiple selection points select between the different Synchronous Ethernet inputs available on a single line card, and the result of these selection points is forwarded to a selection point on the RSP to select between the selected source from each card.

The input signals to the selection points can be:

  • Received directly from a source.

  • The output from another selection point on the same card.

  • The output from a selection point on a different card.

The output of a selection point can be used in a number of ways:

  • Used to drive the signals sent out of a set of sources.

  • As input into another selection point on the card.

  • As input into a selection point on another card.

Use the show frequency synchronization selection command to see a detailed view of the different selection points within the system.


Note

  • We recommend you to configure, and enable Frequency Synchronization selection input on two interfaces per line card.

  • For link aggregation, you must configure and enable Frequency Synchronization selection input on a single bundle member.

Restrictions

  • SyncE is not supported on Gigabit Ethernet 0/0/0/24 to 0/0/0/31 ports.

  • The Precision Time Protocol (PTP) session flaps during Route Processor Failover (RPFO).

SyncE Hardware Support


Note

The table also contains support details of upcoming releases. You can read this table in context of the current release and see relevant Release Notes for more information on supported features and hardware.

This table provides a detailed information on the timing features that are supported on the following hardware variants.

Hardware Variant

Features

Cisco IOS XR Release

Comments

NCS-57C3-MOD-SYS

NCS-57C3-MODS-SYS

E-SyncE

Release 7.9.1

NC57-24DD

SyncE

Release 7.5.1

NC57-18DD-SE

SyncE

Release 7.5.1

NCS-57C1-48Q6-SYS

SyncE

Release 7.5.1

E-SyncE

Release 7.9.1

RP:NC57-MOD-RP2-E with NCS-57C3-MOD-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 40-55, and not on MPA.

RP:NC57-MOD-RP2-E with NCS-57C3-MODS-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 36-51, and not on MPA.

RP: NC55-RP2-E

Line card: NC57-36H6D-S

SyncE

Release 7.3.2

  • Release 7.3.2 - Supports Compatible Mode only

  • Release 7.7.1 - Supports both Native and Compatible mode.

  • SyncE is not supported on 100GE interfaces, when they are used in 1G mode.

NCS-57B1-5DSE-SYS

NCS-57B1-6D24-SYS

SyncE

Release 7.3.1

E-syncE

Release 7.9.1

RP:NC55-RP-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line card:NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-SE-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24Q6H-S

NCS-55A1-24Q6H-SS

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-48Q6H

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24H

SyncE

Release 6.5.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS55A2-MOD

SyncE

Release 6.5.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

E-syncE

Release 7.9.1

E-SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-MOD-A-S

SyncE

Release 6.5.1

SyncE is not supported on 100GE interfaces, when theyare used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-36X100G-A-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS5501-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

SyncE is supported on 10G from ports 8 to 15, but it is not supported on these ports in 1G mode.

SyncE features are supported on the following MPAs:

  • NC55-MPA-2TH-S

  • NC55-MPA-1TH2H-S

  • NC55-MPA-1TH2H-HD-S

  • NC55-MPA-4H-S

  • NC55-MPA-4H-HD-S

  • NC55-MPA-12T-S

Configuring Frequency Synchronization

Enabling Frequency Synchronization on the Router

This task describes the router-level configuration required to enable frequency synchronization.

RP/0/RP0/CPU0:Router# configure
RP/0/RP0/CPU0:Router(config)# frequency synchronization 
RP/0/RP0/CPU0:Router(config-freqsync)# clock-interface timing-mode system 
RP/0/RP0/CPU0:Router(config-freqsync)# quality itu-t option 1 generation 1 
RP/0/RP0/CPU0:Router(config-freqsync)# log selection changes 
RP/0/RP0/CPU0:Router(config-freqsync)# commit

Configuring Frequency Synchronization on an Interface

By default, there is no frequency synchronization on line interfaces. Use this task to configure an interface to participate in frequency synchronization.

Before You Begin

You must enable frequency synchronization globally on the router.

RP/0/RP0/CPU0:R1#config terminal
RP/0/RP0/CPU0:R1(config)#interface TenGigabitEthernet 0/0/0/0 
RP/0/RP0/CPU0:R1(config-if)#frequency synchronization 
RP/0/RP0/CPU0:R1(config-if-freqsync)#selection input 
RP/0/RP0/CPU0:R1(config-if-freqsync)#wait-to-restore 10 
RP/0/RP0/CPU0:R1(config-if-freqsync)#priority 5 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality transmit exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality receive exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#commit
or
RP/0/RP0/CPU0:router(config-freqsync)# commit

Configuring Frequency Synchronization on a Clock Interface

To enable a clock interface to be used as frequency input or output, you must configure the port parameters and frequency synchronization, as described in this task. 

RP/0/RP0/CPU0:R1#configure
RP/0/RP0/CPU0:R1(config)# clock-interface sync 2 location 0/RP0/CPU0
RP/0/RP0/CPU0:R1(config-clock-if)# port-parameters
RP/0/RP0/CPU0:R1(config-clk-parms)# gps-input tod-format cisco pps-input tt1
RP/0/RP0/CPU0:R1(config-clk-parms)# exit
RP/0/RP0/CPU0:R1(config-clock-if)# frequency synchronization
RP/0/RP0/CPU0:R1(config-clk-freqsync)# selection input
RP/0/RP0/CPU0:R1(config-clk-freqsync)# wait-to-restore 1
RP/0/RP0/CPU0:R1(config-clk-freqsync)# quality receive exact itu-t option 1 PRC

SyncE Preference for PTP Receiver Interface

Table 1. Feature History Table

Feature Name

Release Information

Feature Description

SyncE Preference for PTP Receiver Interface

Release 24.4.1

Introduced in this release on: NCS 5500 fixed port routers; NCS 5700 fixed port routers; NCS 5500 modular routers (NCS 5500 line cards; NCS 5700 line cards [Mode: Compatibility; Native])

You can now mitigate synchronization issues when SyncE and PTP sources come from different, non-traceable origins. This feature ensures that SyncE selection among sources with equal Quality Levels (QL) and user priority prefers the interface on which the PTP receiver is selected. If the PTP source fails or its quality degrades, causing the system to switch to another PTP source, SyncE switches to the new PTP source, provided the new interface has the same SyncE QL and priority as the previously selected interface.

This feature introduces synchronous-ethernet prefer-interface ptp-receiver command.

In network deployments, users often utilize multiple clock sources to ensure redundancy, achieving synchronization through T-GM clocks powered by Primary Reference Time Clocks (PRTC) sources like GNSS. These clocks deliver frequency and time synchronization to other network nodes using both SyncE and PTP in hybrid mode. However, issues can arise when SyncE and PTP sources come from different, non-traceable origins, leading to synchronization failures in PTP.

To mitigate this issue, you can configure the synchronous-ethernet prefer-interface ptp-receiver command. When this command is configured, the SyncE selection among sources with equal Quality Levels (QL) and user priority prefers the interface on which the PTP receiver is selected.

  • If either the QL or the user priority of the sources differ, then the syncE selection would follow the G.781 requirements.

  • If the PTP source goes down or if the PTP quality degrades causing the system to switch to another PTP source, then with the above command, the SyncE selection also switches to the new PTP source, provided that the new interface has the same SyncE QL and priority as the previously selected interface.

Benefits of SyncE Preference for PTP Receiver Interface

  • Enhanced Synchronization - Ensures that SyncE and PTP sources are traceable to the same clock source, preventing synchronization issues.

  • Improved Reliability - Reduces the risk of PTP synchronization failures caused by rapid clock drifts.

  • Simplified Configuration - Automates the selection process of SyncE sources to align with the PTP receiver, simplifying network management.

Restrictions for SyncE Preference for PTP Receiver Interface

SyncE auto-selection to the PTP-receiver interface happens only if the interface is part of the show frequency synchronization selection list of SyncE interfaces. If more than two SyncE interfaces are configured, it is possible that the PTP-receiver interface isn't part of the selection list, and auto-switching doesn't occur.

Configure SyncE Preference for PTP Receiver Interface

Enable SyncE preference for PTP Receiver Interface.

Procedure

Step 1

Enter the frequency synchronization mode.

Example:

Router#configure
Router(config)#frequency synchronization

Step 2

Configure SyncE to prefer the PTP receiver interface.

Example:

Router(config-freqsync)#synchronous-ethernet prefer-interface ptp-receiver 
Router(config-freqsync)#commit

Step 3

Verify that SyncE is configured to prefer the PTP receiver interface.

Example:

Router#show ptp interfaces brief 
Intf              Port         Port                  Line
Name              Number       State        Encap    State         Mechanism
--------------------------------------------------------------------------------
Te0/0/0/18        2            Passive      Ethernet up            -          
Te0/0/0/19        1            Slave        Ethernet up            -

The show ptp interfaces breif command gives information about the ptp interfaces and then you can verify that the PTP receiver (Slave) interface is selected for frequency synchronization in show frequency synchronization interfaces brief command output. 

Router#show frequency synchronization interfaces brief 

Flags:  > - Up                D - Down              S - Assigned for selection
        d - SSM Disabled      x - Peer timed out    i - Init state
        s - Output squelched
Fl   Interface                QLrcv QLuse Pri QLsnd Output driven by
==== ======================== ===== ===== === ===== ========================
>S   TenGigE0/0/0/18          PRC   PRC   100 DNU   TenGigE0/0/0/19 
>S   TenGigE0/0/0/19          PRC   PRC   100 PRC   TenGigE0/0/0/19

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this task to check for configuration errors and verify the configuration.

  1. show frequency synchronization selection 

    RP/0/RP0/CPU0:R5# show frequency synchronization selection
Fri Apr 24 12:49:32.833 UTC
Node 0/RP1/CPU0:
==============
Selection point: T0-SEL-B (3 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : CHASSIS-TOD-SEL
    Chassis scoped: LC_TX_SELECT
    Router scoped : None
  Uses frequency selection
  Used for local line interface output
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
     PTP [0/RP1/CPU0]          n/a                         PRC  254  Available  
     Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  

Selection point: 1588-SEL (2 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : None
    Chassis scoped: None
    Router scoped : None
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
     Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  

Selection point: CHASSIS-TOD-SEL (2 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : None
    Chassis scoped: None
    Router scoped : None
  Uses time-of-day selection
  S  Input                     Last Selection Point      Pri  Time  Status
  == ========================  ========================  ===  ====  ===========
  1  PTP [0/RP1/CPU0]          n/a                       100  Yes   Available  
     HundredGigE0/7/0/0        0/RP1/CPU0 T0-SEL-B 4     100  No    Available  

Selection point: ETH_RXMUX (1 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : T0-SEL-B 1588-SEL
    Chassis scoped: None
    Router scoped : None
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        n/a                         PRC   10  Available

  2. show frequency synchronization configuration-errors 

    RP/0/RP0/CPU0:router# show frequency synchronization configuration-errors
Node 0/2/CPU0:
==============
  interface GigabitEthernet0/2/0/0 frequency synchronization
    * Frequency synchronization is enabled on this interface, but isn't enabled globally.
interface GigabitEthernet0/2/0/0 frequency synchronization quality transmit exact itu-t option 2 generation 1 PRS
    * The QL that is configured is from a different QL option set than is configured
globally.

    Displays any errors that are caused by inconsistencies between shared-plane (global) and local-plane (interface) configurations. There are two possible errors that can be displayed:

    • Frequency Synchronization is configured on an interface (line interface or clock-interface), but is not configured globally.

    • The QL option configured on some interface does not match the global QL option. Under an interface (line interface or clock interface), the QL option is specified using the quality transmit and quality receive commands. The value specified must match the value configured in the global quality itu-t option command, or match the default (option 1) if the global quality itu-t option command is not configured.

    Once all the errors have been resolved, meaning there is no output from the command, continue to the next step.

  3. show frequency synchronization interfaces brief 

    RP/0/RP0/CPU0:R5# show frequency synchronization interfaces brief
Flags: > - Up
        d - SSM Disabled
        s - Output squelched
Fl   Interface
D - Down              S - Assigned for selection
 x - Peer timed out    i - Init state
Last Selection Point
Pri Time
Status
                              QLrcv QLuse Pri QLsnd Output driven by
==== ======================== ===== ===== === ===== ========================
>S   TenGigE0/0/0/0           PRC   PRC     1 DNU   TenGigE0/0/0/0
>x   TenGigE0/0/0/1           Fail  n/a   100 PRC   TenGigE0/0/0/0
>x   TwentyFiveGigE0/0/0/30   Fail  n/a   100 PRC   TenGigE0/0/0/0

RP/0/RP0/CPU0:R5#

    Verifies the configuration. Note the following points:

    • All line interface that have frequency synchronization configured are displayed.

    • All clock interfaces and internal oscillators are displayed.

    • Sources that have been nominated as inputs (in other words, have selection input configured) have ‘S’ in the Flags column; sources that have not been nominated as inputs do not have ‘S’ displayed.


      Note

      Internal oscillators are always eligible as inputs.

    • ‘>’ or ‘D’ is displayed in the flags field as appropriate.

    If any of these items are not true, continue to the next step.

  4. show processes fsyncmgr location node-id

    This command verifies that the fsyncmgr process is running on the appropriate nodes.

    RP/0/RP0/CPU0:R5# show processes fsyncmgr location 0/0/cPU0
Thu Feb 1 06:26:32.979 UTC
Job Id: 181
PID: HYPERLINK "tel:3411"3411
Process name: fsyncmgr
Executable path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/bin/fsyncmgr Instance #: 1
Version ID: 00.00.0000
Respawn: ON
Respawn count: 1
Last started: Tue Jan 23 04:26:57 HYPERLINK "tel:2018"2018
Process state: Run
Package state: Normal
core: MAINMEM
Max. core: 0
Level: 100
Placement: None
startup_path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/startup/fsyncmgr.startup Ready: 2.063s
Process cpu time: 168.480 user, 129.980 kernel, 298.460 total
JID TID Stack pri state NAME rt_pri
181 HYPERLINK "tel:3411"3411 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3572"3572 0K 20 Sleeping lwm_debug_threa 0
181 HYPERLINK "tel:3573"3573 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3574"3574 0K 20 Sleeping lwm_service_thr 0
181 HYPERLINK "tel:3575"3575 0K 20 Sleeping qsm_service_thr 0
181 HYPERLINK "tel:3622"3622 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3781"3781 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3789"3789 0K 20 Sleeping fsyncmgr 0

Verifying the ESMC Configuration

show frequency synchronization interfaces 

Router# show frequency synchronization interfaces
                    Interface TenGigE0/0/0/0 (up)
                    Assigned as input for selection
                    Wait-to-restore time 0 minutes
                    SSM Enabled
Peer Up for 2d01h, last SSM received 0.320s ago
Peer has come up 1 times and timed out 0 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 HYPERLINK "tel:178463"178463
                    Received: HYPERLINK "tel:178499"178499 HYPERLINK "tel:178499"178499 0 0
                    Input:
                    Up
                    Last received QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC, Priority: 1, Time-of-day Priority 100
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: DNU
                    Next selection points: ETH_RXMUX
                    Interface TenGigE0/0/0/1 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 2d01h, last SSM received never
                    Peer has come up 0 times and timed out 1 times
                    ESMC SSMs Total Information Event DNU/DUS
                    Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 0
                    Received: 0 0 0 0
                    Input:
                    Down - not assigned for selection
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC
                    Next selection points: ETH_RXMUX
                    Interface TwentyFiveGigE0/0/0/30 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 01:50:24, last SSM received 01:50:30 ago
                    Peer has come up 1 times and timed out 1 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:75086"75086 HYPERLINK "tel:75085"75085 1 0
Received: HYPERLINK "tel:68457"68457 HYPERLINK "tel:68455"68455 2 HYPERLINK "tel:68443"68443
Input:
Down - not assigned for selection
Supports frequency
Output:
Selected source: TenGigE0/0/0/0
Selected source QL: Opt-I/PRC
Effective QL: Opt-I/PRC
Next selection points: ETH_RXMUX

Verifying Controllers Timing LEDs

Router# show controllers timing led status location 0/RP0/CPU0
LED Status:
        BITS0: Off
        BITS1: Off
         Sync: Green
         GNSS: Off
          GPS: NA