Synchronous Ethernet ESMC and SSM

Table 1. Feature History Table

Feature name

Release Information

Feature Description

Enhanced SyncE and extended ESMC

Release 7.8.1

ITU-T G.8262.1 recommendation defines the requirements for timing devices used in synchronizing network equipment. For example, bandwidth, frequency accuracy, holdover, and noise generation. With eSyncE and eESMC support, the routers are capable of handling the following SyncE clocks on the network:

  • Enhanced ethernet equipment clock (eEEC)

  • Enhanced primary reference clock (ePRC)

  • Enhanced primary reference timing clock (ePRTC)

Support for Frequency Synchronization on the Cisco N560-IMA8Q interface module

Release 7.4.1

Based on the ITU-T G.8262 recommendations, precision frequency is enabled on timing devices to deliver frequency synchronization for bandwidth, frequency accuracy, holdover, and noise generation. This support allows for correct network operations when synchronous equipment is timed from either another synchronous equipment clock or a higher-quality clock.

Synchronous Ethernet is an extension of Ethernet designed to provide the reliability found in traditional SONET/SDH and T1/E1 networks to Ethernet packet networks by incorporating clock synchronization features. It supports the Synchronization Status Message (SSM) and Ethernet Synchronization Message Channel (ESMC) for synchronous Ethernet clock synchronization.

Synchronous Ethernet incorporates the Synchronization Status Message (SSM) used in Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) networks. While SONET and SDH transmit the SSM in a fixed location within the frame, Ethernet Synchronization Message Channel (ESMC) transmits the SSM using a protocol: the IEEE 802.3 Organization-Specific Slow Protocol (OSSP) standard.

The ESMC carries a Quality Level (QL) value identifying the clock quality of a given synchronous Ethernet timing source. Clock quality values help a synchronous Ethernet node derive timing from the most reliable source and prevent timing loops.

When configured to use synchronous Ethernet, the router synchronizes to the best available clock source. If no better clock sources are available, the router remains synchronized to the current clock source.

The router supports QL-enabled mode.

Frequency Synchronization Timing Concepts

The Cisco IOS XR frequency synchronization infrastructure is used to select between different frequency sources to set the router backplane frequency and time-of-day. There are two important concepts that must be understood with respect to the frequency synchronization implementation.

Sources

A source is a piece of hardware that inputs frequency signals into the system or transmits them out of the system. There are four types of sources:

  • Line interfaces: This includes SyncE interfaces.

  • Clock interfaces: These are external connectors for connecting other timing signals, such as, GPS, BITS.

  • PTP clock: If IEEE 1588 version 2 is configured on the router, a PTP clock may be available to frequency synchronization as a source of the time-of-day and frequency.

  • Internal oscillator: This is a free-running internal oscillator chip.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) or SSMs contained in the SONET/SDH frames so that devices know the best available source to synchronize to. In order to define a preferred network synchronization flow, and to help prevent timing loops, you can assign priority values to particular timing sources on each router. The combination of QL information and user-assigned priority levels allows each router to choose a timing source to use to clock its SyncE and SONET/SDH interfaces, as described in the ITU standard G.781.

Priority Levels

You can specify the priority of the frequency source on a controller or an interface. Values can range from 1 (highest priority) to 254 (lowest priority). The default value is 100. The priority is used in the clock-selection algorithm to choose between two sources that have the same quality level (QL). Lower priority values are preferred. For example, you can set the priority value for a GNSS clock source by using this command:

Router(config-gnss-freqsync)# priority 5

The router first considers the QL advertised by different timing sources. If two sources have the same QL, the router selects the source with a lower priority value for network synchronization.

Selection Points

A selection point is any point where a choice is made between several frequency signals, and possibly one or more of them are selected. Selection points form a graph representing the flow of timing signals between the different cards in a router running Cisco IOS XR software. For example, one or multiple selection points select between the different Synchronous Ethernet inputs available on a single line card, and the result of these selection points is forwarded to a selection point on the RSP to select between the selected source from each card.

The input signals to the selection points can be:

  • Received directly from a source.

  • The output from another selection point on the same card.

  • The output from a selection point on a different card.

The output of a selection point can be used in a number of ways:

  • Used to drive the signals sent out of a set of sources.

  • As input into another selection point on the card.

  • As input into a selection point on another card.

Use the show frequency synchronization selection command to see a detailed view of the different selection points within the system.

Restrictions

  • SyncE ESMC and SSM are not supported on 1G fibre interfaces of N540X-16Z4G8Q2C-A and N540X-16Z4G8Q2C-D variants.

Configuring Frequency Synchronization

Enabling Frequency Synchronization on the Router

This task describes the router-level configuration required to enable frequency synchronization.

RP/0/RP0/CPU0:Router# configure
RP/0/RP0/CPU0:Router(config)# frequency synchronization 
RP/0/RP0/CPU0:Router(config-freqsync)# clock-interface timing-mode system 
RP/0/RP0/CPU0:Router(config-freqsync)# quality itu-t option 1 generation 1 
RP/0/RP0/CPU0:Router(config-freqsync)# log selection changes 
RP/0/RP0/CPU0:Router(config-freqsync)# commit

Configuring Frequency Synchronization on an Interface

By default, there is no frequency synchronization on line interfaces. Use this task to configure an interface to participate in frequency synchronization.

Before You Begin

You must enable frequency synchronization globally on the router.

RP/0/RP0/CPU0:R1#config terminal
RP/0/RP0/CPU0:R1(config)#interface TenGigabitEthernet 0/0/0/0 
RP/0/RP0/CPU0:R1(config-if)#frequency synchronization 
RP/0/RP0/CPU0:R1(config-if-freqsync)#selection input 
RP/0/RP0/CPU0:R1(config-if-freqsync)#wait-to-restore 10 
RP/0/RP0/CPU0:R1(config-if-freqsync)#priority 5 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality transmit exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality receive exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#commit
or
RP/0/RP0/CPU0:router(config-freqsync)# commit

Configuring Frequency Synchronization on a Clock Interface

To enable a clock interface to be used as frequency input or output, you must configure the port parameters and frequency synchronization, as described in this task.

RP/0/RP0/CPU0:R1#configure
RP/0/RP0/CPU0:R1(config)# clock-interface sync 2 location 0/RP0/CPU0
RP/0/RP0/CPU0:R1(config-clock-if)# port-parameters
RP/0/RP0/CPU0:R1(config-clk-parms)# gps-input tod-format cisco pps-input tt1
RP/0/RP0/CPU0:R1(config-clk-parms)# exit
RP/0/RP0/CPU0:R1(config-clock-if)# frequency synchronization
RP/0/RP0/CPU0:R1(config-clk-freqsync)# selection input
RP/0/RP0/CPU0:R1(config-clk-freqsync)# wait-to-restore 1
RP/0/RP0/CPU0:R1(config-clk-freqsync)# quality receive exact itu-t option 1 PRC

SyncE Preference for PTP Receiver Interface

Table 2. Feature History Table

Feature Name

Release Information

Feature Description

SyncE Preference for PTP Receiver Interface

Release 24.4.1

You can now mitigate synchronization issues when SyncE and PTP sources come from different, non-traceable origins. This feature ensures that SyncE selection among sources with equal Quality Levels (QL) and user priority prefers the interface on which the PTP receiver is selected. If the PTP source fails or its quality degrades, causing the system to switch to another PTP source, SyncE switches to the new PTP source, provided the new interface has the same SyncE QL and priority as the previously selected interface.

This feature introduces these changes:

CLI:

In network deployments, users often utilize multiple clock sources to ensure redundancy, achieving synchronization through T-GM clocks powered by Primary Reference Time Clocks (PRTC) sources like GNSS. These clocks deliver frequency and time synchronization to other network nodes using both SyncE and PTP in hybrid mode. However, issues can arise when SyncE and PTP sources come from different, non-traceable origins, leading to synchronization failures in PTP.

To mitigate this issue, you can configure the synchronous-ethernet prefer-interface ptp-receiver command. When this command is configured, the SyncE selection among sources with equal Quality Levels (QL) and user priority prefers the interface on which the PTP receiver is selected.

  • If either the QL or the user priority of the sources differ, then the syncE selection would follow the G.781 requirements.

  • If the PTP source goes down or if the PTP quality degrades causing the system to switch to another PTP source, then with the above command, the SyncE selection also switches to the new PTP source, provided that the new interface has the same SyncE QL and priority as the previously selected interface.

Benefits of SyncE Preference for PTP Receiver Interface

  • Enhanced Synchronization - Ensures that SyncE and PTP sources are traceable to the same clock source, preventing synchronization issues.

  • Improved Reliability - Reduces the risk of PTP synchronization failures caused by rapid clock drifts.

  • Simplified Configuration - Automates the selection process of SyncE sources to align with the PTP receiver, simplifying network management.

Restrictions for SyncE Preference for PTP Receiver Interface

SyncE auto-selection to the PTP-receiver interface happens only if the interface is part of the show frequency synchronization selection list of SyncE interfaces. If more than two SyncE interfaces are configured, it is possible that the PTP-receiver interface isn't part of the selection list, and auto-switching doesn't occur.

Configure SyncE Preference for PTP Receiver Interface

Enable SyncE preference for PTP Receiver Interface.

Procedure

Step 1

Enter the frequency synchronization mode.

Example:
Router#configure
Router(config)#frequency synchronization 

Step 2

Configure SyncE to prefer the PTP receiver interface.

Example:
Router(config-freqsync)#synchronous-ethernet prefer-interface ptp-receiver 
Router(config-freqsync)#commit

Step 3

Verify that SyncE is configured to prefer the PTP receiver interface.

Example:
Router#show ptp interfaces brief 
Intf              Port         Port                  Line
Name              Number       State        Encap    State         Mechanism
--------------------------------------------------------------------------------
Te0/0/0/18        2            Passive      Ethernet up            -          
Te0/0/0/19        1            Slave        Ethernet up            -          

The show ptp interfaces breif command gives information about the ptp interfaces and then you can verify that the PTP receiver (Slave) interface is selected for frequency synchronization in show frequency synchronization interfaces brief command output.

Router#show frequency synchronization interfaces brief 

Flags:  > - Up                D - Down              S - Assigned for selection
        d - SSM Disabled      x - Peer timed out    i - Init state
        s - Output squelched
Fl   Interface                QLrcv QLuse Pri QLsnd Output driven by
==== ======================== ===== ===== === ===== ========================
>S   TenGigE0/0/0/18          PRC   PRC   100 DNU   TenGigE0/0/0/19 
>S   TenGigE0/0/0/19          PRC   PRC   100 PRC   TenGigE0/0/0/19 

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this task to check for configuration errors and verify the configuration.

  1. show frequency synchronization selection

    RP/0/RP0/CPU0:R5# show frequency synchronization selection
    Fri Apr 24 12:49:32.833 UTC
    Node 0/RP1/CPU0:
    ==============
    Selection point: T0-SEL-B (3 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : CHASSIS-TOD-SEL
        Chassis scoped: LC_TX_SELECT
        Router scoped : None
      Uses frequency selection
      Used for local line interface output
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         PTP [0/RP1/CPU0]          n/a                         PRC  254  Available  
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: 1588-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: CHASSIS-TOD-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses time-of-day selection
      S  Input                     Last Selection Point      Pri  Time  Status
      == ========================  ========================  ===  ====  ===========
      1  PTP [0/RP1/CPU0]          n/a                       100  Yes   Available  
         HundredGigE0/7/0/0        0/RP1/CPU0 T0-SEL-B 4     100  No    Available  
    
    Selection point: ETH_RXMUX (1 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : T0-SEL-B 1588-SEL
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        n/a                         PRC   10  Available  
    
    
  2. show frequency synchronization configuration-errors

    RP/0/RP0/CPU0:router# show frequency synchronization configuration-errors
    Node 0/2/CPU0:
    ==============
      interface GigabitEthernet0/2/0/0 frequency synchronization
        * Frequency synchronization is enabled on this interface, but isn't enabled globally.
    interface GigabitEthernet0/2/0/0 frequency synchronization quality transmit exact itu-t option 2 generation 1 PRS
        * The QL that is configured is from a different QL option set than is configured
    globally.

    Displays any errors that are caused by inconsistencies between shared-plane (global) and local-plane (interface) configurations. There are two possible errors that can be displayed:

    • Frequency Synchronization is configured on an interface (line interface or clock-interface), but is not configured globally.

    • The QL option configured on some interface does not match the global QL option. Under an interface (line interface or clock interface), the QL option is specified using the quality transmit and quality receive commands. The value specified must match the value configured in the global quality itu-t option command, or match the default (option 1) if the global quality itu-t option command is not configured.

    Once all the errors have been resolved, meaning there is no output from the command, continue to the next step.

  3. show frequency synchronization interfaces brief

    RP/0/RP0/CPU0:R5# show frequency synchronization interfaces brief
    Thu Feb 1 06:30:02.945 UTC
    Flags: > - Up
            d - SSM Disabled
            s - Output squelched
    Fl   Interface
    D - Down              S - Assigned for selection
     x - Peer timed out    i - Init state
    Last Selection Point
    Pri Time
    Status
                                  QLrcv QLuse Pri QLsnd Output driven by
    ==== ======================== ===== ===== === ===== ========================
    >S   TenGigE0/0/0/0           PRC   PRC     1 DNU   TenGigE0/0/0/0
    >x   TenGigE0/0/0/1           Fail  n/a   100 PRC   TenGigE0/0/0/0
    >x   TwentyFiveGigE0/0/0/30   Fail  n/a   100 PRC   TenGigE0/0/0/0
    
    RP/0/RP0/CPU0:R5#

    Verifies the configuration. Note the following points:

    • All line interface that have frequency synchronization configured are displayed.

    • All clock interfaces are displayed.

    • Sources that have been nominated as inputs (in other words, have selection input configured) have ‘S’ in the Flags column; sources that have not been nominated as inputs do not have ‘S’ displayed.


      Note


      Internal oscillators are always eligible as inputs.


    • ‘>’ or ‘D’ is displayed in the flags field as appropriate.

    If any of these items are not true, continue to the next step.

  4. show processes fsyncmgr location node-id

    This command verifies that the fsyncmgr process is running on the appropriate nodes.

    RP/0/RP0/CPU0:R5# show processes fsyncmgr location 0/0/cPU0
    Thu Feb 1 06:26:32.979 UTC
    Job Id: 181
    PID: HYPERLINK "tel:3411"3411
    Process name: fsyncmgr
    Executable path: /opt/cisco/XR/packages/ncs560-iosxr-fwding-1.0.0.0-r63226I/all/bin/fsyncmgr Instance #: 1
    Version ID: 00.00.0000
    Respawn: ON
    Respawn count: 1
    Last started: Tue Jan 23 04:26:57 HYPERLINK "tel:2018"2018
    Process state: Run
    Package state: Normal
    core: MAINMEM
    Max. core: 0
    Level: 100
    Placement: None
    startup_path: /opt/cisco/XR/packages/ncs560-iosxr-fwding-1.0.0.0-r63226I/all/startup/fsyncmgr.startup Ready: 2.063s
    Process cpu time: 168.480 user, 129.980 kernel, 298.460 total
    JID TID Stack pri state NAME rt_pri
    181 HYPERLINK "tel:3411"3411 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3572"3572 0K 20 Sleeping lwm_debug_threa 0
    181 HYPERLINK "tel:3573"3573 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3574"3574 0K 20 Sleeping lwm_service_thr 0
    181 HYPERLINK "tel:3575"3575 0K 20 Sleeping qsm_service_thr 0
    181 HYPERLINK "tel:3622"3622 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3781"3781 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3789"3789 0K 20 Sleeping fsyncmgr 0

Verifying the ESMC Configuration

show frequency synchronization interfaces

RP/0/RP0/CPU0:R5# show frequency synchronization interfaces
                    Thu Feb 1 06:33:26.575 UTC
                    Interface TenGigE0/0/0/0 (up)
                    Assigned as input for selection
                    Wait-to-restore time 0 minutes
                    SSM Enabled
Peer Up for 2d01h, last SSM received 0.320s ago
Peer has come up 1 times and timed out 0 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 HYPERLINK "tel:178463"178463
                    Received: HYPERLINK "tel:178499"178499 HYPERLINK "tel:178499"178499 0 0
                    Input:
                    Up
                    Last received QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC, Priority: 1, Time-of-day Priority 100
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: DNU
                    Next selection points: ETH_RXMUX
                    Interface TenGigE0/0/0/1 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 2d01h, last SSM received never
                    Peer has come up 0 times and timed out 1 times
                    ESMC SSMs Total Information Event DNU/DUS
                    Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 0
                    Received: 0 0 0 0
                    Input:
                    Down - not assigned for selection
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC
                    Next selection points: ETH_RXMUX
                    Interface TwentyFiveGigE0/0/0/30 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 01:50:24, last SSM received 01:50:30 ago
                    Peer has come up 1 times and timed out 1 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:75086"75086 HYPERLINK "tel:75085"75085 1 0
Received: HYPERLINK "tel:68457"68457 HYPERLINK "tel:68455"68455 2 HYPERLINK "tel:68443"68443
Input:
Down - not assigned for selection
Supports frequency
Output:
Selected source: TenGigE0/0/0/0
Selected source QL: Opt-I/PRC
Effective QL: Opt-I/PRC
Next selection points: ETH_RXMUX

Verifying Synchronous Ethernet LEDs

RP/0/RP0/CPU0:ios# show led
Mon Nov 2 09:29:34.637 UTC
================================================================================
Location LED Name Mode Color
================================================================================
0/FT0
Status OPERATIONAL GREEN
0/RP0/CPU0
Alarm OPERATIONAL AMBER
Status OPERATIONAL GREEN
Sync OPERATIONAL AMBER