Synchronous Ethernet ESMC and SSM

Synchronous Ethernet is an extension of Ethernet designed to provide the reliability found in traditional SONET/SDH and T1/E1 networks to Ethernet packet networks by incorporating clock synchronization features. It supports the Synchronization Status Message (SSM) and Ethernet Synchronization Message Channel (ESMC) for synchronous Ethernet clock synchronization.

Synchronous Ethernet incorporates the Synchronization Status Message (SSM) used in Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) networks. While SONET and SDH transmit the SSM in a fixed location within the frame, Ethernet Synchronization Message Channel (ESMC) transmits the SSM using a protocol: the IEEE 802.3 Organization-Specific Slow Protocol (OSSP) standard.

The ESMC carries a Quality Level (QL) value identifying the clock quality of a given synchronous Ethernet timing source. Clock quality values help a synchronous Ethernet node derive timing from the most reliable source and prevent timing loops.

When configured to use synchronous Ethernet, the router synchronizes to the best available clock source. If no better clock sources are available, the router remains synchronized to the current clock source.

The router supports QL-enabled mode.

Frequency Synchronization Timing Concepts

The Cisco IOS XR frequency synchronization infrastructure is used to select between different frequency sources to set the router backplane frequency and time-of-day. There are two important concepts that must be understood with respect to the frequency synchronization implementation.

Sources

A source is a piece of hardware that inputs frequency signals into the system or transmits them out of the system. There are four types of sources:

  • Line interfaces. This includes SyncE interfaces.

  • Clock interfaces. These are external connectors for connecting other timing signals, such as, GPS, BITS.

  • PTP clock. If IEEE 1588 version 2 is configured on the router, a PTP clock may be available to frequency synchronization as a source of the time-of-day and frequency.

  • Internal oscillator. This is a free-running internal oscillator chip.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) or SSMs contained in the SONET/SDH frames so that devices know the best available source to synchronize to. In order to define a preferred network synchronization flow, and to help prevent timing loops, you can assign priority values to particular timing sources on each router. The combination of QL information and user-assigned priority levels allows each router to choose a timing source to use to clock its SyncE and SONET/SDH interfaces, as described in the ITU standard G.781.

Selection Points

A selection point is any point where a choice is made between several frequency signals, and possibly one or more of them are selected. Selection points form a graph representing the flow of timing signals between the different cards in a router running Cisco IOS XR software. For example, one or multiple selection points select between the different Synchronous Ethernet inputs available on a single line card, and the result of these selection points is forwarded to a selection point on the RSP to select between the selected source from each card.

The input signals to the selection points can be:

  • Received directly from a source.

  • The output from another selection point on the same card.

  • The output from a selection point on a different card.

The output of a selection point can be used in a number of ways:

  • Used to drive the signals sent out of a set of sources.

  • As input into another selection point on the card.

  • As input into a selection point on another card.

Use the show frequency synchronization selection command to see a detailed view of the different selection points within the system.


Note

  • We recommend you to configure, and enable Frequency Synchronization selection input on two interfaces per line card.

  • For link aggregation, you must configure and enable Frequency Synchronization selection input on a single bundle member.

Restrictions

  • SyncE is not supported on Gigabit Ethernet 0/0/0/24 to 0/0/0/31 ports.

  • The Precision Time Protocol (PTP) session flaps during Route Processor Failover (RPFO).

SyncE Hardware Support Matrix


Note

The table also contains support details of upcoming releases. You can read this table in context of the current release and see relevant Release Notes for more information on supported features and hardware.

This table provides a detailed information on the timing features that are supported on the following hardware variants.

Hardware Variant

Features

Cisco IOS XR Release

Comments

NCS-57D2-18DD-SYS

SyncE

Release 7.8.1

With this release,SyncE is supported on 400G, 100G and 40G ports.

NCS-57C3-MOD-SYS

NCS-57C3-MODS-SYS

E-SyncE

Release 7.9.1

NC57-24DD

SyncE

Release 7.5.1

NC57-18DD-SE

SyncE

Release 7.5.1

NCS-57C1-48Q6-SYS

SyncE

Release 7.5.1

E-SyncE

Release 7.9.1

RP:NC57-MOD-RP2-E with NCS-57C3-MOD-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 40-55, and not on MPA.

RP:NC57-MOD-RP2-E with NCS-57C3-MODS-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 36-51, and not on MPA.

RP: NC55-RP2-E

Line card: NC57-36H6D-S

SyncE

Release 7.3.2

  • Release 7.3.2 - Supports Compatible Mode only

  • Release 7.7.1 - Supports both Native and Compatible mode.

  • SyncE is not supported on 100GE interfaces, when they are used in 1G mode.

NCS-57B1-5DSE-SYS

NCS-57B1-6D24-SYS

SyncE

Release 7.3.1

E-syncE

Release 7.9.1

RP:NC55-RP-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line card:NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-SE-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24Q6H-S

NCS-55A1-24Q6H-SS

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-48Q6H

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24H

SyncE

Release 6.5.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS55A2-MOD

SyncE

Release 6.5.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

E-syncE

Release 7.9.1

E-SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-MOD-A-S

SyncE

Release 6.5.1

SyncE is not supported on 100GE interfaces, when theyare used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-36X100G-A-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS5501-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

SyncE is supported on 10G from ports 8 to 15, but it is not supported on these ports in 1G mode.

SyncE features are supported on the following MPAs:

  • NC57-MPA-12L-S

  • NC55-MPA-2TH-S

  • NC55-MPA-1TH2H-S

  • NC55-MPA-1TH2H-HD-S

  • NC55-MPA-4H-S

  • NC55-MPA-4H-HD-S

  • NC55-MPA-12T-S

Table 1. Timing Features Supported on MPAs

Hardware Variant

Features

Cisco IOS XR Release

Comments

NC57-MPA-1FH1D-S

SyncE

Release 7.8.1

Class-C is supported only on QSFP 100G and 40G ports in native mode.

Router processor: NC57-MOD-RP2-E with NCS-57C3-MOD-S and NCS-57C3-MOD-SE-S

This feature provides only functionality support for SyncE on QSFP 400G/200G and CFP2 DCO ports.

NC55-MPA-4H-S

SyncE

Release 7.7.1

Class B

NC55-MPA-2TH-S

SyncE

Release 7.7.1

SyncE is supported on 100G and 200G mode.

Route Processor:RP2-E

NC55-MPA-1TH2H-S

SyncE

Release 7.7.1

SyncE is supported on 100G mode.

Route Processor:RP2-E

Class C is not supported on 200G port.

NC57-MPA-12L-S

SyncE

SyncE clock recovery

 Release 7.6.1 Route Processors:

  • RP-E: Class B

  • RP2-E Enhanced: Class C

Configuring Frequency Synchronization

Enabling Frequency Synchronization on the Router

This task describes the router-level configuration required to enable frequency synchronization.

RP/0/RP0/CPU0:Router# configure
RP/0/RP0/CPU0:Router(config)# frequency synchronization 
RP/0/RP0/CPU0:Router(config-freqsync)# clock-interface timing-mode system 
RP/0/RP0/CPU0:Router(config-freqsync)# quality itu-t option 1 generation 1 
RP/0/RP0/CPU0:Router(config-freqsync)# log selection changes 
RP/0/RP0/CPU0:Router(config-freqsync)# commit

Configuring Frequency Synchronization on an Interface

By default, there is no frequency synchronization on line interfaces. Use this task to configure an interface to participate in frequency synchronization.

Before You Begin

You must enable frequency synchronization globally on the router.

RP/0/RP0/CPU0:R1#config terminal
RP/0/RP0/CPU0:R1(config)#interface TenGigabitEthernet 0/0/0/0 
RP/0/RP0/CPU0:R1(config-if)#frequency synchronization 
RP/0/RP0/CPU0:R1(config-if-freqsync)#selection input 
RP/0/RP0/CPU0:R1(config-if-freqsync)#wait-to-restore 10 
RP/0/RP0/CPU0:R1(config-if-freqsync)#priority 5 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality transmit exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality receive exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#commit
or
RP/0/RP0/CPU0:router(config-freqsync)# commit

Configuring Frequency Synchronization on a Clock Interface

To enable a clock interface to be used as frequency input or output, you must configure the port parameters and frequency synchronization, as described in this task. 

RP/0/RP0/CPU0:R1#configure
RP/0/RP0/CPU0:R1(config)# clock-interface sync 2 location 0/RP0/CPU0
RP/0/RP0/CPU0:R1(config-clock-if)# port-parameters
RP/0/RP0/CPU0:R1(config-clk-parms)# gps-input tod-format cisco pps-input tt1
RP/0/RP0/CPU0:R1(config-clk-parms)# exit
RP/0/RP0/CPU0:R1(config-clock-if)# frequency synchronization
RP/0/RP0/CPU0:R1(config-clk-freqsync)# selection input
RP/0/RP0/CPU0:R1(config-clk-freqsync)# wait-to-restore 1
RP/0/RP0/CPU0:R1(config-clk-freqsync)# quality receive exact itu-t option 1 PRC

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this task to check for configuration errors and verify the configuration.

  1. show frequency synchronization selection 

    RP/0/RP0/CPU0:R5# show frequency synchronization selection
Fri Apr 24 12:49:32.833 UTC
Node 0/RP1/CPU0:
==============
Selection point: T0-SEL-B (3 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : CHASSIS-TOD-SEL
    Chassis scoped: LC_TX_SELECT
    Router scoped : None
  Uses frequency selection
  Used for local line interface output
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
     PTP [0/RP1/CPU0]          n/a                         PRC  254  Available  
     Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  

Selection point: 1588-SEL (2 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : None
    Chassis scoped: None
    Router scoped : None
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
     Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  

Selection point: CHASSIS-TOD-SEL (2 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : None
    Chassis scoped: None
    Router scoped : None
  Uses time-of-day selection
  S  Input                     Last Selection Point      Pri  Time  Status
  == ========================  ========================  ===  ====  ===========
  1  PTP [0/RP1/CPU0]          n/a                       100  Yes   Available  
     HundredGigE0/7/0/0        0/RP1/CPU0 T0-SEL-B 4     100  No    Available  

Selection point: ETH_RXMUX (1 inputs, 1 selected)
  Last programmed 3d04h ago, and selection made 3d04h ago
  Next selection points
    SPA scoped    : None
    Node scoped   : T0-SEL-B 1588-SEL
    Chassis scoped: None
    Router scoped : None
  Uses frequency selection
  S  Input                     Last Selection Point         QL  Pri  Status
  == ========================  ========================  =====  ===  ===========
  4  HundredGigE0/7/0/0        n/a                         PRC   10  Available

  2. show frequency synchronization configuration-errors 

    RP/0/RP0/CPU0:router# show frequency synchronization configuration-errors
Node 0/2/CPU0:
==============
  interface GigabitEthernet0/2/0/0 frequency synchronization
    * Frequency synchronization is enabled on this interface, but isn't enabled globally.
interface GigabitEthernet0/2/0/0 frequency synchronization quality transmit exact itu-t option 2 generation 1 PRS
    * The QL that is configured is from a different QL option set than is configured
globally.

    Displays any errors that are caused by inconsistencies between shared-plane (global) and local-plane (interface) configurations. There are two possible errors that can be displayed:

    • Frequency Synchronization is configured on an interface (line interface or clock-interface), but is not configured globally.

    • The QL option configured on some interface does not match the global QL option. Under an interface (line interface or clock interface), the QL option is specified using the quality transmit and quality receive commands. The value specified must match the value configured in the global quality itu-t option command, or match the default (option 1) if the global quality itu-t option command is not configured.

    Once all the errors have been resolved, meaning there is no output from the command, continue to the next step.

  3. show frequency synchronization interfaces brief 

    RP/0/RP0/CPU0:R5# show frequency synchronization interfaces brief
Flags: > - Up
        d - SSM Disabled
        s - Output squelched
Fl   Interface
D - Down              S - Assigned for selection
 x - Peer timed out    i - Init state
Last Selection Point
Pri Time
Status
                              QLrcv QLuse Pri QLsnd Output driven by
==== ======================== ===== ===== === ===== ========================
>S   TenGigE0/0/0/0           PRC   PRC     1 DNU   TenGigE0/0/0/0
>x   TenGigE0/0/0/1           Fail  n/a   100 PRC   TenGigE0/0/0/0
>x   TwentyFiveGigE0/0/0/30   Fail  n/a   100 PRC   TenGigE0/0/0/0

RP/0/RP0/CPU0:R5#

    Verifies the configuration. Note the following points:

    • All line interface that have frequency synchronization configured are displayed.

    • All clock interfaces and internal oscillators are displayed.

    • Sources that have been nominated as inputs (in other words, have selection input configured) have ‘S’ in the Flags column; sources that have not been nominated as inputs do not have ‘S’ displayed.


      Note

      Internal oscillators are always eligible as inputs.

    • ‘>’ or ‘D’ is displayed in the flags field as appropriate.

    If any of these items are not true, continue to the next step.

  4. show processes fsyncmgr location node-id

    This command verifies that the fsyncmgr process is running on the appropriate nodes.

    RP/0/RP0/CPU0:R5# show processes fsyncmgr location 0/0/cPU0
Thu Feb 1 06:26:32.979 UTC
Job Id: 181
PID: HYPERLINK "tel:3411"3411
Process name: fsyncmgr
Executable path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/bin/fsyncmgr Instance #: 1
Version ID: 00.00.0000
Respawn: ON
Respawn count: 1
Last started: Tue Jan 23 04:26:57 HYPERLINK "tel:2018"2018
Process state: Run
Package state: Normal
core: MAINMEM
Max. core: 0
Level: 100
Placement: None
startup_path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/startup/fsyncmgr.startup Ready: 2.063s
Process cpu time: 168.480 user, 129.980 kernel, 298.460 total
JID TID Stack pri state NAME rt_pri
181 HYPERLINK "tel:3411"3411 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3572"3572 0K 20 Sleeping lwm_debug_threa 0
181 HYPERLINK "tel:3573"3573 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3574"3574 0K 20 Sleeping lwm_service_thr 0
181 HYPERLINK "tel:3575"3575 0K 20 Sleeping qsm_service_thr 0
181 HYPERLINK "tel:3622"3622 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3781"3781 0K 20 Sleeping fsyncmgr 0
181 HYPERLINK "tel:3789"3789 0K 20 Sleeping fsyncmgr 0

Verifying the ESMC Configuration

show frequency synchronization interfaces 

Router# show frequency synchronization interfaces
                    Interface TenGigE0/0/0/0 (up)
                    Assigned as input for selection
                    Wait-to-restore time 0 minutes
                    SSM Enabled
Peer Up for 2d01h, last SSM received 0.320s ago
Peer has come up 1 times and timed out 0 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 HYPERLINK "tel:178463"178463
                    Received: HYPERLINK "tel:178499"178499 HYPERLINK "tel:178499"178499 0 0
                    Input:
                    Up
                    Last received QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC, Priority: 1, Time-of-day Priority 100
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: DNU
                    Next selection points: ETH_RXMUX
                    Interface TenGigE0/0/0/1 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 2d01h, last SSM received never
                    Peer has come up 0 times and timed out 1 times
                    ESMC SSMs Total Information Event DNU/DUS
                    Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 0
                    Received: 0 0 0 0
                    Input:
                    Down - not assigned for selection
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC
                    Next selection points: ETH_RXMUX
                    Interface TwentyFiveGigE0/0/0/30 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 01:50:24, last SSM received 01:50:30 ago
                    Peer has come up 1 times and timed out 1 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:75086"75086 HYPERLINK "tel:75085"75085 1 0
Received: HYPERLINK "tel:68457"68457 HYPERLINK "tel:68455"68455 2 HYPERLINK "tel:68443"68443
Input:
Down - not assigned for selection
Supports frequency
Output:
Selected source: TenGigE0/0/0/0
Selected source QL: Opt-I/PRC
Effective QL: Opt-I/PRC
Next selection points: ETH_RXMUX

Verifying Controllers Timing LEDs

Router# show controllers timing led status location 0/RP0/CPU0
LED Status:
        BITS0: Off
        BITS1: Off
         Sync: Green
         GNSS: Off
          GPS: NA