Synchronous Ethernet ESMC and SSM

Synchronous Ethernet is an extension of Ethernet designed to provide the reliability found in traditional SONET/SDH and T1/E1 networks to Ethernet packet networks by incorporating clock synchronization features. It supports the Synchronization Status Message (SSM) and Ethernet Synchronization Message Channel (ESMC) for synchronous Ethernet clock synchronization.

Synchronous Ethernet incorporates the Synchronization Status Message (SSM) used in Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) networks. While SONET and SDH transmit the SSM in a fixed location within the frame, Ethernet Synchronization Message Channel (ESMC) transmits the SSM using a protocol: the IEEE 802.3 Organization-Specific Slow Protocol (OSSP) standard.

The ESMC carries a Quality Level (QL) value identifying the clock quality of a given synchronous Ethernet timing source. Clock quality values help a synchronous Ethernet node derive timing from the most reliable source and prevent timing loops.

When configured to use synchronous Ethernet, the router synchronizes to the best available clock source. If no better clock sources are available, the router remains synchronized to the current clock source.

The router supports QL-enabled mode.

Frequency Synchronization Timing Concepts

The Cisco IOS XR frequency synchronization infrastructure is used to select between different frequency sources to set the router backplane frequency and time-of-day. There are two important concepts that must be understood with respect to the frequency synchronization implementation.

Sources

A source is a piece of hardware that inputs frequency signals into the system or transmits them out of the system. There are four types of sources:

  • Line interfaces. This includes SyncE interfaces.

  • Clock interfaces. These are external connectors for connecting other timing signals, such as, GPS, BITS.

  • PTP clock. If IEEE 1588 version 2 is configured on the router, a PTP clock may be available to frequency synchronization as a source of the time-of-day and frequency.

  • Internal oscillator. This is a free-running internal oscillator chip.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) or SSMs contained in the SONET/SDH frames so that devices know the best available source to synchronize to. In order to define a preferred network synchronization flow, and to help prevent timing loops, you can assign priority values to particular timing sources on each router. The combination of QL information and user-assigned priority levels allows each router to choose a timing source to use to clock its SyncE and SONET/SDH interfaces, as described in the ITU standard G.781.

Selection Points

A selection point is any point where a choice is made between several frequency signals, and possibly one or more of them are selected. Selection points form a graph representing the flow of timing signals between the different cards in a router running Cisco IOS XR software. For example, one or multiple selection points select between the different Synchronous Ethernet inputs available on a single line card, and the result of these selection points is forwarded to a selection point on the RSP to select between the selected source from each card.

The input signals to the selection points can be:

  • Received directly from a source.

  • The output from another selection point on the same card.

  • The output from a selection point on a different card.

The output of a selection point can be used in a number of ways:

  • Used to drive the signals sent out of a set of sources.

  • As input into another selection point on the card.

  • As input into a selection point on another card.

Use the show frequency synchronization selection command to see a detailed view of the different selection points within the system.


Note


  • We recommend you to configure, and enable Frequency Synchronization selection input on two interfaces per line card.

  • For link aggregation, you must configure and enable Frequency Synchronization selection input on a single bundle member.


Restrictions

  • SyncE is not supported on Gigabit Ethernet 0/0/0/24 to 0/0/0/31 ports.

  • The Precision Time Protocol (PTP) session flaps during Route Processor Failover (RPFO).

SyncE Hardware Support Matrix


Note


The table also contains support details of upcoming releases. You can read this table in context of the current release and see relevant Release Notes for more information on supported features and hardware.


This table provides a detailed information on the timing features that are supported on the following hardware variants.

Hardware Variant

Features

Cisco IOS XR Release

Comments

NCS-57C3-MOD-SYS

NCS-57C3-MODS-SYS

E-SyncE

Release 7.9.1

NC57-24DD

SyncE

Release 7.5.1

NC57-18DD-SE

SyncE

Release 7.5.1

NCS-57C1-48Q6-SYS

SyncE

Release 7.5.1

E-SyncE

Release 7.9.1

RP:NC57-MOD-RP2-E with NCS-57C3-MOD-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 40-55, and not on MPA.

RP:NC57-MOD-RP2-E with NCS-57C3-MODS-SYS

SyncE

Release 7.4.1

1G Fiber clock recovery is supported from IOS XR Release 7.6.1 on SFP28 ports 0-7, 36-51, and not on MPA.

RP: NC55-RP2-E

Line card: NC57-36H6D-S

SyncE

Release 7.3.2

  • Release 7.3.2 - Supports Compatible Mode only

  • Release 7.7.1 - Supports both Native and Compatible mode.

  • SyncE is not supported on 100GE interfaces, when they are used in 1G mode.

NCS-57B1-5DSE-SYS

NCS-57B1-6D24-SYS

SyncE

Release 7.3.1

E-syncE

Release 7.9.1

RP:NC55-RP-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line cards: NC55-MOD-A-S and NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP2-E with Line card:NC55-32T16Q4H-AT

SyncE

Release 7.1.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-SE-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-36H-S

SyncE

Release 7.0.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24Q6H-S

NCS-55A1-24Q6H-SS

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-48Q6H

SyncE

Release 6.6.25

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS-55A1-24H

SyncE

Release 6.5.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS55A2-MOD

SyncE

Release 6.5.1

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

E-syncE

Release 7.9.1

E-SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-MOD-A-S

SyncE

Release 6.5.1

SyncE is not supported on 100GE interfaces, when theyare used in 1G mode.

RP:NC55-RP-E

Linecard:NC55-36X100G-A-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

NCS5501-SE

SyncE

Release 6.3.2

SyncE is not supported on 25GE or 100GE interfaces, when they are used in 1G mode.

SyncE is supported on 10G from ports 8 to 15, but it is not supported on these ports in 1G mode.

SyncE features are supported on the following MPAs:

  • NC55-MPA-2TH-S

  • NC55-MPA-1TH2H-S

  • NC55-MPA-1TH2H-HD-S

  • NC55-MPA-4H-S

  • NC55-MPA-4H-HD-S

  • NC55-MPA-12T-S

Configuring Frequency Synchronization

Enabling Frequency Synchronization on the Router

This task describes the router-level configuration required to enable frequency synchronization.

RP/0/RP0/CPU0:Router# configure
RP/0/RP0/CPU0:Router(config)# frequency synchronization 
RP/0/RP0/CPU0:Router(config-freqsync)# clock-interface timing-mode system 
RP/0/RP0/CPU0:Router(config-freqsync)# quality itu-t option 1 generation 1 
RP/0/RP0/CPU0:Router(config-freqsync)# log selection changes 
RP/0/RP0/CPU0:Router(config-freqsync)# commit

Configuring Frequency Synchronization on an Interface

By default, there is no frequency synchronization on line interfaces. Use this task to configure an interface to participate in frequency synchronization.

Before You Begin

You must enable frequency synchronization globally on the router.

RP/0/RP0/CPU0:R1#config terminal
RP/0/RP0/CPU0:R1(config)#interface TenGigabitEthernet 0/0/0/0 
RP/0/RP0/CPU0:R1(config-if)#frequency synchronization 
RP/0/RP0/CPU0:R1(config-if-freqsync)#selection input 
RP/0/RP0/CPU0:R1(config-if-freqsync)#wait-to-restore 10 
RP/0/RP0/CPU0:R1(config-if-freqsync)#priority 5 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality transmit exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality receive exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#commit
or
RP/0/RP0/CPU0:router(config-freqsync)# commit

Configuring Frequency Synchronization on a Clock Interface

To enable a clock interface to be used as frequency input or output, you must configure the port parameters and frequency synchronization, as described in this task.

RP/0/RP0/CPU0:R1#configure
RP/0/RP0/CPU0:R1(config)# clock-interface sync 2 location 0/RP0/CPU0
RP/0/RP0/CPU0:R1(config-clock-if)# port-parameters
RP/0/RP0/CPU0:R1(config-clk-parms)# gps-input tod-format cisco pps-input tt1
RP/0/RP0/CPU0:R1(config-clk-parms)# exit
RP/0/RP0/CPU0:R1(config-clock-if)# frequency synchronization
RP/0/RP0/CPU0:R1(config-clk-freqsync)# selection input
RP/0/RP0/CPU0:R1(config-clk-freqsync)# wait-to-restore 1
RP/0/RP0/CPU0:R1(config-clk-freqsync)# quality receive exact itu-t option 1 PRC

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this task to check for configuration errors and verify the configuration.

  1. show frequency synchronization selection

    RP/0/RP0/CPU0:R5# show frequency synchronization selection
    Fri Apr 24 12:49:32.833 UTC
    Node 0/RP1/CPU0:
    ==============
    Selection point: T0-SEL-B (3 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : CHASSIS-TOD-SEL
        Chassis scoped: LC_TX_SELECT
        Router scoped : None
      Uses frequency selection
      Used for local line interface output
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         PTP [0/RP1/CPU0]          n/a                         PRC  254  Available  
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: 1588-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: CHASSIS-TOD-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses time-of-day selection
      S  Input                     Last Selection Point      Pri  Time  Status
      == ========================  ========================  ===  ====  ===========
      1  PTP [0/RP1/CPU0]          n/a                       100  Yes   Available  
         HundredGigE0/7/0/0        0/RP1/CPU0 T0-SEL-B 4     100  No    Available  
    
    Selection point: ETH_RXMUX (1 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : T0-SEL-B 1588-SEL
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        n/a                         PRC   10  Available  
    
    
  2. show frequency synchronization configuration-errors

    RP/0/RP0/CPU0:router# show frequency synchronization configuration-errors
    Node 0/2/CPU0:
    ==============
      interface GigabitEthernet0/2/0/0 frequency synchronization
        * Frequency synchronization is enabled on this interface, but isn't enabled globally.
    interface GigabitEthernet0/2/0/0 frequency synchronization quality transmit exact itu-t option 2 generation 1 PRS
        * The QL that is configured is from a different QL option set than is configured
    globally.

    Displays any errors that are caused by inconsistencies between shared-plane (global) and local-plane (interface) configurations. There are two possible errors that can be displayed:

    • Frequency Synchronization is configured on an interface (line interface or clock-interface), but is not configured globally.

    • The QL option configured on some interface does not match the global QL option. Under an interface (line interface or clock interface), the QL option is specified using the quality transmit and quality receive commands. The value specified must match the value configured in the global quality itu-t option command, or match the default (option 1) if the global quality itu-t option command is not configured.

    Once all the errors have been resolved, meaning there is no output from the command, continue to the next step.

  3. show frequency synchronization interfaces brief

    RP/0/RP0/CPU0:R5# show frequency synchronization interfaces brief
    Flags: > - Up
            d - SSM Disabled
            s - Output squelched
    Fl   Interface
    D - Down              S - Assigned for selection
     x - Peer timed out    i - Init state
    Last Selection Point
    Pri Time
    Status
                                  QLrcv QLuse Pri QLsnd Output driven by
    ==== ======================== ===== ===== === ===== ========================
    >S   TenGigE0/0/0/0           PRC   PRC     1 DNU   TenGigE0/0/0/0
    >x   TenGigE0/0/0/1           Fail  n/a   100 PRC   TenGigE0/0/0/0
    >x   TwentyFiveGigE0/0/0/30   Fail  n/a   100 PRC   TenGigE0/0/0/0
    
    RP/0/RP0/CPU0:R5#

    Verifies the configuration. Note the following points:

    • All line interface that have frequency synchronization configured are displayed.

    • All clock interfaces and internal oscillators are displayed.

    • Sources that have been nominated as inputs (in other words, have selection input configured) have ‘S’ in the Flags column; sources that have not been nominated as inputs do not have ‘S’ displayed.


      Note


      Internal oscillators are always eligible as inputs.


    • ‘>’ or ‘D’ is displayed in the flags field as appropriate.

    If any of these items are not true, continue to the next step.

  4. show processes fsyncmgr location node-id

    This command verifies that the fsyncmgr process is running on the appropriate nodes.

    RP/0/RP0/CPU0:R5# show processes fsyncmgr location 0/0/cPU0
    Thu Feb 1 06:26:32.979 UTC
    Job Id: 181
    PID: HYPERLINK "tel:3411"3411
    Process name: fsyncmgr
    Executable path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/bin/fsyncmgr Instance #: 1
    Version ID: 00.00.0000
    Respawn: ON
    Respawn count: 1
    Last started: Tue Jan 23 04:26:57 HYPERLINK "tel:2018"2018
    Process state: Run
    Package state: Normal
    core: MAINMEM
    Max. core: 0
    Level: 100
    Placement: None
    startup_path: /opt/cisco/XR/packages/ncs540-iosxr-fwding-1.0.0.0-r63226I/all/startup/fsyncmgr.startup Ready: 2.063s
    Process cpu time: 168.480 user, 129.980 kernel, 298.460 total
    JID TID Stack pri state NAME rt_pri
    181 HYPERLINK "tel:3411"3411 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3572"3572 0K 20 Sleeping lwm_debug_threa 0
    181 HYPERLINK "tel:3573"3573 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3574"3574 0K 20 Sleeping lwm_service_thr 0
    181 HYPERLINK "tel:3575"3575 0K 20 Sleeping qsm_service_thr 0
    181 HYPERLINK "tel:3622"3622 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3781"3781 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3789"3789 0K 20 Sleeping fsyncmgr 0

Verifying the ESMC Configuration

show frequency synchronization interfaces

Router# show frequency synchronization interfaces
                    Interface TenGigE0/0/0/0 (up)
                    Assigned as input for selection
                    Wait-to-restore time 0 minutes
                    SSM Enabled
Peer Up for 2d01h, last SSM received 0.320s ago
Peer has come up 1 times and timed out 0 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 HYPERLINK "tel:178463"178463
                    Received: HYPERLINK "tel:178499"178499 HYPERLINK "tel:178499"178499 0 0
                    Input:
                    Up
                    Last received QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC, Priority: 1, Time-of-day Priority 100
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: DNU
                    Next selection points: ETH_RXMUX
                    Interface TenGigE0/0/0/1 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 2d01h, last SSM received never
                    Peer has come up 0 times and timed out 1 times
                    ESMC SSMs Total Information Event DNU/DUS
                    Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 0
                    Received: 0 0 0 0
                    Input:
                    Down - not assigned for selection
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC
                    Next selection points: ETH_RXMUX
                    Interface TwentyFiveGigE0/0/0/30 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 01:50:24, last SSM received 01:50:30 ago
                    Peer has come up 1 times and timed out 1 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:75086"75086 HYPERLINK "tel:75085"75085 1 0
Received: HYPERLINK "tel:68457"68457 HYPERLINK "tel:68455"68455 2 HYPERLINK "tel:68443"68443
Input:
Down - not assigned for selection
Supports frequency
Output:
Selected source: TenGigE0/0/0/0
Selected source QL: Opt-I/PRC
Effective QL: Opt-I/PRC
Next selection points: ETH_RXMUX

Verifying Controllers Timing LEDs

Router# show controllers timing led status location 0/RP0/CPU0
LED Status:
        BITS0: Off
        BITS1: Off
         Sync: Green
         GNSS: Off
          GPS: NA