Synchronous Ethernet ESMC and SSM

Synchronous Ethernet is an extension of Ethernet designed to provide the reliability found in traditional SONET/SDH and T1/E1 networks to Ethernet packet networks by incorporating clock synchronization features. It supports the Synchronization Status Message (SSM) and Ethernet Synchronization Message Channel (ESMC) for synchronous Ethernet clock synchronization.

Synchronous Ethernet incorporates the Synchronization Status Message (SSM) used in Synchronous Optical Networking (SONET) and Synchronous Digital Hierarchy (SDH) networks. While SONET and SDH transmit the SSM in a fixed location within the frame, Ethernet Synchronization Message Channel (ESMC) transmits the SSM using a protocol: the IEEE 802.3 Organization-Specific Slow Protocol (OSSP) standard.

The ESMC carries a Quality Level (QL) value identifying the clock quality of a given synchronous Ethernet timing source. Clock quality values help a synchronous Ethernet node derive timing from the most reliable source and prevent timing loops.

When configured to use synchronous Ethernet, the router synchronizes to the best available clock source. If no better clock sources are available, the router remains synchronized to the current clock source.

The router supports QL-enabled mode.

Frequency Synchronization Timing Concepts

The Cisco IOS XR frequency synchronization infrastructure is used to select between different frequency sources to set the router backplane frequency and time-of-day. There are two important concepts that must be understood with respect to the frequency synchronization implementation.

Sources

A source is a piece of hardware that inputs frequency signals into the system or transmits them out of the system. There are four types of sources:

  • Line interfaces. This includes SyncE interfaces.

  • Clock interfaces. These are external connectors for connecting other timing signals, such as, GPS, BITS.

  • PTP clock. If IEEE 1588 version 2 is configured on the router, a PTP clock may be available to frequency synchronization as a source of the time-of-day and frequency.

  • Internal oscillator. This is a free-running internal oscillator chip.

Each timing source has a Quality Level (QL) associated with it which gives the accuracy of the clock. This QL information is transmitted across the network via SSMs over the Ethernet Synchronization Messaging Channel (ESMC) or SSMs contained in the SONET/SDH frames so that devices know the best available source to synchronize to. In order to define a preferred network synchronization flow, and to help prevent timing loops, you can assign priority values to particular timing sources on each router. The combination of QL information and user-assigned priority levels allows each router to choose a timing source to use to clock its SyncE and SONET/SDH interfaces, as described in the ITU standard G.781.

Selection Points

A selection point is any point where a choice is made between several frequency signals, and possibly one or more of them are selected. Selection points form a graph representing the flow of timing signals between the different cards in a router running Cisco IOS XR software. For example, one or multiple selection points select between the different Synchronous Ethernet inputs available on a single line card, and the result of these selection points is forwarded to a selection point on the RSP to select between the selected source from each card.

The input signals to the selection points can be:

  • Received directly from a source.

  • The output from another selection point on the same card.

  • The output from a selection point on a different card.

The output of a selection point can be used in a number of ways:

  • Used to drive the signals sent out of a set of sources.

  • As input into another selection point on the card.

  • As input into a selection point on another card.

Use the show frequency synchronization selection command to see a detailed view of the different selection points within the system.

Restrictions

  • The Precision Time Protocol (PTP) session flaps during Route Processor Failover (RPFO).

Configuring Frequency Synchronization

Enabling Frequency Synchronization on the Router

This task describes the router-level configuration required to enable frequency synchronization.

RP/0/RP0/CPU0:Router# configure
RP/0/RP0/CPU0:Router(config)# frequency synchronization 
RP/0/RP0/CPU0:Router(config-freqsync)# clock-interface timing-mode system 
RP/0/RP0/CPU0:Router(config-freqsync)# quality itu-t option 1 generation 1 
RP/0/RP0/CPU0:Router(config-freqsync)# log selection changes 
RP/0/RP0/CPU0:Router(config-freqsync)# commit

Configuring Frequency Synchronization on an Interface

By default, there is no frequency synchronization on line interfaces. Use this task to configure an interface to participate in frequency synchronization.

Before You Begin

You must enable frequency synchronization globally on the router.

RP/0/RP0/CPU0:R1#config terminal
RP/0/RP0/CPU0:R1(config)#interface TenGigabitEthernet 0/0/0/0 
RP/0/RP0/CPU0:R1(config-if)#frequency synchronization 
RP/0/RP0/CPU0:R1(config-if-freqsync)#selection input 
RP/0/RP0/CPU0:R1(config-if-freqsync)#wait-to-restore 10 
RP/0/RP0/CPU0:R1(config-if-freqsync)#priority 5 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality transmit exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#quality receive exact itu-t option 1 PRC 
RP/0/RP0/CPU0:R1(config-if-freqsync)#commit
or
RP/0/RP0/CPU0:router(config-freqsync)# commit

Configuring Frequency Synchronization on a Clock Interface

To enable a clock interface to be used as frequency input or output, you must configure the port parameters and frequency synchronization, as described in this task.

RP/0/RP0/CPU0:R1#configure
RP/0/RP0/CPU0:R1(config)# clock-interface sync 2 location 0/RP0/CPU0
RP/0/RP0/CPU0:R1(config-clock-if)# port-parameters
RP/0/RP0/CPU0:R1(config-clk-parms)# gps-input tod-format cisco pps-input tt1
RP/0/RP0/CPU0:R1(config-clk-parms)# exit
RP/0/RP0/CPU0:R1(config-clock-if)# frequency synchronization
RP/0/RP0/CPU0:R1(config-clk-freqsync)# selection input
RP/0/RP0/CPU0:R1(config-clk-freqsync)# wait-to-restore 1
RP/0/RP0/CPU0:R1(config-clk-freqsync)# quality receive exact itu-t option 1 PRC

Verifying the Frequency Synchronization Configuration

After performing the frequency synchronization configuration tasks, use this task to check for configuration errors and verify the configuration.

  1. show frequency synchronization selection

    RP/0/RP0/CPU0:R5# show frequency synchronization selection
    Fri Apr 24 12:49:32.833 UTC
    Node 0/RP1/CPU0:
    ==============
    Selection point: T0-SEL-B (3 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : CHASSIS-TOD-SEL
        Chassis scoped: LC_TX_SELECT
        Router scoped : None
      Uses frequency selection
      Used for local line interface output
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         PTP [0/RP1/CPU0]          n/a                         PRC  254  Available  
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: 1588-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        0/RP1/CPU0 ETH_RXMUX 4      PRC   10  Locked     
         Internal0 [0/RP1/CPU0]    n/a                         SEC  255  Available  
    
    Selection point: CHASSIS-TOD-SEL (2 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : None
        Chassis scoped: None
        Router scoped : None
      Uses time-of-day selection
      S  Input                     Last Selection Point      Pri  Time  Status
      == ========================  ========================  ===  ====  ===========
      1  PTP [0/RP1/CPU0]          n/a                       100  Yes   Available  
         HundredGigE0/7/0/0        0/RP1/CPU0 T0-SEL-B 4     100  No    Available  
    
    Selection point: ETH_RXMUX (1 inputs, 1 selected)
      Last programmed 3d04h ago, and selection made 3d04h ago
      Next selection points
        SPA scoped    : None
        Node scoped   : T0-SEL-B 1588-SEL
        Chassis scoped: None
        Router scoped : None
      Uses frequency selection
      S  Input                     Last Selection Point         QL  Pri  Status
      == ========================  ========================  =====  ===  ===========
      4  HundredGigE0/7/0/0        n/a                         PRC   10  Available  
    
    
  2. show frequency synchronization configuration-errors

    RP/0/RP0/CPU0:router# show frequency synchronization configuration-errors
    Node 0/2/CPU0:
    ==============
      interface GigabitEthernet0/2/0/0 frequency synchronization
        * Frequency synchronization is enabled on this interface, but isn't enabled globally.
    interface GigabitEthernet0/2/0/0 frequency synchronization quality transmit exact itu-t option 2 generation 1 PRS
        * The QL that is configured is from a different QL option set than is configured
    globally.

    Displays any errors that are caused by inconsistencies between shared-plane (global) and local-plane (interface) configurations. There are two possible errors that can be displayed:

    • Frequency Synchronization is configured on an interface (line interface or clock-interface), but is not configured globally.

    • The QL option configured on some interface does not match the global QL option. Under an interface (line interface or clock interface), the QL option is specified using the quality transmit and quality receive commands. The value specified must match the value configured in the global quality itu-t option command, or match the default (option 1) if the global quality itu-t option command is not configured.

    Once all the errors have been resolved, meaning there is no output from the command, continue to the next step.

  3. show frequency synchronization interfaces brief

    RP/0/RP0/CPU0:R5# show frequency synchronization interfaces brief
    Thu Feb 1 06:30:02.945 UTC
    Flags: > - Up
            d - SSM Disabled
            s - Output squelched
    Fl   Interface
    D - Down              S - Assigned for selection
     x - Peer timed out    i - Init state
    Last Selection Point
    Pri Time
    Status
                                  QLrcv QLuse Pri QLsnd Output driven by
    ==== ======================== ===== ===== === ===== ========================
    >S   TenGigE0/0/0/0           PRC   PRC     1 DNU   TenGigE0/0/0/0
    >x   TenGigE0/0/0/1           Fail  n/a   100 PRC   TenGigE0/0/0/0
    >x   TwentyFiveGigE0/0/0/30   Fail  n/a   100 PRC   TenGigE0/0/0/0
    
    RP/0/RP0/CPU0:R5#

    Verifies the configuration. Note the following points:

    • All line interface that have frequency synchronization configured are displayed.

    • All clock interfaces are displayed.

    • Sources that have been nominated as inputs (in other words, have selection input configured) have ‘S’ in the Flags column; sources that have not been nominated as inputs do not have ‘S’ displayed.


      Note


      Internal oscillators are always eligible as inputs.


    • ‘>’ or ‘D’ is displayed in the flags field as appropriate.

    If any of these items are not true, continue to the next step.

  4. show processes fsyncmgr location node-id

    This command verifies that the fsyncmgr process is running on the appropriate nodes.

    RP/0/RP0/CPU0:R5# show processes fsyncmgr location 0/0/cPU0
    Thu Feb 1 06:26:32.979 UTC
    Job Id: 181
    PID: HYPERLINK "tel:3411"3411
    Process name: fsyncmgr
    Executable path: /opt/cisco/XR/packages/ncs560-iosxr-fwding-1.0.0.0-r63226I/all/bin/fsyncmgr Instance #: 1
    Version ID: 00.00.0000
    Respawn: ON
    Respawn count: 1
    Last started: Tue Jan 23 04:26:57 HYPERLINK "tel:2018"2018
    Process state: Run
    Package state: Normal
    core: MAINMEM
    Max. core: 0
    Level: 100
    Placement: None
    startup_path: /opt/cisco/XR/packages/ncs560-iosxr-fwding-1.0.0.0-r63226I/all/startup/fsyncmgr.startup Ready: 2.063s
    Process cpu time: 168.480 user, 129.980 kernel, 298.460 total
    JID TID Stack pri state NAME rt_pri
    181 HYPERLINK "tel:3411"3411 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3572"3572 0K 20 Sleeping lwm_debug_threa 0
    181 HYPERLINK "tel:3573"3573 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3574"3574 0K 20 Sleeping lwm_service_thr 0
    181 HYPERLINK "tel:3575"3575 0K 20 Sleeping qsm_service_thr 0
    181 HYPERLINK "tel:3622"3622 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3781"3781 0K 20 Sleeping fsyncmgr 0
    181 HYPERLINK "tel:3789"3789 0K 20 Sleeping fsyncmgr 0

Verifying the ESMC Configuration

show frequency synchronization interfaces

RP/0/RP0/CPU0:R5# show frequency synchronization interfaces
                    Thu Feb 1 06:33:26.575 UTC
                    Interface TenGigE0/0/0/0 (up)
                    Assigned as input for selection
                    Wait-to-restore time 0 minutes
                    SSM Enabled
Peer Up for 2d01h, last SSM received 0.320s ago
Peer has come up 1 times and timed out 0 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 HYPERLINK "tel:178463"178463
                    Received: HYPERLINK "tel:178499"178499 HYPERLINK "tel:178499"178499 0 0
                    Input:
                    Up
                    Last received QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC, Priority: 1, Time-of-day Priority 100
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: DNU
                    Next selection points: ETH_RXMUX
                    Interface TenGigE0/0/0/1 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 2d01h, last SSM received never
                    Peer has come up 0 times and timed out 1 times
                    ESMC SSMs Total Information Event DNU/DUS
                    Sent: HYPERLINK "tel:178479"178479 HYPERLINK "tel:178477"178477 2 0
                    Received: 0 0 0 0
                    Input:
                    Down - not assigned for selection
                    Supports frequency
                    Output:
                    Selected source: TenGigE0/0/0/0
                    Selected source QL: Opt-I/PRC
                    Effective QL: Opt-I/PRC
                    Next selection points: ETH_RXMUX
                    Interface TwentyFiveGigE0/0/0/30 (up)
                    Wait-to-restore time 5 minutes
                    SSM Enabled
                    Peer Timed Out for 01:50:24, last SSM received 01:50:30 ago
                    Peer has come up 1 times and timed out 1 times
ESMC SSMs Total Information Event DNU/DUS
Sent: HYPERLINK "tel:75086"75086 HYPERLINK "tel:75085"75085 1 0
Received: HYPERLINK "tel:68457"68457 HYPERLINK "tel:68455"68455 2 HYPERLINK "tel:68443"68443
Input:
Down - not assigned for selection
Supports frequency
Output:
Selected source: TenGigE0/0/0/0
Selected source QL: Opt-I/PRC
Effective QL: Opt-I/PRC
Next selection points: ETH_RXMUX

Verifying Synchronous Ethernet LEDs

RP/0/RP0/CPU0:ios# show led
Mon Nov 2 09:29:34.637 UTC
================================================================================
Location LED Name Mode Color
================================================================================
0/FT0
Status OPERATIONAL GREEN
0/RP0/CPU0
Alarm OPERATIONAL AMBER
Status OPERATIONAL GREEN
Sync OPERATIONAL AMBER