Contents

Introduction

Background

Optics

Interface

Port Speed

Fault Signaling Mechanism

DWDM Tunable Optic

IPoDWDM - Coherent DSP

Detailed Troubleshooting

First Things to Check

Alarms

Show Controller Optics

CoherentDSP

Show Controllers PHY

NPU

Summary

Collect Logs

Introduction

This document describes how to troubleshoot interface down, or interface flap issues.

This document is specific to Cisco IOS®XR, but it is not restricted to a specific software release.

This document is specific to the NCS platform.

Background

There are multiple reasons why interface down, or interface flap can happen. The links can go down due to multiple devices as shown in image 1. It shows an NCS router with one front panel port connected to the NPU (Network Processor Unit) via PHY (Physical Layer) device in between (known as PHY based ports). There are platforms where the front panel ports are directly connected to the NPU (known as PHYless ports). A single router can have both variants.

Image 1

Optics

Interface

Note that an interface is always described in this way on Cisco IOS XR.

R/S/I/P  : Rack/Slot/Instance/Port/Breakout

R/S/I/P/B : Rack/Slot/Instance/Port/

Port Speed

Some platforms/line cards support multiple speeds on some ports. Examples are: 1/10GE, 25/40GE, 40/100GE, 1/10/25GE.

The speed can be configured on each individual port or per group of ports. Look at the hardware documentation for the port mapping on the NCS. Check if the port speed is set correctly. Some ports can auto-sense the speed, others need configuration.

Some ports can be configured per group (usually a quad: a group of four).

For example:

(config)#hw-module quad 0 location 0/0/CPU0 mode ?

WORD  10g or 25g

Some ports can be configured individually. This is usually the 100G ports. 100G is the default speed, but you can configure 40G.

For example:

(config)#controller optics 0/0/1/0

(config-Optics)#speed 40g

Fault Signaling Mechanism

Note that a Link Fault Signaling mechanism can play a role when you troubleshoot the interface issue. On 10G and higher, there is Link Fault Signaling mechanism included in the PHY layer. So when a local fault is detected, the remote end is informed.

Image 2

On GigE interface, there is no such onboard mechanism. Check if “ethernet oam” is configured as an alternative.

For example:

interface GigabitEthernet0/6/0/2

ethernet oam

uni-directional link-fault detection

action

uni-directional link-fault efd

DWDM Tunable Optic

On DWDM optic that supports frequency tuning, use this config to set the frequency that can be used (based on DWDM circuit setup by the circuit provider):

This is the command:

controller optics 0/6/0/2

dwdm-carrier {100MHz-grid | 50GHz-grid} frequency {frequency}

Verify the settings:

RP/0/RP0/CPU0:NCS-5508#show controllers optics 0/6/0/2 dwdm-carrier-map

DWDM Carrier Band:: OPTICS_C_BAND

MSA ITU channel range supported: 1~96

DWDM Carrier Map table

----------------------------------------------------

ITU Ch      G.694.1     Frequency     Wavelength

Num        Ch Num        (THz)          (nm)

----------------------------------------------------

1       -35           191.35      1566.723

----------------------------------------------------

2       -34           191.40      1566.314

----------------------------------------------------

3       -33           191.45      1565.905

----------------------------------------------------

4       -32           191.50      1565.496

----------------------------------------------------

...

----------------------------------------------------

95        59           196.05      1529.163

----------------------------------------------------

96        60           196.10      1528.773

----------------------------------------------------

On DWDM optics, all DWDM specific tuning is done under controller dwdm config :

RP/0/RP0/CPU0:NCS-5508(config)#controller dwdm 0/6/0/2 ?

admin-state       Configure the transport admin state of the controller: in-service, out-of-service or maintenance

g709              Configure G709 parameters

laser             To be deprecated and removed in 7.5.1 release

log               Proactive log

loopback          Configure loopback mode

network           Configure DWDM network information

pm                Configure pm parameters

proactive         Enable Proactive Protection Feature

rx-los-threshold  Configure transponder receive power LOS threshold

transmit-power    Configure transponder transmit power

vtxp-monitor      Enable VTXP monitoring function

wavelength        Configure ITU Channel, Wavelength and Frequency

<cr>

RP/0/RP0/CPU0:NCS-5508(config-dwdm)#g709 fec ?

15sdfec                       15%-SD Forward Error Correction

15sdfecde                     15%-SD Forward Error Correction with Diff

ci-bch                        Continuously Interleaved BCH FEC

disable                       Disable FEC

enhanced                      Enhanced FEC mode

high-gain                     To be deprecated and removed in 7.5.1 release

high-gain-hd-fec              7% HD FEC (Staircase FEC)

high-gain-multivendor-hd-fec  7% HD FEC (Staircase FEC) Multivendor Interoperable

high-gain-sd-fec              7% CISCO SD FEC (Soft-Decision FEC)

long-haul                     To be deprecated and removed in 7.5.1 release

long-haul-hd-fec              20% HD FEC (Staircase FEC)

long-haul-sd-fec              20% CISCO SD FEC (Soft-Decision FEC)

standard                      Standard FEC mode

IPoDWDM - Coherent DSP

Coherent DSP supports high rate speed (100/200/400G) over long distances. The DWDM processing is handled by Coherent DSP (Digital Signal Processor).

Examples of such NCS modules: NC55-6X200-DWDM-S or NC55-MPA-2TH-S (with CFP2-WDM-D-1HL)

RP/0/RP0/CPU0:NCS-5508(config)# controller coherentDSP 0/6/0/0

RP/0/RP0/CPU0:NCS-5508(config-CoDSP)#fec ?

CFEC                 Forward Error Correction C_FEC

EnhancedHG20         Forward Error Correction Enhanced_HG20

EnhancedHG7          Forward Error Correction Enhanced_HG7

EnhancedI4           Forward Error Correction Enhanced_I_4

EnhancedI7           Forward Error Correction Enhanced_I_7

EnhancedSD15         Forward Error Correction Soft-Decision 15

EnhancedSD15DE       Forward Error Correction Soft-Decision 15 with DE

EnhancedSD20         Forward Error Correction Soft-Decision 20

EnhancedSD27         Forward Error Correction Soft-Decision 27

EnhancedSD7          Forward Error Correction Soft-Decision 7

EnhancedStaircaseDE  Forward Error Correction Enhanced Staircase FEC with DE

EnhancedSwizzle      Forward Error Correction Enhanced_Swizzle

OFEC                 Forward Error Correction O_FEC

Standard             Forward Error Correction Standard

none                 No Forward Error Correction

Detailed Troubleshooting

This section covers the basic debugging/techniques from optics perspective.

First Things to Check

Check if the optics show up in the inventory. If the optics are not shown, check if the optics are inserted properly in the R/S/I/P and cables with no issues are used.

++++++++++ show inventory details [18:06:56.572 UTC Thu Apr 06 2023] ++++++++++

NAME: "0/RP0", DESCR: "NC55A1 24Q6H SS Route Processor Card"

PID: NCS-55A1-24Q6H-SS , VID: V01, SN: FOC2528002Q

MFG_NAME: Cisco Systems, Inc., SNMP_IDX: 1 , Type: Module

PN: 73-20057-02

NAME: "TenGigE0/0/0/0", DESCR: "Cisco SFP+ 10G ZR Pluggable Optics Module"

PID: SFP-10G-ZR , VID: V02, SN: BD211218N1T

MFG_NAME: CISCO-PRE , SNMP_IDX: 2129921 , Type: Module

PN: TSFP10G-1558.17

NAME: "TenGigE0/0/0/1", DESCR: "Cisco SFP+ 10G ZR Pluggable Optics Module"

PID: SFP-10G-ZR , VID: V02, SN: BD211218N3K

MFG_NAME: CISCO-PRE , SNMP_IDX: 2134017 , Type: Module

PN: TSFP10G-1557.36

RP/0/RP0/CPU0:NCS-5501#show controllers te0/0/0/14 internal

Internal data for interface: TenGigE0/0/0/14

Subport Number      : 255

Port Number         : 14 *

Bay Number          : 0 *

Board Type          : 0x60020201 *

Port Type           : 10GE *

Bandwidth(Kbps)     : 10000000 *

Transport mode      : LAN *

BIA MAC addr        : 008a:9617:4838

Oper. MAC addr      : 008a:9617:4838

Egress MAC addr     : 008a:9617:4838

Port Available      : true *

Status polling is   : disabled *

Status events are   : disabled

I/F Handle          : 0x00000158 *

Cfg Link Enabled    : enabled

H/W Tx Enable       : yes

MTU                 : 1514 *

H/W Speed           : 10 Gbps *

H/W Duplex          : Full *

H/W Loopback Type   : None *

FEC                 : Not Configured *

H/W FlowCtrl Type   : Disabled *

H/W AutoNeg Enable  : Off *

H/W Link Defects    : No Fault *

Link Up             : yes *

Link Led Status     : Green ON *

Pluggable Present   : Yes *

Pluggable Type      : SFP+ 10G CU3M

Pluggable PID       : SFP-H10GB-CU3M *

Pluggable Compl.    : Failed - Bad Vendor CRC

If the transceiver is not recognized, check if transceiver type is supported here: https://tmgmatrix.cisco.com/.

The vendor CRC should be correct.

Check the signal strength.

Requires DOM (Digital Optical Monitoring) support on the transceiver!

RP/0/RP0/CPU0:BRU-SPCORE-P2#show controllers hundredGigE0/0/0/10 phy

QSFP8636 EEPROM port: 10

Xcvr Type: QSFP28

Ext Type: 3.5W, CLEI, TX CDR, RX CDR,

Connector Type: MPO

Ethernet Compliance Codes: 100G BASE-SR4,

BR, nominal: 25500 Mbps

Length SMF: 0KM, OM3: 70M, OM2: 0M, OM1: 0M, Cable: 50M

Deice Tech: 850nm VCSEL,

Vendor Name: CISCO-FINISAR

Vendor OUI: 00.90.65

Vendor Part Number: FTLC9555REPM-C1   (rev.: A )

Wavelength: 850.000 nm

Wavelength Tolerance: 10.000 nm

Vendor Serial Number: FIW2638016W

Date Code (yy/mm/dd): 22/09/13  lot code:

Diagnostic Monitoring Type: RX Avg, TX,

Enhanced Options: Init Complete Flag Impl,

Extended Module Codes:

Options:

L-Tx/Rx LOS:

L-Tx Fault:

L-Tx/Rx LOL:

Module DDM: Volt, Temp, TX Power, TX Bias, RX Power,

MSA Data (Lower Memory)

0x0000: 11 07 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0010: 00 00 00 00 00 00 1d 75 : 00 00 81 2f 00 00 00 00

0x0020: 00 00 20 b6 2e 9a 2d ba : 27 44 0d ed 0e 0c 0e 0c

0x0030: 0e 0c 28 46 2a dc 29 1f : 2a 72 00 00 00 00 00 00

0x0040: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0050: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 01 00 00

0x0060: 00 00 ff 00 00 00 00 00 : 00 01 1f 00 00 00 00 00

0x0070: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

MSA Data (Upper Memory Page 00)

0x0080: 11 dc 0c 80 00 00 00 00 : 00 00 00 05 ff 00 00 23

0x0090: 00 00 32 00 43 49 53 43 : 4f 2d 46 49 4e 49 53 41

0x00a0: 52 20 20 20 00 00 90 65 : 46 54 4c 43 39 35 35 35

0x00b0: 52 45 50 4d 2d 43 31 20 : 41 20 42 68 07 d0 46 ef

0x00c0: 02 07 ff f6 46 49 57 32 : 36 33 38 30 31 36 57 20

0x00d0: 20 20 20 20 32 32 30 39 : 31 33 20 20 0c 10 68 3a

0x00e0: 00 00 02 3c c0 ff c6 b6 : 3b 05 e6 30 86 bb 80 05

0x00f0: df 65 71 00 00 00 00 00 : 00 00 00 00 79 eb c1 06

CLEI Code: CMUIATKCAA

Part Number: FTLC9555REPM-C1 (ver.: V03)

Product Id: QSFP-100G-SR4-S

MSA Data (Upper Memory Page 02)

0x0180: 43 4d 55 49 41 54 4b 43 : 41 41 31 30 2d 33 31 34

0x0190: 32 2d 30 33 56 30 33 20 : 01 00 00 31 20 20 20 20

0x01a0: 20 66 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x01b0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 aa aa

0x01c0: 51 53 46 50 2d 31 30 30 : 47 2d 53 52 34 2d 53 20

0x01d0: 20 20 20 20 00 00 00 00 : 00 00 00 00 00 00 00 65

0x01e0: 31 33 33 39 39 37 31 36 : 31 d8 00 00 00 00 00 00

0x01f0: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

Module

Thresholds:                   Alarm High          Warning High          Warning Low          Alarm Low

Temperature:            +75.000 C             +70.000 C              +0.000 C              -5.000 C

Voltage:           3.630 Volt            3.465 Volt            3.135 Volt            2.970 Volt

Temperature: +29.457 C

Voltage: 3.315 Volt

Lanes

Thresholds:                   Alarm High          Warning High          Warning Low          Alarm Low

Bias:         15.000 mAmps          14.000 mAmps           3.000 mAmps           2.000 mAmps

Transmit Power:  3.46740 mW (5.40004 dBm)   1.73780 mW (2.40000 dBm)   0.14450 mW (-8.40132 dBm)   0.05750 mW (-12.40332 dBm)

Receive Power:  3.46740 mW (5.40004 dBm)   1.73780 mW (2.40000 dBm)   0.09330 mW (-10.30118 dBm)   0.03720 mW (-14.29457 dBm)

TxRxIOMagId:

RxOpAmpSupprt:

Lane            Temp            Bias                   Tx Power                   Rx Power

0             N/A     7.130 mAmps   1.03100 mW (0.13259 dBm)   1.18990 mW (0.75510 dBm)

1             N/A     7.192 mAmps   1.09760 mW (0.40444 dBm)   1.19300 mW (0.76640 dBm)

2             N/A     7.192 mAmps   1.05440 mW (0.23005 dBm)   1.17110 mW (0.68594 dBm)

3             N/A     7.192 mAmps   1.08020 mW (0.33504 dBm)   1.00480 mW (0.02080 dBm)

MSA Data (Upper Memory Page 03)

0x0200: 4b 00 fb 00 46 00 00 00 : 00 00 00 00 00 00 00 00

0x0210: 8d cc 74 04 87 5a 7a 76 : 00 00 00 00 00 00 00 00

0x0220: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0230: 87 72 01 74 43 e2 03 a5 : 1d 4c 03 e8 1b 58 05 dc

0x0240: 87 72 02 3f 43 e2 05 a5 : 00 00 00 00 00 00 00 00

0x0250: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0260: a6 0f 00 00 00 00 00 00 : 00 00 66 66 00 00 33 33

0x0270: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

Use this command to verify for any issues.

RP/0/RP0/CPU0:BRU-SPCORE-P2#show controllers optics 0/0/0/10 summary

Port                Controller State        Admin State            LED State         Lane   Laser Bias   TX Power     RX Power     Wavelength     Optics Type       Description

(dBm)        (dBm)        (nm)

------------------  ----------------------  ---------------------  ----------------  -----  ------------ -----------  ----------   -----------    --------------    -----------

Optics 0/0/0/10     Up                      In Service             Green             0      7.1mA        0.12         0.75         850.00         QSFP28-100G-SR4

1      7.2mA        0.41         0.75

2      7.2mA        0.22         0.69

3      7.2mA        0.31         0.02

Alarms

Check the alarms for topic entries.

RP/0/RP0/CPU0:NCS#show alarms brief

--------------------------------------------------------------------------------

Active Alarms for 0/0

--------------------------------------------------------------------------------

Location        Severity     Group            Set Time                   Description

--------------------------------------------------------------------------------

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOS LANE-0 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOS LANE-1 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOS LANE-2 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOS LANE-3 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOL LANE-0 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOL LANE-1 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOL LANE-2 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX LOL LANE-3 ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-0 LOW ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-1 LOW ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-2 LOW ALARM

0/0/CPU0        Major        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-3 LOW ALARM

0/0/CPU0        Minor        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-0 LOW WARNING

0/0/CPU0        Minor        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-1 LOW WARNING

0/0/CPU0        Minor        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-2 LOW WARNING

0/0/CPU0        Minor        Software         04/26/2023 15:50:19 CEST    Optics0/0/0/35 - hw_optics:  RX POWER LANE-3 LOW WARNING

0/0/CPU0        Major        Software         05/02/2023 11:18:17 CEST    Optics0/0/0/26 - hw_optics:  OPTICS Module is not supported on this port

0/0/CPU0        Major        Software         05/02/2023 11:18:17 CEST    Optics0/0/0/26 - hw_optics: Port speed not supported on the optics

0/0/CPU0        Major        Software         05/02/2023 11:18:23 CEST    Optics0/0/0/26 - hw_optics:  RX LOS LANE-0 ALARM

0/0/CPU0        Major        Software         05/02/2023 11:18:23 CEST    Optics0/0/0/26 - hw_optics:  RX LOS LANE-1 ALARM

…

Show Controller Optics

Check if the SFP/QSFP is detected with the command “show controller optics <r/s/i/p>”.

Check the this:

- Check if the inserted optics is displayed properly (check the PID and VID).

- Check if the laser is enabled.

- Check if power levels are good.

- Check if any alarms/faults are raised.

This is sample output if the optics are detected and in service.

Check is the “Controller State” is Up or Down.

Check if the Laser State is On or Off. The state Off is seen when the interface is shut. If this is not the case, collect this:

- show tech-support optics

- show tech-support ofa

RP/0/RP0/CPU0:ios#show controllers optics 0/0/0/12

Controller State: Up

Transport Admin State: In Service

Laser State: Off

Check the LED state. The state Off is seen when the interface is shutdown. When the interface is not shutdown, the state can be Down-Yellow or Up-Green.

If there is port breakout: If even one subport is up, all subports show Green. If at least one subport is admin up (no shut), all subports show Yellow. If all subports are admin down (shut), then the Led state is Off.

Check the FEC state. It is possible that some platform types have the FEC disabled. Then the FEC part is not displayed.If FEC is enabled on PHY, it should show PHY FEC status not NPU. If the FEC status is not correct, check the FEC Status in “show controller <speed> <r/s/i/p>”.

Read this article to learn more on FEC: Understanding FEC and Its Implementation in Cisco Optics

Download this table for more details on FEC types for each Cisco optic: https://www.cisco.com/c/dam/en/us/products/se/2022/4/Collateral/fec-summary-table.pdf

Check if any alarms are raised next to “Detected Alarms”. In case of Link Issues, check if any RX-LOS/RX-LOL/TX-LOS/TX-LOL alarms are displayed here or in “show alarms brief/history” for the interface and timestamp when the link flapped. In case of RX alarms, check if the peer has TX alarms. If the peer has TX alarms, go to next step. If there are no peer TX alarms, try to reconnect the cable and or the pluggable item. Try to replace them with another if needed.

Check the TX Power. If the port is expected to be Up, but it has -40 Tx power, go to next step. Check the RX Power. If the port is expected to be Up, but it has -40dBm Rx power, check if the peer has TX alarms. If the peer has no TX alarms, try to reconnect the cable and or the pluggable item. Try to replace them with another if needed.

RP/0/RP0/CPU0:ios#show controllers optics 0/0/0/13

Controller State: Down

Transport Admin State: In Service

Laser State: Off

Optics not present

Optics Type: Unavailable

DWDM Carrier Info: Unavailable, MSA ITU Channel= Unavailable, Frequency= Unavailable , Wavelength= Unavailable

TX Power = Unavailable

RX Power = Unavailable

LED State: Off

FEC State: FEC ENABLED

Optics Status

Optics Type: SFP28 25G SR-S

Wavelength = 850.00 nm

Alarm Status:

-------------

Detected Alarms: None

LOS/LOL/Fault Status:

Laser Bias Current = 0.0 mA Actual

TX Power = -40.00 dBm

RX Power = -0.93

Performance Monitoring: Disable

THRESHOLD VALUES

----------------

Parameter                           High Alarm  Low Alarm  High Warning  Low Warning

------------------------            ----------  ---------  ------------  -----------

Rx Power Threshold(dBm)                 5.4       -14.2              2.3       -10.3

Tx Power Threshold(dBm)                 5.4       -12.4              2.3        -8.4

LBC Threshold(mA)                     10.00        2.00             8.00         3.00

Temp. Threshold(celsius)              75.00       -5.00            70.00         0.00

Voltage Threshold(volt)                3.63        2.97             3.46         3.13

Polarization parameters not supported by optics

Temperature = 28.00 Celsius               Voltage = 3.28 V

Transceiver Vendor Details

Form Factor              : SFP28

Optics type              : SFP28 25G SR-S

Name                      : CISCO-AVAGO

OUI Number               : 00.17.6a

Part Number              : SFBR-725SMZ-CS1

Rev Number               : 01

Serial Number            : AVD2227E1FU

PID                       : SFP-25G-SR-S

VID                       : V01

Date Code(yy/mm/dd) : 18/07/07

This is sample output if the optics are not detected or not in service.

RP/0/RP0/CPU0:ios# RP/0/RP0/CPU0:ios#show controllers optics 0/0/0/13

UTC Controller State: Down

Transport Admin State: In Service

Laser State: Off

Optics not present

Optics Type: Unavailable

DWDM Carrier Info: Unavailable, MSA ITU Channel= Unavailable, Frequency= Unavailable , Wavelength= Unavailable

TX Power = Unavailable          RX Power = Unavailable

CoherentDSP

If the link is down for QDD-400G-ZR-S / QDD-400G-ZRP-S, along with a verification of the alarms and data from "show controller optics", do check for alarms in "show controller coherentDSP <R/S/I/P>".

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/0/1/0

Port : CoherentDSP 0/0/1/0

Controller State : Up

Inherited Secondary State : Normal

Configured Secondary State : Normal

Derived State : In Service

Loopback mode : None

BER Thresholds : SF = 1.0E-5 SD = 1.0E-7

Performance Monitoring : Enable

Bandwidth : 400.0Gb/s

Alarm Information:LOS = 0 LOF = 0 LOM = 0

OOF = 0 OOM = 0 AIS = 0IAE = 0 B

IAE = 0 SF_BER = 0

SD_BER = 0 BDI = 0 TIM = 0

FECMISMATCH = 0 FEC-UNC = 0 FLEXO_GIDM = 0

FLEXO-MM = 0 FLEXO-LOM = 0 FLEXO-RDI = 0

FLEXO-LOF = 0

Detected Alarms : None

If LOS/LOF/BER is seen, then check the fiber connection, the remote end TX values, and the mode of operation of the local and the peer end ZR/ZRP.

ZR/ZRP supports multiple modes of operation. Check this link configuration guide.

The configuration mode, FEC, DAC, and the modulation need to match with the interface configuration and with the peer configuration.

Use this command: show optics-driver debug optics port <fp_port> instance <bayinst> location <LC>.

RP/0/RP0/CPU0:ios#show optics-driver debug optics port 0 instance 0 location 0/0/CPU0

flexcoh_hdlr : [0x0]

R_S_I_P : [0.0.1.0]

module_type : [400G-ZRPLUS:Cisco-Qualified-Module]

Traffic-Setup : [Requested/Programmed]

client_rate : [100_GAUI_2_KP4_FEC/100_GAUI_2_KP4_FEC]                      <- needs to match the configured interface speed

trunk_rate : [400G Muxponder/400G Muxponder] line_fec_mode : [oFEC/oFEC]   <- needs to match the peer end for link up

dac_rate : [1x1.25 => pulse_shaping On/1x1.25 => pulse_shaping On]         <- needs to match the peer end for link up

modulation : [16QAM/16QAM]                                                 <- needs to match the peer end for link up

framing_format : [400G_ZR/400G_ZR]

framing_mode : [Enhanced/Enhanced]

hw_cfg_in_progress : [False]

hw transactions : [3]

polling enabled : [True]

pm_notify enabled : [True]

alarms_notify enabled : [True]

sdk laser oper state : [Enabled]

sdk hw laser oper state : [Enabled]

hw laser oper state : [Enabled]

sdk channel-frequency : [1931000]

hw channel-frequency : [1931000]

sdk tx-power : [-100 0.1dBm]

hw tx-power : [-113 0.1dBm]

hw tx-power-range : [-2289, -65135 0.01dBm]

sdk cd-min : [-13000]hw cd-min : [-13000]

sdk cd-max : [13000]hw cd-max : [13000]

sdk baud-rate : [60.138546]

hw baud-rate : [60.138546]

sdk hw thresholds : [Valid]

config-thresh-flags : [0x0]

trf-cfg-lsr-pm-flags : [0x0]

polling_mask : [0xf]

is_fw_dl_in_progress : [False]

is_fw_commit_in_progress: [False]

sdk dsp-internal-loopback: [Disabled]

hw dsp-internal-loopback: [Disabled]

sdk dsp-line-loopback : [Disabled]

hw dsp-line-loopback : [Disabled]

Flexcoh SDK API execution status

--------------------------------------------------------------------------------------------------

traffic | tx-power | cd-min | cd-max | frequency | laser-set | pm-set | alarm-set | poll_set |

==================================================================================================

Success | Success  | Success| Success| Success   | Success   | Success| Success   | Success  |      <- No failure to be seen in any of the status

--------------------------------------------------------------------------------------------------

Show Controllers PHY

Collect ths information for the correct interface. This command dumps the EEPROM information.

RP/0/RP0/CPU0:ios#show controllers tenGigE 0/0/1/0 phy

SFP EEPROM port: 32

Xcvr Type: SFP

Xcvr Code: SFP+ 10G SR

Encoding: 64B66B

Bit Rate: 10300 Mbps

Link Reach 50u fiber: 80 meter

Link Reach 62.5u fiber: 20 meter

Vendor Name: CISCO-SUMITOMO

Vendor OUI: 00.00.5f

Vendor Part Number: SPP5100SR-C5 (rev.: A )

Laser wavelength: 850 nm (fraction: 0.00 nm)

Optional SFP Signal: Tx_Disable, Tx_Fault, LOS

Vendor Serial Number: SPC17050AZ0

Date Code (yy/mm/dd): 13/01/31 lot code: MA

Diagnostic Monitoring: DOM, Int. Cal.,

Enhanced Options: SW RX LOS Mon., SW TX Fault Mon, SW TX Disable, Alarm/Warning Flags

MSA Data

0x0000: 03 04 07 10 00 00 00 00 : 00 00 00 06 67 00 00 00

0x0010: 08 02 00 1e 43 49 53 43 : 4f 2d 53 55 4d 49 54 4f

0x0020: 4d 4f 20 20 00 00 00 5f : 53 50 50 35 31 30 30 53

0x0030: 52 2d 43 35 20 20 20 20 : 41 20 20 20 03 52 00 e6

0x0040: 00 1a 00 00 53 50 43 31 : 37 30 35 30 41 5a 30 20

0x0050: 20 20 20 20 31 33 30 31 : 33 31 4d 41 68 f0 03 7a

0x0060: 00 00 0b ea 11 8a 3a 43 : 9d 9c 2b 0d 84 89 fd c5

0x0070: a4 0e 5b 00 00 00 00 00 : 00 00 00 00 8b 64 8d fc

Thresholds: Alarm High Warning High Warning Low Alarm Low

Temperature: +75.000 C +70.000 C +0.000 C -5.000 C

Voltage: 3.630 Volt 3.465 Volt 3.135 Volt 2.970 Volt

Bias: 10.500 mAmps 9.000 mAmps 2.500 mAmps 2.000 mAmps

Transmit Power: 1.47910 mW (1.69998 dBm) 0.74130 mW (-1.30006 dBm) 0.18620 mW (-7.30020 dBm) 0.07410 mW (-11.30182 dBm)

Receive Power: 1.58490 mW (2.00002 dBm) 0.79430 mW (-1.00015 dBm) 0.10230 mW (-9.90124 dBm) 0.04070 mW (-13.90406 dBm)

Temperature: 24.012

Voltage: 3.304 Volt

Tx Bias: 0.000 mAmps

Tx Power: 0.000 mW (<-40.00 dBm)

Rx Power: 0.000 mW (<-40.00 dBm)

Oper. Status/Control: Tx Disabled, Rx Rate Select, LOS,

EEPROM Memory (A2 lower)

0x0100: 4b 00 fb 00 46 00 00 00 : 8d cc 74 04 87 5a 7a 75

0x0110: 14 82 03 e8 11 94 04 e2 : 39 c7 02 e5 1c f5 07 46

0x0120: 3d e9 01 97 1f 07 03 ff : 00 00 00 00 00 00 00 00

0x0130: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0140: 00 00 00 00 3f 80 00 00 : 00 00 00 00 01 00 00 00

0x0150: 01 00 00 00 01 00 00 00 : 01 00 00 00 00 00 00 27

0x0160: 18 03 81 13 00 00 00 00 : 00 00 00 00 00 00 b2 00

0x0170: 00 40 00 00 00 40 00 00 : 00 00 00 00 00 00 00 00

CLEI Code: COUIA8NCAA

Part Number: 10-2415-03 (ver.: V03 )

Temp/Alarm/Power Flags: COM, commercial 0C to 70C

Minimum Temperature: 0

Maximum Temperature: 70

Calibration Constants: LBC Scale, Temperature, Laser bias current, Output power,

Product Id: SFP-10G-SR

EEPROM Memory (A2 upper)

0x0180: 43 4f 55 49 41 38 4e 43 : 41 41 31 30 2d 32 34 31

0x0190: 35 2d 30 33 56 30 33 20 : 01 00 46 00 00 00 00 c6

0x01a0: 00 00 00 00 00 00 00 00 : 00 00 85 99 8f 00 a8 3b

0x01b0: d4 4b 00 00 1e 00 0a ff : 16 93 0f 8e 00 00 aa aa

0x01c0: 53 46 50 2d 31 30 47 2d : 53 52 20 20 20 20 20 20

0x01d0: 20 20 20 20 32 33 00 00 : 00 00 00 00 00 00 00 35

0x01e0: 14 1b 20 20 20 26 20 26 : 00 00 00 00 00 00 00 00

0x01f0: 00 00 00 00 00 fb 00 00 : ff ff ff ff 00 00 00 aa

MSA Data LOWER PAGE (QSA)

0x0000: 0d 00 02 01 00 00 01 00 : 00 00 00 00 00 00 00 00

0x0010: 00 00 00 00 00 01 00 00 : 00 00 00 00 00 00 00 00

0x0020: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0030: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0040: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0050: 00 00 44 32 50 30 0f 00 : 00 00 00 00 00 00 00 00

0x0060: 00 00 00 00 01 01 00 00 : 00 00 00 00 00 00 00 00

0x0070: 00 00 00 00 00 00 00 00 : 00 00 00 00 00 00 00 00

MSA Data UPPER PAGE (QSA)

0x0080: 0d 9b 80 00 00 00 00 00 : 00 00 00 00 00 00 00 00

0x0090: 00 00 00 00 43 49 53 43 : 4f 2d 44 4e 49 20 20 20

0x00a0: 20 20 20 20 00 30 30 30 : 43 41 5a 41 44 45 52 4f

0x00b0: 2d 30 32 20 20 20 20 20 : 30 33 00 00 00 00 46 32

0x00c0: 00 00 00 00 44 54 59 32 : 32 32 31 30 36 47 38 20

0x00d0: 20 20 20 20 32 30 31 38 : 30 35 33 30 00 00 f2 c2

0x00e0: f2 00 07 d0 45 c2 18 57 : 2b 29 67 3f 51 03 49 be

0x00f0: 37 c4 da 00 00 00 00 00 : 00 00 00 00 81 96 b0 b1

NPU

This section covers specific when the interface is directly connected to the NPU. So, there is no PHY. These ports are PHYless ports.

RP/0/RP0/CPU0:ios#show controllers twentyFiveGigE 0/0/0/12 (partial output)

Operational data for interface TwentyFiveGigE0/0/0/12:

State:    Administrative state: disabled                ==> Check if admin state display is correct

Operational state: Down (Reason: Link is shutdown)  ==> Check if link state display is correct. If interface is down with Remote fault/Link Loss (local fault), verify the peer.

MAC address information:

Operational address: fc58.9a01.8e10

Burnt-in address: fc58.9a01.8e10

Autonegotiation disabled.

Priority flow control information for interface TwentyFiveGigE0/0/0/12:

Forward error correction: Standard (Reed-Solomon)  ==> Check if FEC status is correct.

RP/0/RP0/CPU0:ios#show controllers tenGigE 0/0/0/14

Operational data for interface TenGigE0/0/0/14:

State:    Administrative state: enabled

Operational state: Down (Reason: Link loss or low light, no loopback) ==> This router has a Local Fault/Down.

LED state: Yellow On

RP/0/RP0/CPU0:ios#show controllers tenGigE 0/0/0/15

Operational data for interface TenGigE0/0/0/15:

State:

Administrative state: enabled

Operational state: Down (Reason: Remote Fault)   ==> The peer has a Fault

LED state: Yellow On

Summary

- Check if the admin state of the interface is down. If yes, then bring it to admin up state with the command "no shut" on interface.

- Check the FEC status on both the router and the peer device. If there is any mismatch, try to correct them.

- Check the auto negotiation status on both the router and the peer device. If there is any mismatch, try to correct them.

- If FEC and auto negotiation are correctly configured, check the alarms status on both the router and the peer in the “show controller optics” output.

RX-LOS, RX-LOL : Problem in the receive direction of the signal.

TX-LOS, TX-LOL : Signal received by SFP/QSFP from NPU or PHY has the issue.

If there is a Link Down/Flap with RX-LOL/RX-LOS:

- Check if peer has TX-LOS/TX-LOL.

- If there is no TX alarms, try to change the cable/pluggable (SFP/QSFP).

If there is a Link Down/Flap with TX-LOL/TX-LOS:

You can apply a loopback external to exclude the remote peer and the fiber to the remote peer. Use a fiber cable locally to physically loop the transmit (Tx) signal into the receive (Rx) port. Optionally use an attenuator, to achieve this:

RP/0/RSP0/CPU0:NCS(config)#int Hu0/3/0/31

RP/0/RSP0/CPU0:NCS(config-if)#loopback ?

external Enable external loopback (requires loopback connector)

internal Enable internal loopback

line Enable line loopback

RP/0/RSP0/CPU0:NCS(config-if)#loopback external ?

<cr>

RP/0/RSP0/CPU0:NCS(config-if)#loopback external

RP/0/RSP0/CPU0:NCS(config-if)#commit

You can apply a loopback internal config on both the router and the peer to verify the optics by a bypass of the fiber/optics. This means that if the interface still does not come up, it indicates that the issue is not with the optical part!

RP/0/RP0/CPU0:BRU-SPCORE-P2(config)#int hundredGigE 0/0/0/10

RP/0/RP0/CPU0:BRU-SPCORE-P2(config-if)#loopback internal ?

<cr>

RP/0/RP0/CPU0:BRU-SPCORE-P2(config-if)#loopback internal

RP/0/RP0/CPU0:BRU-SPCORE-P2(config-if)#commit

Collect Logs

These commands on both the router and the peer can provide a cause of the issue.

- show ip interface brief

- show alarms brief

- show controller optics <r/s/i/p> of the interface with the issue

- show tech-support optics

This displays alarms, LED status, Laser status & other optics info

- show controller <tengige/hundredgige/other> <r/s/i/p> internal of interface with the issue.

Displays FEC, AN, Up/Down, Local fault/Remote fault, and so on.

- show controllers npu voq-usage interface all instance all location all

Gives front panel port to pp port, npu, and core mapping.

Full collection of logs for the interface issue:

- show version

- show running-config

- show install active

- show platform

- show tech-support ethernet platform location 0/x/CPU0

- show tech-support ethernet interfaces

- show tech-support ethernet controllers location 0/x/CPU0

- show tech-support dpa location 0/x/CPU0

- show tech-support ofa location 0/x/CPU0

- show tech-support optics location 0/x/CPU0

- show tech-support coherent location 0/x/CPU0

- show tech-support pfi location all

- show tech-support qos platform location 0/x/CPU0

- show tech-support fabric

- show controllers npu voq-usage interface all instance 0 location all

- show optics trace all

- show ethernet infra trace location 0/x/cpu0

- show ethernet v-ether trace location 0/x/CPU0

- show vether-ea trace all location 0/x/CPU0

- show portmapper trace all location 0/0/CPU0