Troubleshoot the Network Convergence System (NCS) 1001 NCS1K-EDFA
Available Languages
Contents
Introduction
This document describes commands and procedures to troubleshoot a variety of issues on the NCS1001 amplifier modules, the NCS1K-EDFA.
Components Used
NCS1001-K9 x2
NCS1K-EDFA x4
NCS1K-PSM x2
NCS1K-CNTLR-K9 x2
NCS1K1-FAN x6
NCS1K-2KW-AC x4
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Background Information
This document assumes a working knowledge of XR 6.0.1 and higher, and understanding of basic Dense Wavelength Division Multiplexing (DWDM) principles such as optical amplification.
Optical Transport Signal (OTS) Controller Mapping
ots 0/{1|3}/0/0 ---> COM
ots 0/{1|3}/0/1 ---> LINE
ots 0/{1|3}/0/2 ---> Optical Supervisory Channel (OSC)
ots 0/{1|3}/0/3 ---> COM-CHK (RX only)
ots 0/2/0/0 ------> PSM COM
ots 0/2/0/1 ------> PSM Working (W)
ots 0/2/0/2 ------> PSM Protect (P)
Initial Configuration
For the NCS1K-EDFA to function as expected, it must have a certain minimum configuration.
Topology
This example topology represents a standard PSM-protected point-to-point network.
NCS1001 Protected Topology
For additional topologies, refer to the Solution Guide for Cisco NCS 1000 Series.
Module Configuration
The NCS1K-EDFA and, if present, the NCS1K-PSM require a hw-module configuration. This document does not detail the configuration options available. Refer to the Configuration Guide for Cisco NCS 1001 for detailed instructions.
hw-module location 0/RP0/CPU0 slot 1
ampli grid-mode 100GHz
ampli node-type TERM
!
hw-module location 0/RP0/CPU0 slot 2
psm primary-path WORKING
psm section-protection
!
hw-module location 0/RP0/CPU0 slot 3
ampli grid-mode 100GHz
ampli node-type TERM
Optical Transport Signal (OTS) Controller Configuration
Each NCS1K-EDFA ots controller must have an amplifier mode and rx-low-threshold configured. For automatic control mode, configure a channel power set point with ampli-channel-power. For manual mode, configure an amplifier gain with ampli-gain. The amplifier mode must match between endpoints. Optionally, configure the safety-control-mode and channel-power-max-delta.
controller Ots0/1/0/0
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 0
channel-power-max-delta 50
safety-control-mode auto
!
controller Ots0/1/0/1
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 30
channel-power-max-delta 50
safety-control-mode auto
!
controller Ots0/1/0/2
rx-low-threshold -280
!
controller Ots0/3/0/3
rx-low-threshold -300
!
controller Ots0/3/0/0
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 0
channel-power-max-delta 50
safety-control-mode auto
!
controller Ots0/3/0/1
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 30
channel-power-max-delta 50
safety-control-mode auto
!
controller Ots0/3/0/2
rx-low-threshold -280
!
controller Ots0/3/0/3
rx-low-threshold -300
!
Configure PSM Thresholds
The NCS1K-PSM requires at minimum an rx-low-threshold on the working (W) and protect (P) receive ports to function as expected. Best practice recommends to configure the threshold 5 dBm less than the current incoming power.
controller Ots0/2/0/1
rx-low-threshold -200
!
controller Ots0/2/0/2
rx-low-threshold -200
!
Alternately, use the psm auto-threshold for the system to automatically configure thresholds.
hw-module location 0/RP0/CPU0 slot 2
psm primary-path WORKING
psm auto-threshold
psm section-protection
!
Verify Amplifier Operation
The show controller ots command gives information on optical performance for each controller. To view optical power, amplifier gain, and other parameters, use the wildcard *.
RP/0/RP0/CPU0:NCS1001-1# show controller ots 0/3/0/* summary
Thu Sep 7 17:08:24.360 UTC
Port Type Status TX Power TX Total Power RX Power RX Total Power RX Voa Attenuation TX Voa Attenuation Ampli Gain Ampli Tilt
(dBm) (dBm) (dBm) (dBm) (dBm) (dBm)
---- ---- ------ ----------- ------------- ---------- --------------- ------------------ ------------------ ---------- ----------
Ots0_3_0_0 Com N/A 5.00 5.09 -4.90 -4.90 Unavailable Unavailable 5.00 0.00
Ots0_3_0_1 Line N/A 9.40 9.39 0.00 0.39 Unavailable Unavailable 14.30 0.00
Ots0_3_0_2 Osc N/A -11.30 Unavailable 2.50 Unavailable Unavailable Unavailable Unavailable Unavailable
Ots0_3_0_3 Com-Check N/A Unavailable Unavailable -40.00 Unavailable Unavailable Unavailable Unavailable Unavailable
The command show hw-module slot {1|3} ampli-trail-view all gives details on the amplifier state and configuration.
RP/0/RP0/CPU0:NCS1001-1# show hw-module slot 3 ampli-trail-view all
Thu Sep 7 17:14:29.956 UTC
Ampli Trail View - BST and PRE Amplifiers
=======================================================
Port: 0/COM Port: 1/LINE
-------------- --------------
Rx Power = -4.90 dBm Rx Power = 0.00 dBm
Rx Total Power = -4.90 dBm Rx Total Power = 0.40 dBm
Rx Low Threshold = -23.0 dBm Rx Low Threshold = -23.0 dBm
Port: 1/LINE Port: 0/COM
-------------- --------------
Tx Power = 9.40 dBm Tx Power = 5.00 dBm
Tx Total Power = 9.40 dBm Tx Total Power = 5.10 dBm
Tx Low Threshold = -20.0 dBm Tx Low Threshold = -20.0 dBm
Bst Gain = 14.30 dB Pre Gain = 5.00 dB
Bst Tilt = 0.00 Pre Tilt = 0.00
Bst Channel Power = 3.00 dBm Pre Channel Power = 0.00 dBm
Bst Control Mode = Auto Pre Control Mode = Auto
Bst Safety Mode = ALS Auto Pre Safety Mode = ALS Disabled
Bst Osri = Off Pre Osri = Off
Bst Gain Range = Normal Pre Gain Range = Normal
Check for Alarms and Conditions
The command show alarms { brief | detailed } system { active | history } gives the current or historical (cleared) alarms on the sytem. show logging provides additional information on system events.
RP/0/RP0/CPU0:NCS1001-2# show alarms brief system active
Thu Sep 7 14:05:41.268 UTC
------------------------------------------------------------------------------------
Active Alarms
------------------------------------------------------------------------------------
Location Severity Group Set Time Description
------------------------------------------------------------------------------------
0/1 Minor Controller 09/07/2023 14:05:27 UTC Ots0/1/0/0 - Auto Ampli Control Disabled
Some alarms are suppressed to a condition in the presence of a higher priority alarm. For example, a Loss of Signal on the LINE-RX supercedes any alarms on an individual channel related to the same OTS controller.
RP/0/RP0/CPU0:NCS1001-2# show alarms brief system conditions
Thu Sep 7 17:24:42.999 UTC
Historical alarms can help identify the cause of current alarms or conditions, and identify patterns. This example shows a pattern of Auto Ampli Control Disabled alarms raising and clearing.
RP/0/RP0/CPU0:NCS1001-2# show alarms brief system history
Thu Sep 7 17:29:46.386 UTC
------------------------------------------------------------------------------------
History Alarms
------------------------------------------------------------------------------------
Location Severity Group Set Time Description
Clear Time
------------------------------------------------------------------------------------
0/1 Minor Controller 09/07/2023 17:07:50 UTC Ots0/1/0/0 - Auto Ampli Control Disabled
09/07/2023 17:08:00 UTC
0/1 Minor Controller 09/07/2023 17:08:40 UTC Ots0/1/0/0 - Auto Ampli Control Disabled
09/07/2023 17:09:10 UTC
0/1 Minor Controller 09/07/2023 17:15:20 UTC Ots0/1/0/1 - Auto Ampli Control Disabled
09/07/2023 17:15:30 UTC
0/1 Minor Controller 09/07/2023 17:19:30 UTC Ots0/1/0/0 - Auto Ampli Control Disabled
Verify Optical Channel (OCH) Power
To verify active channels on the line system, use the command show hw-module slot {1|3} channel-trail-view active. To see all channel power levels across the spectrum, replace active with all. The channel power must remain within the user-defined thresholds, and the maximum difference between the highest and lowest channel power must remain within the threshold defined by channel-power-max-delta. The example output has a 3.3 dBm difference between channel power levels, so must have a channel-power-max-delta of at least 4 dBm.
RP/0/RP0/CPU0:NCS1001-2# show hw-module slot 3 channel-trail-view active
Thu Sep 7 17:47:07.211 UTC
Channel Trail View - Active - dBm
===========================================
BST: 0/COM=>1/LINE PRE: 1/LINE=>0/COM
Och Name Wavelength(nm) Freq(GHz) Width(GHz) Rx pwr(dBm) Tx pwr(dBm) Rx pwr(dBm) Tx pwr(dBm)
-------------------------------------------------------------------------------------------------------------------------------
Ots-Och0_3_0_0_6 1532.68 195600.0 100.0 -9.10 5.10 -11.50 0.40
Ots-Och0_3_0_0_7 1533.46 195500.0 100.0 -12.40 1.70 -11.50 0.30
Ots-Och0_3_0_0_8 1534.25 195400.0 100.0 -11.10 3.10 -11.60 0.30
Ots-Och0_3_0_0_9 1535.04 195300.0 100.0 -12.00 2.40 -13.50 -1.40
-------------------------------------------------------------------------------------------------------------------------------
Check Historical Performance Monitoring (PM)
To view the historical performance for an ots controller, use the pm history command. This can help to identify temporary power fluctuations or other transient issues no longer present on the system.
show controller ots 0/{1|3}/0/{1-3} pm history {15-min|24-hour|30-sec|flex-bin} optics 1 [ Bucket [ bucket ] ]
RP/0/RP0/CPU0:NCS1001_61# show controller ots 0/1/0/0 pm history 15-min optics 1 Bucket 1
Thu Sep 7 17:38:16.744 UTC
Optics in interval 1 [17:15:00 - 17:30:00 Thu Sep 7 2023]
Optics history bucket type : Valid
MIN AVG MAX
LBC[% ] : 9.2 9.2 9.2
OPT[dBm] : 6.80 6.80 6.80
OPR[dBm] : -4.40 -4.37 -4.30
AGN[dB] : 15.2 15.2 15.2
ATL[dB] : 0.0 0.0 0.0
Last clearing of "show controllers OPTICS" counters never
You can also view the historical performance of individual channels with show controller ots-och 0/{1|3}/0/{1-96} pm history ... command.
RP/0/RP0/CPU0:NCS1001-2# show controller ots-och 0/3/0/0/6 pm history 15-min optics 1 Bucket 1
Thu Sep 7 17:54:36.183 UTC
Optics in interval 1 [17:30:00 - 17:45:00 Thu Sep 7 2023]
Optics history bucket type : Valid
MIN AVG MAX
OPT[dBm] : 0.40 0.40 0.40
OPR[dBm] : -9.10 -9.10 -9.10
Last clearing of "show controllers OPTICS" counters never
Alarm Troubleshooting
Automatic Power Reduction (APR)
The NCS1K-EDFA enters the APR state under some conditions such as Loss of Signal (LOS), automatically reducing amplifier gain until the condition clears. While in APR, the EDFA sends an 8 second recovery pulse every 100 seconds. If the amplfiier remains in APR after the conditions clears, use the recovery steps described in the Change the Amplifier State section.
If the ots controller has safety-control-mode auto configured, the EDFA can enter APR due to the loss of the Optical Supervisory Channel (OSC) or COM-CHK. If these controllers do not have a fiber connection and are not in use, you can shut down the controller to restore the amplifier state. This example places the 0/1 OSC controller in the shutdown state.
RP/0/RP0/CPU0:NCS1001_61# configure
Thu Sep 7 18:15:35.544 UTC
RP/0/RP0/CPU0:NCS1001_61(config)# controller ots 0/1/0/2
RP/0/RP0/CPU0:NCS1001_61(config-Ots)# shutdown
RP/0/RP0/CPU0:NCS1001_61(config-Ots)# commit
Thu Sep 7 18:15:48.650 UTC
Loss of Signal or Loss of Continuity
Use the show controller ots commands to identify the source of the optical power failure.
Verify the affected controller(s) have a functional fiber connection, and confirm power levels with an optical power meter.
Auto Ampli Control Disabled
When configured in ampli-control-mode auto, the EDFA can raise the Auto Ampli Control Disabled alarm when a condition prevents it from automatically regulating.
If the difference between the highest and lowest channel power exceeds the channel-power-max delta threshold, the amplifier also raises this alarm. Identify the channel or channels causing the threshold crossing with show hw-module slot {1|3} channel-trail-view all.
Confirm the incoming power cannot cause the amplifier to exceed its maximum or minimum gain. Refer to the Cisco Network Convergence System 1001 Data Sheet for these details.
Example:
Controller ots-och 0/3/0/0/6 has a receive channel power of 2.1 dBm.
Controller ots 0/3/0/1 has the configuration:
controller Ots0/3/0/1
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 0
!
The amplifier cannot reach the channel power set point of 0 dBm as the incoming channel power exceeds the set point. The amplifier cannot provide gain until the ots 0/3/0/0/6 receive power reduces beneath the ots 0/3/0/1 set point.
Similarly, a receive power of -26.0 dBm on ots-och 0/3/0/0/6 and ampli-channel power 30 on controller ots 0/3/0/1 exceeds the maximum gain of the line amplifier, 24 dB, resulting in the AMPLI-GAIN-LOW alarm as it cannot reach the configured set point.
Incorrect or Invalid Channel Power
The presence of unexpected channel power on ots-och channels from show hw-module slot 3 channel-trail-view all can indicate an mismatch between the configured grid mode and channel spectral width. Ensure the spectral width of the incoming channels is less than the configured grid-mode configuration. For example, a 400G channel exceeds the spacing of the grid-mode 50GHz configuration, resulting in optical power readings on adjacent ots-och controllers. Instead use grid-mode 75GHz or 100GHz. Alternately, configure a custom channel width using the flex-channel-id command. For complete instructions on configuring flex channels, refer to the Configuration Guide for Cisco NCS 1001.
hw-module location 0/RP0/CPU0 slot 3
ampli grid-mode gridless
ampli flex-mode flex-spectrum
ampli flex-channel-id [ id ] chan-central-freq [ frequency ] chan-width [ width ]
!
Change the Amplifier State
Caution: These commands cause service impact.
Verify the configuration took effect with show run controller ots 0/{1|3}/0/{1-3}.
Toggle Optical Safety Remote Interlock (OSRI)
Setting OSRI to on disables the ots controller transmit power.
RP/0/RP0/CPU0:NCS1001_61# configure
Thu Sep 7 19:45:01.638 UTC
RP/0/RP0/CPU0:NCS1001_61(config)# controller ots 0/3/0/1
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# osri on
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# commit
Thu Sep 7 19:45:15.772 UTC
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# do show run controller ots 0/3/0/1
Thu Sep 7 19:45:28.214 UTC
controller Ots0/3/0/1
osri on
rx-low-threshold -230
ampli-control-mode automatic
ampli-channel-power 30
!
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# no osri on
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# commit
Thu Sep 7 19:45:57.608 UTC
Toggle the OTS Controller
RP/0/RP0/CPU0:NCS1001-2# configure
Thu Sep 7 19:45:01.638 UTC
RP/0/RP0/CPU0:NCS1001-2(config)# controller ots 0/3/0/1
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# shutdown
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# commit
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# no shutdown
RP/0/RP0/CPU0:NCS1001-2(config-Ots)# commit
Restart the Amplifier
Perform a module reload from the admin context.
RP/0/RP0/CPU0:NCS1001-2# admin
Thu Sep 7 19:52:50.273 UTC
Last login: Wed Sep 6 01:33:34 2023 from 192.0.0.4
root connected from 192.0.0.4 using ssh on sysadmin-vm:0_RP0
sysadmin-vm:0_RP0# hw-module location 0/3 reload
Thu Sep 7 19:53:01.988 UTC+00:00
Reload hardware module ? [no,yes] yes
result Card graceful reload request on 0/3 succeeded.
Restart the NCS 1001
To fully power cycle the chassis and all modules, use hw-module location all reload. The device becomes inaccessible for several minutes while it reboots.
RP/0/RP0/CPU0:NCS1001-2# admin
Thu Sep 7 19:52:50.273 UTC
Last login: Wed Sep 6 01:33:34 2023 from 192.0.0.4
root connected from 192.0.0.4 using ssh on sysadmin-vm:0_RP0
sysadmin-vm:0_RP0# hw-module location all reload
Thu Sep 7 19:53:01.988 UTC+00:00
Reload hardware module ? [no,yes] yes
Related Information
For additional information on alarm troubleshooting on the NCS1001, refer to the Troubleshooting Guide for Cisco NCS 1001.
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
04-Oct-2023 |
Initial Release |
Feedback