Upgrading Field-Programmable Device

An FPD is a field programmable logic device which contains non-volatile, re-programmable memory to define its internal wiring and functionality. The contents of this non-volatile memory are called the FPD image or FPD firmware. Over the lifespan of an FPD, FPD firmware images may need upgrades for bug fixes or functionality improvements. These upgrades are performed in the field with minimum system impact.

Prerequisites for FPD Image Upgrades

You must install the FPD pie before you install the SMUs or Service Packs. If you install the SMU or Service Packs before the FPD pie, the FPDs on the line card may not upgrade. In such cases, you must remove the SMUs and Service Packs and reload the router.

Overview of FPD Image Upgrade Support

An FPD image is used to upgrade the software on an FPD.

FPD versions must be compatible with the Cisco IOS XR software that is running on the router; if an incompatibility exists between an FPD version and the Cisco IOS XR software, the device with the FPGA may not operate properly until the incompatibility is resolved.


Note

  • It is mandatory to upgrade all the required FPDs before doing a reload when you are upgrading FPDs on line cards. This is because, partial FPD component upgrades might result in booting errors (in some cases).

  • You must not reload any line card or the router before all FPD image upgrades are completed successfully.

Note

Cisco NCS5500 routers do not support auto upgrade for PSU FPDs.

FPD upgrade service

The main tasks of the FPD upgrade service are:

  • Check FPD image version to decide if a specific firmware image needs an upgrade or not.

  • Manual FPD Image Upgrade using the upgrade hw-module fpd command.

  • Invoke the appropriate device driver with a name of the new image to load.

An FPD image package is used to upgrade FPD images. The install activate command is used to place the FPD binary files into the expected location on the boot devices.

Supported Upgrade Methods

Method

Remarks

Manual Upgrade

Upgrade using CLI, force upgrade supported.

Determining Upgrade Requirement

Use the show hw-module fpd command to determine if an FPD upgrade is required. Check for NEED UPGD in the Status column.

Example

Router: #show hw - module fpd

Wed Dec 14 07:08:08.424 UTC
 
Auto-upgrade:Disabled
                                                                         FPD Versions
                                                                        ==============
Location   Card type             HWver FPD device       ATR Status    Running   Programd 
-----------------------------------------------------------------------------------------   
0/0        NC55-18H18F           1.0   MIFPGA               NEED UPGD   7.01      7.01      
0/0        NC55-18H18F           1.0   Bootloader           CURRENT     1.14      1.14   
0/0        NC55-18H18F           1.0   IOFPGA               CURRENT     0.07      0.07   
0/0        NC55-18H18F           1.0   SATA-M600-MCT        CURRENT     0.23      0.23

Use the show fpd package command to find out which FPGAs are supported with your current software release and minimum hardware requirements for each module.

Manual FPD upgrade

Manual FPD upgrade is performed using the upgrade hw-module fpd command. All cards or all FPGA in a card can be upgraded. If reload is required to activate FPD, the upgrade should be complete. Line-cards, fabric cards and RP cards cannot be reloaded during the process of the FPD upgrade.

FPD upgrade is transaction-based:

  • Each fpd upgrade CLI execution is one transaction.

  • Only one transaction is allowed at any given time.

  • One transaction may include one or many FPD upgrades

The force option can be used to forcibly upgrade the FPD (regardless of whether it is required or not). It triggers all FPDs to be upgraded or downgraded. The force option can also be used to downgrade or upgrade the FPGAs even after the version check.


Note

  • Sometimes, FPDs can have primary and backup images.

  • Force FPD upgrade with upgrade hw-module location all fpd all force command affects forwarding over BVI interface. You must reload involved locations to recover.

  • The use of the force option when performing an FPD upgrade is not recommended except under explicit direction from Cisco engineering or TAC for a one-time purpose only.

  • FPD upgrade should be performed in Admin mode only.

  • A new FPD upgrade should be issued only when previous FPD upgrades have been completed on the same FPD with the following syslog message: 

    RP/0/RP0/CPU0:May 10 10:11:44.414 UTC: fpd-serv[205]: %INFRA-FPD_Manager-1-UPGRADE_ALERT : FPD Upgrade Completed (use "show hw-module fpd" to check upgrade status)

How to Upgrade FPD Images

You must determine if an FPD image upgrade is needed using the show hw-module fpd command and perform the upgrade, if needed, under the following circumstances:

  • Migrate the software to a later Cisco IOS XR software release.

  • Swap line cards from a system running a different Cisco IOS XR software release.

  • Insert a new line card.

In the event of an FPD incompatibility with your card, you might receive the following error message:

LC/0/0/CPU0:Jul 5 03:00:18.929 UTC: optics_driver[220]: %L2-OPTICS-3-BAD_FPGA_IMAGE : Detected bad MI FPGA image programmed in MI FPGA SPI flash in 0/0/CPU0 location: Failed to validate meta data CRC 
LC/0/0/CPU0:Jul 5 03:00:19.019 UTC: optics_driver[220]: %L2-OPTICS-3-BACKUP_FPGA_LOADED : Detected Backup FPGA image running on 0/0/CPU0 - primary image corrupted (@0x8c = 0x44)
RP/0/RP0/CPU0:Jul 5 03:00:48.987 UTC: fpd-serv[301]: %PKT_INFRA-FM-3-FAULT_MAJOR : ALARM_MAJOR :FPD-NEED-UPGRADE :DECLARE :0/0:

Upgrades to the Cisco IOS XR software might result in an FPD incompatibility. Ensure that you perform the FPD upgrade procedure and resolve all incompatibilities, for the cards to function properly.


Note

The use of the force option when performing a FPD upgrade is not recommended except under explicit direction from Cisco engineering or TAC for a one-time purpose only.


Note

Starting with Cisco IOS XR Release 24.1.1, line card reload after PSU FPD upgrade is no longer needed.
Before you begin

  • You must have a redundant PSU available before upgrading the FPD image for the PSU.

  • The FPD upgrade procedure is performed while the card is online. At the end of the procedure the card must be reloaded before the FPD upgrade is complete. To reload the card, you can use the hw-module location <location> reload command in Admin mode, during the next maintenance window. The upgrade procedure is not complete until the card is reloaded.

  • During the FPD upgrade, you must not do the following:

    • Reload, perform an online insertion and removal (OIR) of a line card (LC), or power down the chassis. Doing so may cause the node to enter an unusable state.

    • Press Ctrl-C if the console appears to hang without any output. Doing so may abort the upgrade.

  • If you are not sure whether a card requires an FPD upgrade, you can install the card and use the show hw-module fpd command to determine if the FPD image on the card is compatible with the currently running Cisco IOS XR software release.

Procedure
  Command or Action Purpose

Step 1

show hw-module fpd location {all | node-id}

Example:

RP/0/RP0/CPU0:router# show hw-module fpd location all

or 


RP/0/RP0/CPU0:router# show hw-module fpd location 0/4/cpu0

Displays the current FPD image versions for the specified card or all cards installed in the router. Use this command to determine if you must upgrade the FPD image on your card.

Step 2

admin

Example:

RP/0/RP0/CPU0:router# admin

Enters mode.

Step 3

(Optional) show fpd package

Example:
 (Optional)

Displays which cards are supported with your current Cisco IOS XR software release, which FPD image you need for each card, and what the minimum hardware requirements are for the various modules. (A minimum hardware requirement version of 0.0 indicates that all hardware can support this FPD image version.)

If there are multiple FPD images for your card, use this command to determine which FPD image to use if you want to upgrade only a specific FPD type.

Step 4

upgrade hw-module fpd {all | fpga-type} [ force] location [all | node-id]

Example:

# upgrade hw-module fpd 
all location 0/3/1 
.
.
.
Successfully upgraded 1 FPD for SPA-2XOC48POS/RPR 
  on location 0/3/1

Upgrades all the current FPD images that must be upgraded on the specified card with new images.

Before continuing to the next step, wait for confirmation that the FPD upgrade has successfully completed. Status messages, similar to these, are displayed to the screen until the FPD upgrade is completed: 


FPD upgrade started.
FPD upgrade in progress..
FPD upgrade in progress..
FPD upgrade sent to location xxxx
FPD upgrade sent to location yyyy
FPD upgrade in progress..
FPD upgrade finished for location xxx
FPD upgrade in progress..
FPD upgrade finished for location yyyy
FPD upgrade completed.

The “FPD upgrade in progress.” message is printed every minute. These logs are information logs, and as such, are displayed if the logging console informational command is configured.

If Ctrl-C is pressed while the FPD upgrade is in progress, the following warning message is displayed: 


FPD upgrade in progress on some hardware, 
aborting now is not recommended as it might 
cause HW programming failure and result in 
RMA of the hardware.
Do you want to continue? [Confirm(y/n)]

If you confirm that you want to abort the FPD upgrade procedure, this message is displayed: 

 
FPD upgrade process has been aborted, please 
check the status of the hardware and reissue 
the upgrade command if required.

Note

 

  • If your card supports multiple FPD images, you can use the show fpd package admin command to determine what specific image to upgrade in the upgrade hw-module fpd command.

  • A message is displayed when router modules cannot get upgraded during upgrade with location all option indicating that the FPGA is intentionally skipped during upgrade. To upgrade such FPGAs, you can use the CLI command with a particular location explicitly specified. For example, upgrade hw-module fpd all location 0/3/1 .

  • It is recommended to upgrade all FPGAs on a given node using the upgrade hw-module fpd all location {all | node-id} command. Do not upgrade the FPGA on a node using the upgrade hw-module fpd <individual-fpd> location {all | node-id} as it may cause errors in booting the card.

Step 5

exit

Example:
sysadmin-vm:0_RP0# exit

Step 6

hw-module location{ node-id | all } reload

Use the hw-module location reload command to reload a line card. 
sysadmin-vm:0_RP0# hw-module location 0/3 reload

Step 7

exit

Step 8

show hw-module fpd

Verifies that the FPD image on the card has been successfully upgraded by displaying the status of all FPDs in the system.

Configuration Examples for FPD Image Upgrade

The following examples indicates the use of commands associated with the FPD image upgrade procedure.

show fpd package Command Output: Example

Use the show fpd package command in System Admin EXEC mode to find out which line cards are supported with your current Cisco IOS XR software release, which FPD image package you need for each line card, and what the minimum hardware requirements are for each module. If multiple FPD images are available for your card, they are listed as Subtype fpga2, fpga3, and so on.


Note

The FPD name used in the FPD Description column of the output of the show fpd package command includes the last ten characters of DCO-PID. Depending on the slot and port numbers, the FPD name is appended with DCO_0, DCO_1, or DCO_2. For example, the FPD names for CFP2-WDM-D-1HL in port 0 and port 1 are -WDM-D-1HL_DCO_0 and WDM-D-1HL_DCO_1 respectively.

The following example shows sample output from the show fpd package command: 

show fpd package
Tue Jan 22 13:56:00.212 UTC
 
=============================== ================================================
                                        Field Programmable Device Package
                                ================================================
                                               Req     SW      Min Req   Min Req
Card Type            FPD Description           Reload  Ver     SW Ver  Board Ver
=================== ========================== ====== ======= ======== =========
--------------------------------------------------------------------------------
NC55-1200W-ACFW      LIT-PriMCU-ACFW(A)         NO      2.09    2.09    0.0 
--------------------------------------------------------------------------------
NC55-900W-ACFW-I     LIT-PriMCU-ACFW-I(A)       NO      1.04    1.04    0.0 
--------------------------------------------------------------------------------
NC55-900W-DCFW-I     LIT-PriMCU-DCFW-I(A)       NO      2.260   2.260   0.0 
--------------------------------------------------------------------------------
NC55-930W-DCFW-C     LIT-PriMCU-DCFW-C(A)       NO      2.259   2.259   0.0 
--------------------------------------------------------------------------------
NC55-MPA-12T-S       MPAFPGA                    YES     0.27    0.27    0.0 
--------------------------------------------------------------------------------
NC55-MPA-1TH2H-S     -WDM-D-1HL_DCO_2           NO     38.518  38.518   0.1 
                     MPAFPGA                    YES     0.53    0.53    0.0 
                     WDM-DE-1HL_DCO_2           NO     38.518  38.518   0.1 
                     WDM-DS-1HL_DCO_2           NO     38.268  38.268   0.1 
--------------------------------------------------------------------------------
NC55-MPA-2TH-HX-S    -WDM-D-1HL_DCO_0           NO     38.518  38.518   0.1 
                     -WDM-D-1HL_DCO_1           NO     38.518  38.518   0.1 
                     MPAFPGA                    YES     0.53    0.53    0.0 
                     WDM-DE-1HL_DCO_0           NO     38.518  38.518   0.1 
                     WDM-DE-1HL_DCO_1           NO     38.518  38.518   0.1 
                     WDM-DS-1HL_DCO_0           NO     38.268  38.268   0.1 
                     WDM-DS-1HL_DCO_1           NO     38.268  38.268   0.1 
--------------------------------------------------------------------------------
NC55-MPA-2TH-S       -WDM-D-1HL_DCO_0           NO     38.518  38.518   0.1 
                     -WDM-D-1HL_DCO_1           NO     38.518  38.518   0.1 
                     MPAFPGA                    YES     0.53    0.53    0.0 
                     WDM-DE-1HL_DCO_0           NO     38.518  38.518   0.1 
                     WDM-DE-1HL_DCO_1           NO     38.518  38.518   0.1 
                     WDM-DS-1HL_DCO_0           NO     38.268  38.268   0.1 
                     WDM-DS-1HL_DCO_1           NO     38.268  38.268   0.1 
--------------------------------------------------------------------------------
NC55-MPA-4H-HD-S     MPAFPGA                    YES     0.53    0.53    0.0 
--------------------------------------------------------------------------------
NC55-MPA-4H-HX-S     MPAFPGA                    YES     0.53    0.53    0.0 
--------------------------------------------------------------------------------
NC55-MPA-4H-S        MPAFPGA                    YES     0.53    0.53    0.0 
--------------------------------------------------------------------------------
NC55A2-MOD-SE-H-S    Bootloader(A)              YES     1.11    1.11    0.0 
                     CPU-IOFPGA(A)              YES     1.18    1.18    0.1 
                     MB-IOFPGA(A)               YES     0.18    0.18    0.1 
                     MB-MIFPGA                  YES     0.19    0.19    0.0 
                     SATA(A)                    NO      5.00    5.00    0.0 
--------------------------------------------------------------------------------
NCS-55A2-MOD-HD-S    Bootloader(A)              YES     1.11    1.11    0.0 
                     CPU-IOFPGA(A)              YES     1.18    1.18    0.1 
                     MB-IOFPGA(A)               YES     0.18    0.18    0.1 
                     MB-MIFPGA                  YES     0.19    0.19    0.0 
                     SATA(A)                    NO      5.00    5.00    0.0 
--------------------------------------------------------------------------------
NCS-55A2-MOD-HX-S    Bootloader(A)              YES     1.11    1.11    0.0 
                     CPU-IOFPGA(A)              YES     1.18    1.18    0.1 
                     MB-IOFPGA(A)               YES     0.18    0.18    0.1 
                     MB-MIFPGA                  YES     0.19    0.19    0.0 
                     SATA(A)                    NO      5.00    5.00    0.0 
--------------------------------------------------------------------------------
NCS-55A2-MOD-S       Bootloader(A)              YES     1.11    1.11    0.0 
                     CPU-IOFPGA(A)              YES     1.18    1.18    0.1 
                     MB-IOFPGA(A)               YES     0.18    0.18    0.1 
                     MB-MIFPGA                  YES     0.19    0.19    0.0 
                     SATA(A)                    NO      5.00    5.00    0.0 
--------------------------------------------------------------------------------
NCS-55A2-MOD-SE-S    Bootloader(A)              YES     1.11    1.11    0.0 
                     CPU-IOFPGA(A)              YES     1.18    1.18    0.1 
                     MB-IOFPGA(A)               YES     0.18    0.18    0.1 
                     MB-MIFPGA                  YES     0.19    0.19    0.0 
                     SATA(A)                    NO      5.00    5.00    0.0 
                     STATSFPGA                  YES     0.01    0.01    0.0

This table describes the significant fields shown in the display:

Table 1. show fpd package Field Descriptions

Field

Description

Card Type

Module part number.

FPD Description

Description of all FPD images available for the line card.

Type

Hardware type. Possible types can be:

  • spa—Shared port adapter

  • lc—Line card

Subtype

FPD subtype. These values are used in the upgrade hw-module fpd command to indicate a specific FPD image type to upgrade.

SW Version

FPD software version recommended for the associated module running the current Cisco IOS XR software.

Min Req SW Vers

Minimum required FPD image software version to operate the card. Version 0.0 indicates that a minimum required image was not programmed into the card.

Min Req HW Vers

Minimum required hardware version for the associated FPD image. A minimum hardware requirement of version 0.0 indicates that all hardware can support this FPD image version.

upgrade hw-module fpd Command Output: Example

Use the upgrade hw-module fpd command to upgrade the FPD image on a line card. The upgrade can be executed for all FPDs or for specific FPDs that need an upgrade. To upgrade all FPDs, use upgrade hw-module fpd all location all command. To upgrade a specific FPD image type, use the FPD subtype value in the upgrade hw-module fpd command.

show platform Command Output: Example

Use the show platform command to verify that the line card is up and running.

Auto FPD Upgrade

Table 2. Feature History Table

Feature Name

Release Information

Feature Description

Auto FPD Upgrade

Release 7.3.2

This functionality enables automatic upgrade and reload for field-programmable devices (FPDs) whenever the Cisco IOS XR image has a newer FPD version. This functionality upgrades all route processors and line card FPDs simultaneously while displaying upgrade triggers on the console.

Effective Cisco IOS XR Release 7.3.2, you can enable automatic upgrade of FPD by using the “fpd auto-upgrade enable” command.

To automatically upgrade all FPDs, use:

RP/0/RP0/CPU0:IOS(config)#fpd auto-upgrade enable

To reload the interface modules following the fpd auto-upgrade, use:

 RP/0/RP0/CPU0:IOS(config)#fpd auto-reload enable

Limitations and Usage Guidelines

Limitations

  • FPD auto-upgrade should be enabled only in the XR VM and not in the System Admin VM.

  • With auto-upgrade enabled, if any card is in RELOAD REQUIRED state, auto-upgrade is re-triggered during any SSO or FPD-serv process restart.

  • When an interface module (IM) or route processor (RP) is in RELOAD REQUIRED state and auto-upgrade is enabled, FPD upgrades are triggered again.

  • With auto-upgrade enabled, if line card is inserted, an auto-upgrade is triggered. During this phase optics alarms are generated. If auto-reload is not enabled, you must reload the line cards manually to clear these alarms.

  • SATA allows you to upgrade or downgrade when an FPD version change is available. Therefore, when auto-upgrade is enabled, the system automatically downgrades if lower versions are available. This behavior is specific only to SATA FPDs.

  • FPD auto-reload is applicable for line cards only. Line cards are automatically reloaded after the fpd auto-upgrade process is completed.

  • You must disable auto-upgrade during XR ISSU; otherwise, the router goes into a state where redundancy cannot be achieved. In this case, standby RP must be reloaded to achieve redundancy.

  • Cisco NCS 5500 Series Routers do not support ISSU.

  • TimingICs do not support auto fpd upgrade on NCS5500 Series Routers as the TimingIC requires a card reload immediately after upgrade. For the same reason, the TimingICs are not upgraded if the user specifies location all in the auto fpd upgrade command. To upgrade a TimingIC FPD, specify the FPD name along with the card location. For example, upgrade hw-module fpd TimngIC-A location 0/RP0/cpu0.

  • Auto FPD upgrade is not supported on the following line cards that support QDD-400G-ZR-S and QDD-400G-ZRP-S modules:

    • NC57-24DD

    • NC57-18DD-SE

    • NC57-36H6D-S

    • NC57-48Q2D-S

    • NC57-MPA-2D4H-S

    • NC57-MOD-S

    • NC57-MPA-1FH1D-S

Usage Guidelines—Online Insertion of Line Cards

When a line card with a lower FPD version is inserted, one of the following scenarios apply:

  • If fpd auto-upgrade and auto-reload are enabled, and a new line card is inserted, the system upgrades the line card FPDs automatically with the latest FPDs and reloads the line cards.

  • If fpd auto-upgrade and auto-reload are both disabled, no action is required.

  • If fpd auto-upgrade is enabled and auto-reload is disabled, the following alarms are displayed on the console: 

    RP/0/RP1/CPU0:Jun 1 10:05:46.095 UTC: optics_driver[231]: %PKT_INFRA-FM-3-FAULT_MAJOR : ALARM_MAJOR :OPTICS SUPPORTED_ERROR :DECLARE : Optics0/5/0/6: Optics0/5/0/6
RP/0/RP1/CPU0:Jun 1 10:05:46.096 UTC: optics_driver[231]: %PKT_INFRA-FM-2-FAULT_CRITICAL : ALARM_CRITICAL :OPTICS NOT SUPPORTED :DECLARE : Optics0/5/0/6: Optics0/5/0/6

    You must reload the line cards manually to clear these alarms

Usage Guidelines—Online Insertion of RPs

When fpd auto-upgrade is enabled and a new RP is inserted, the system upgrades the RP FPDs automatically with the latest FPDs.


Note

RPs are not reloaded automatically. You must manually reload the RP or chassis for the latest FPD version to reflect.

Note

Reload of active RPs and line cards impacts the network traffic.
Table 3. Action Required on FPDs After Auto Upgrade

FPD

Action Required

IOFPGA

Manual reload required

ADM

Upgraded version available immediately

PRIMARY-BIOS

Manual reload required

SATA

Upgraded version available immediately

PSOC

Upgraded version available immediately

IMFPGA

Manual reload required, if auto-reload is not configured

Automatic FPD Upgrade for PSU

During the Power Supply Unit (PSU) insertion and installation process, the routers can now automatically upgrade the Field-Programmable Devices (FPD) associated with the PSUs.

Starting with Cisco IOS-XR Release 7.5.2, the automatic FPD upgrade includes the FPDs associated with the PSUs by default. This means that when automatic FPD upgrade is enabled, the FPDs associated with the PSUs will also be upgraded. The upgrades for the PSUs will occur sequentially, so the FPD upgrades for the PSUs will take longer than for other components.

You can choose to exclude PSUs from the automatic upgrade process to reduce the time taken for FPD automatic upgrade by preventing them from being upgraded upon insertion or during a system upgrade using the fpd auto-upgrade exclude pm command.

Configuration example for excluding PSUs from automatic FPD upgrade:

Configuration

Router# config
Router(config)# fpd auto-upgrade enable
Router(config)# fpd auto-upgrade exclude pm
Router(config)# commit

Show Running Configuration

Router# show running-config fpd auto-upgrade
fpd auto-upgrade enable
fpd auto-upgrade include pm

Upgrade Failure

On failure of an FPD upgrade, you get a warning with the following syslog message:

LC/0/5/CPU0:Jun 27 05:02:25.742 UTC: optics_driver[216]: %INFRA-FPD_Driver-1-UPGRADE_ALERT : FPD MIFPGA@0/5 image programming completed with UPGD FAIL state Info: [Image verification failed at offset 0x5c8, flash value = 0x0, image value = 0x40, image size = 4194304] 
LC/0/5/CPU0:Jun 27 05:02:26.570 UTC: optics_driver[216]: %INFRA-FPD_Driver-1-UPGRADE_ALERT : FPD MIFPGA@0/5 image programming completed with UPGD FAIL state Info: [Image verification failed at offset 0x1e, flash value = 0x56, image value = 0xff, image size = 4194304]

When you use the show hw-module fpd command, the status column displays UPGD FAIL to indicate failure of the FPD upgrade.


Note

  • Do not reload the line card with a failed FPD upgrade image.

  • Upgrade failed FPDs will be fixed with a manual upgrade.

  • Contact Cisco TAC or your account representative if the FPD upgrade failure is not repaired.