Upgrading Field-Programmable Device

An FPD is a field programmable logic device which contains non-volatile, re-programmable memory to define its internal wiring and functionality. The contents of this non-volatile memory are called the FPD image or FPD firmware. Over the lifespan of an FPD, FPD firmware images may need upgrades for bug fixes or functionality improvements. These upgrades are performed in the field with minimum system impact.

Prerequisites for FPD Image Upgrades

You must install the FPD pie before you install the SMUs or Service Packs. If you install the SMU or Service Packs before the FPD pie, the FPDs on the line card may not upgrade. In such cases, you must remove the SMUs and Service Packs and reload the router.

Overview of FPD Image Upgrade Support

An FPD image is used to upgrade the software on an FPD.

FPD versions must be compatible with the Cisco IOS XR software that is running on the router; if an incompatibility exists between an FPD version and the Cisco IOS XR software, the device with the FPGA may not operate properly until the incompatibility is resolved.


Note


  • It is mandatory to upgrade all the required FPDs before doing a reload when you are upgrading FPDs on line cards. This is because, partial FPD component upgrades might result in booting errors (in some cases).

  • You must not reload any line card or the router before all FPD image upgrades are completed successfully.


FPD upgrade service

The main tasks of the FPD upgrade service are:

  • Check FPD image version to decide if a specific firmware image needs an upgrade or not.

  • Automatic FPD Image Upgrade (if enabled).

  • Manual FPD Image Upgrade using the upgrade hw-module fpd command.

  • Invoke the appropriate device driver with a name of the new image to load.

An FPD image package is used to upgrade FPD images. The install activate command is used to place the FPD binary files into the expected location on the boot devices.

Supported Upgrade Methods

Method

Remarks

Manual Upgrade

Upgrade using CLI, force upgrade supported.

Auto Upgrade

Upgrade using install SMU activation or during image upgrade. User can enable/disable auto upgrade feature.

Determining Upgrade Requirement

Use the show hw-module fpd command to determine if an FPD upgrade is required. Check for NEED UPGD in the Status column.

Example

Router: #show hw - module fpd

Wed Dec 14 07:08:08.424 UTC
 
Auto-upgrade:Disabled
                                                                         FPD Versions
                                                                        ==============
Location   Card type             HWver FPD device       ATR Status    Running   Programd 
-----------------------------------------------------------------------------------------   
0/0        NC55-18H18F           1.0   MIFPGA               NEED UPGD   7.01      7.01      
0/0        NC55-18H18F           1.0   Bootloader           CURRENT     1.14      1.14   
0/0        NC55-18H18F           1.0   IOFPGA               CURRENT     0.07      0.07   
0/0        NC55-18H18F           1.0   SATA-M600-MCT        CURRENT     0.23      0.23   

Use the show fpd package command to find out which FPGAs are supported with your current software release and minimum hardware requirements for each module.

Manual FPD upgrade

Manual FPD upgrade is performed using the upgrade hw-module fpd command. All cards or all FPGA in a card can be upgraded. If reload is required to activate FPD, the upgrade should be complete. Line-cards, fabric cards and RP cards cannot be reloaded during the process of the FPD upgrade.

FPD upgrade is transaction-based:

  • Each fpd upgrade CLI execution is one transaction.

  • Only one transaction is allowed at any given time.

  • One transaction may include one or many FPD upgrades

The force option can be used to forcibly upgrade the FPD (regardless of whether it is required or not). It triggers all FPDs to be upgraded or downgraded. The force option can also be used to downgrade or upgrade the FPGAs even after the version check.


Note


  • Sometimes, FPDs can have primary and backup images.

  • Force FPD upgrade with upgrade hw-module location all fpd all force command affects forwarding over BVI interface. You must reload involved locations to recover.

  • The use of the force option when performing an FPD upgrade is not recommended except under explicit direction from Cisco engineering or TAC for a one-time purpose only.

  • FPD upgrade should be performed in Admin mode only.

  • A new FPD upgrade should be issued only when previous FPD upgrades have been completed on the same FPD with the following syslog message:

    RP/0/RP0/CPU0:May 10 10:11:44.414 UTC: fpd-serv[205]: %INFRA-FPD_Manager-1-UPGRADE_ALERT : FPD Upgrade Completed (use "show hw-module fpd" to check upgrade status)


Note


On the Cisco N540-FH-CSR-SYS and Cisco N540-FH-AGG-SYS routers, when upgrading or downgrading from Cisco IOS XR Releases 7.3.2, 7.4.x, or 7.5.1 to Release 7.4.1, data path FPGA (DPFPGA) ports do not come up due to the 'layer 1 port creation' failure. For the DPFPGA ports to come up successfully, you must manually upgrade the DPFPGA after the software upgrade or downgrade to Cisco IOS XR Release 7.4.1.

DPFPGA ports:

  • On N540-FH-CSR-SYS: Ports 0-13

  • On N540-FH-AGG-SYS: Ports 0-23


How to Upgrade FPD Images

You must determine if an FPD image upgrade is needed using the show hw-module fpd command and perform the upgrade, if needed, under the following circumstances:

  • Migrate the software to a later Cisco IOS XR software release.

In the event of an FPD incompatibility with your card, you might receive the following error message:

LC/0/0/CPU0:Jul 5 03:00:18.929 UTC: optics_driver[220]: %L2-OPTICS-3-BAD_FPGA_IMAGE : Detected bad MI FPGA image programmed in MI FPGA SPI flash in 0/0/CPU0 location: Failed to validate meta data CRC 
LC/0/0/CPU0:Jul 5 03:00:19.019 UTC: optics_driver[220]: %L2-OPTICS-3-BACKUP_FPGA_LOADED : Detected Backup FPGA image running on 0/0/CPU0 - primary image corrupted (@0x8c = 0x44)
RP/0/RP0/CPU0:Jul 5 03:00:48.987 UTC: fpd-serv[301]: %PKT_INFRA-FM-3-FAULT_MAJOR : ALARM_MAJOR :FPD-NEED-UPGRADE :DECLARE :0/0:

Upgrades to the Cisco IOS XR software might result in an FPD incompatibility. Ensure that you perform the FPD upgrade procedure and resolve all incompatibilities, for the cards to function properly.


Note


The use of the force option when performing a FPD upgrade is not recommended except under explicit direction from Cisco engineering or TAC for a one-time purpose only.


Before you begin
  • The FPD upgrade procedure is performed while the card is online. At the end of the procedure the card must be reloaded before the FPD upgrade is complete. To reload the card, you can use the hw-module location <location> reload command in Admin mode, during the next maintenance window. The upgrade procedure is not complete until the card is reloaded.

  • During the FPD upgrade, you must not do the following:

    • Reload, perform an online insertion and removal (OIR) of a line card (LC), or power down the chassis. Doing so may cause the node to enter an unusable state.

    • Press Ctrl-C if the console appears to hang without any output. Doing so may abort the upgrade.

  • If you are not sure whether a card requires an FPD upgrade, you can install the card and use the show hw-module fpd command to determine if the FPD image on the card is compatible with the currently running Cisco IOS XR software release.

Configuration Examples for FPD Image Upgrade

The following examples indicates the use of commands associated with the FPD image upgrade procedure.

show fpd package Command Output: Example

Use the show fpd package command in System Admin EXEC mode to find out which line cards are supported with your current Cisco IOS XR software release, which FPD image package you need for each line card, and what the minimum hardware requirements are for each module. If multiple FPD images are available for your card, they are listed as Subtype fpga2, fpga3, and so on.


Note


The FPD name used in the FPD Description column of the output of the show fpd package command includes the last ten characters of DCO-PID. Depending on the slot and port numbers, the FPD name is appended with DCO_0, DCO_1, or DCO_2. For example, the FPD names for CFP2-WDM-D-1HL in port 0 and port 1 are -WDM-D-1HL_DCO_0 and WDM-D-1HL_DCO_1 respectively.


upgrade hw-module fpd Command Output: Example

Use the upgrade hw-module fpd command to upgrade the FPD image. The upgrade can be executed for all FPDs or for specific FPDs that need an upgrade. To upgrade all FPDs, use upgrade hw-module fpd all location all command. To upgrade a specific FPD image type, use the FPD subtype value in the upgrade hw-module fpd command.

show platform Command Output: Example

Use the show platform command to verify that the line card is up and running.

Auto FPD Upgrade

Table 1. Feature History Table

Feature Name

Release Information

Feature Description

Auto FPD Upgrade

Release 7.3.2

This functionality enables automatic upgrade and reload for field-programmable devices (FPDs) whenever the Cisco IOS XR image has a newer FPD version. This functionality upgrades all route processors and line card FPDs simultaneously while displaying upgrade triggers on the console.

Effective Cisco IOS XR Release 7.3.2, you can enable automatic upgrade of FPD by using the “fpd auto-upgrade enable” command.

To automatically upgrade all FPDs, use:

RP/0/RP0/CPU0:IOS(config)#fpd auto-upgrade enable

To reload the interface modules following the fpd auto-upgrade, use:

 RP/0/RP0/CPU0:IOS(config)#fpd auto-reload enable

Limitations and Usage Guidelines

Limitations

  • FPD auto-upgrade should be enabled only in the XR VM and not in the System Admin VM.

  • With auto-upgrade enabled, if any card is in RELOAD REQUIRED state, auto-upgrade is re-triggered during any SSO or FPD-serv process restart.

  • When an interface module (IM) or route processor (RP) is in RELOAD REQUIRED state and auto-upgrade is enabled, FPD upgrades are triggered again.

  • With auto-upgrade enabled, if line card is inserted, an auto-upgrade is triggered. During this phase optics alarms are generated. If auto-reload is not enabled, you must reload the line cards manually to clear these alarms.

  • SATA allows you to upgrade or downgrade when an FPD version change is available. Therefore, when auto-upgrade is enabled, the system automatically downgrades if lower versions are available. This behavior is specific only to SATA FPDs.

  • FPD auto-reload is applicable for line cards only. Line cards are automatically reloaded after the fpd auto-upgrade process is completed.

  • You must disable auto-upgrade during XR ISSU; otherwise, the router goes into a state where redundancy cannot be achieved. In this case, standby RP must be reloaded to achieve redundancy.

  • TimingICs do not support auto fpd upgrade on NCS5500 Series Routers as the TimingIC requires a card reload immediately after upgrade. For the same reason, the TimingICs are not upgraded if the user specifies location all in the auto fpd upgrade command. To upgrade a TimingIC FPD, specify the FPD name along with the card location. For example, upgrade hw-module fpd TimngIC-A location 0/RP0/cpu0.

Usage Guidelines—Online Insertion of Line Cards

When a line card with a lower FPD version is inserted, one of the following scenarios apply:

  • If fpd auto-upgrade and auto-reload are enabled, and a new line card is inserted, the system upgrades the line card FPDs automatically with the latest FPDs and reloads the line cards.

  • If fpd auto-upgrade and auto-reload are both disabled, no action is required.

  • If fpd auto-upgrade is enabled and auto-reload is disabled, the following alarms are displayed on the console:

    RP/0/RP1/CPU0:Jun 1 10:05:46.095 UTC: optics_driver[231]: %PKT_INFRA-FM-3-FAULT_MAJOR : ALARM_MAJOR :OPTICS SUPPORTED_ERROR :DECLARE : Optics0/5/0/6: Optics0/5/0/6
    RP/0/RP1/CPU0:Jun 1 10:05:46.096 UTC: optics_driver[231]: %PKT_INFRA-FM-2-FAULT_CRITICAL : ALARM_CRITICAL :OPTICS NOT SUPPORTED :DECLARE : Optics0/5/0/6: Optics0/5/0/6

    You must reload the line cards manually to clear these alarms

Usage Guidelines—Online Insertion of RPs

When fpd auto-upgrade is enabled and a new RP is inserted, the system upgrades the RP FPDs automatically with the latest FPDs.


Note


RPs are not reloaded automatically. You must manually reload the RP or chassis for the latest FPD version to reflect.



Note


Reload of active RPs and line cards impacts the network traffic.


Table 2. Action Required on FPDs After Auto Upgrade

FPD

Action Required

IOFPGA

Manual reload required

ADM

Upgraded version available immediately

PRIMARY-BIOS

Manual reload required

SATA

Upgraded version available immediately

PSOC

Upgraded version available immediately

IMFPGA

Manual reload required, if auto-reload is not configured

Configuring Auto FPD During System Upgrade

In case of Software upgrade (without ISSU), configure the fpd auto-upgrade enable command. All the FPDs are automatically upgraded in the currently installed image (V1). After the upgrade, the router automatically reloads and comes up with the new image (V2) with the upgraded FPDs already running. No additional reloads are required.


Note


System reloads are part of the SU process, therefore you can disable the FPD auto reload functionality by using the fpd auto-reload disable command.


  1. Enable FPD auto-upgrade
    RP/0/RP0/CPU0:IOS#conf    
    RP/0/RP0/CPU0:IOS(config)#fpd auto-upgrade enable
    RP/0/RP0/CPU0:IOS#commit
    
  2. Check for FPD Versions

  3. Check that Auto Upgrades are Triggered for FPDs with Newer Versions Available


    Note


    At this step, all RSP, IMs, and fan FPD upgrades are initiated and completed. All cards are upgraded before the router reloads.


  4. Check the RP FPD Versions and FPD Status

    When the router is operational after the reload, all the RP, IMs, and fan FPDs are upgraded to the latest FPD versions.

Automatic FPD Upgrade for PSU

During the installation and Power Supply Unit (PSU) insertion process, the Field-Programmable Devices (FPD) associated with the PSUs are automatically upgraded.


Note


The PSUs are upgraded sequentially, hence the PSU FPD upgrades take longer. You can choose to exclude PSUs from the auto upgrade flow. This restricts the PSUs from being upgraded either upon insertion, or during system upgrade.


To exclude the PSU FPDs from auto upgrading, use the following CLI:

fpd auto-upgrade exclude pm

RP/0/RSP0/CPU0:router# show running-config fpd auto-upgrade
Wed Mar 30 20:52:55.079 UTC
fpd auto-upgrade enable
fpd auto-upgrade exclude pm

Note


When you upgrade from an earlier unsupported version to a version that supports Automatic FPD upgrade for PSU, the PSU upgrade might happen on bootup.


Upgrade Failure

On failure of an FPD upgrade, you get a warning with the following syslog message:

LC/0/5/CPU0:Jun 27 05:02:25.742 UTC: optics_driver[216]: %INFRA-FPD_Driver-1-UPGRADE_ALERT : FPD MIFPGA@0/5 image programming completed with UPGD FAIL state Info: [Image verification failed at offset 0x5c8, flash value = 0x0, image value = 0x40, image size = 4194304] 
LC/0/5/CPU0:Jun 27 05:02:26.570 UTC: optics_driver[216]: %INFRA-FPD_Driver-1-UPGRADE_ALERT : FPD MIFPGA@0/5 image programming completed with UPGD FAIL state Info: [Image verification failed at offset 0x1e, flash value = 0x56, image value = 0xff, image size = 4194304]

When you use the show hw-module fpd command, the status column displays UPGD FAIL to indicate failure of the FPD upgrade.


Note


  • Do not reload the line card with a failed FPD upgrade image.

  • Upgrade failed FPDs will be fixed with a manual upgrade.

  • Contact Cisco TAC or your account representative if the FPD upgrade failure is not repaired.