Field Notice: FN72077 - FPR9300 and FPR4100 Series Security Appliances - Some Appliances Might Fail to Pass Traffic After 3.2 Years of Uptime - Power Cycle Required - Software Upgrade Recommended

Due to a flaw in solid-state drive (SSD) firmware, the SSD that is internal to the FPR9300 Supervisor module and FPR4100 Series security appliances will no longer respond after approximately 3.2 years of cumulative operation. After the first unresponsive event occurs, every subsequent power-cycle allows the SSD to operate for approximately six weeks of cumulative operation before the SSD will no longer respond again. 

After 28,224 hours (approximately 3.2 years) of accumulated Power On Hours (POH), a memory buffer overrun condition occurs which triggers the firmware event in the SSD. This event causes the drive to become unresponsive until it is power-cycled. No data loss will occur when the memory buffer overrun firmware event occurs. A power-cycle of the FPR9300 Supervisor module and FPR4100 Series security appliances restores normal operation of the drive. The drive continues to operate normally for 1008 additional accumulated POH (six weeks), at which time the drive becomes unresponsive again. Power-cycling the FPR9300 Supervisor module and FPR4100 Series security appliance again will re-initiate the 1008-hour window.

Note: This issue affects an internal SSD component that is not field-replaceable and does not appear in show inventory commands. The field-replaceable SSDs are not affected by this issue.

The FPR9300 and FPR4100 Series security appliances no longer pass network traffic. Users with valid credentials might not be able to log in to the management console.

Workaround

A power-cycle of the FPR9300 Supervisor module or FPR4100 Series security appliance is required in order to temporarily recover from this issue. However, this failure will reappear after 1008 hours of operation.

Note: Proactive reloads before the 28,224 hour or 1008 hour marks will not reset the timer that triggers this issue. The issue is related to cumulative, not consecutive, hours of operation (total power on time) for affected SSDs.

Solution

In order to prevent occurrence of this issue and disruption to the network and operations, Cisco recommends to proactively upgrade the SSD firmware before the accumulated uptime reaches 28,224 hours.

Refer to the Serial Number Validation section to determine if your security appliance might be affected. Use the FPR9300 supervisor module serial number or the FPR4100 series chassis serial number for validation.

If the system is already impacted, the SSD firmware upgrade will permanently resolve this defect.

A product return and replacement (RMA) is not recommended because the firmware upgrade process will resolve the issue.

A service contract is not required to download the referenced software images.

Note: Both Step 1 and Step 2 must be performed in this sequence to complete the SSD firmware update.

Step 1: Upgrade the FXOS chassis software to one of the following versions. This software is available from the Cisco Software Download Center:

• FXOS 2.2.2.148 or later
• FXOS 2.3.1.215 or later
• FXOS 2.4.1.273 or later
• FXOS 2.6.1.229 or later
• FXOS 2.7.1.143 or later
• FXOS 2.8.1.152 or later
• FXOS 2.9.1.143 or later
• FXOS 2.10 or later

See the Cisco Firepower 4100/9300 Upgrade Guide for instructions on how to upgrade the FXOS software.

Step 2: After upgrading the FXOS software, apply the Firepower 4100/9300 Firmware Upgrade Package version 1.0.19 or later to update the SSD firmware revision.

After Firmware Upgrade Package 1.0.19 or later has completed installation, you can enter the following commands to view the SSD firmware revision.

firepower-chassis /firmware/firmware-install # top

firepower-chassis# scope chassis 1

firepower-chassis /chassis # show sup version

SUP FIRMWARE:

    ROMMON:

        Running-Vers: 1.0.15

        Package-Vers: 1.0.19

        Activate-Status: Ready

    FPGA:

        Running-Vers: 2.00

        Package-Vers: 1.0.19

        Activate-Status: Ready

    SSD:

        Running-Vers: MU03

        Model: Micron_M500IT_MTFDDAT128MBD

 

If the SSD Model is Micron_M500IT_* and the Running-Vers is MU03 or later, then the SSD firmware update was successful. Other SSD Models are not affected by the issue.

Note: Reimaging the security appliance will not downgrade the SSD firmware revision after it has been updated.

FPR9300 and FPR4100 - Obtain the Chassis Serial Number for Validation

In order to determine whether your product might be affected by this issue, validate the chassis serial number of the security appliance.

Note: The Serial Number Validation Tool is not a complete asset management tracking tool and may not show all affected units due to the software dependency for this issue. As described in the "Solution" section of this field notice, Cisco recommends using the show sup version command to verify the SSD firmware version on the chassis.

For units that have already failed due to this issue, a visual inspection of the security appliance or review of the Sales Order documentation is required.

The chassis serial number can be obtained from the CLI or through visual inspection of the security appliance.

CLI

firepower# scope chassis
firepower /chassis # show inventory

Chassis    PID             Vendor            Serial (SN) HW Revision
---------- --------------- ----------------- ----------- -----------

         1 FPR-4110-K9     Cisco Systems Inc JMX1234ABCD 0

FPR9300 - Obtain the Supervisor Module Serial Number for Validation

For customers with FPR9300 platforms, the Supervisor module serial number must also be validated.

The Supervisor module serial number can be obtained from the CLI.

CLI

firepower# scope chassis
firepower /chassis # show inventory expand
Chassis 1:
    Servers:
        Server 1/1:
            Equipped Product Name: Cisco Firepower 9000 Series Security Module
 
*** Output continues ***
       
Fabric Card 1:
    Description: Firepower 9300 Supervisor
    Number of Ports: 8
    State: Online
    Vendor: Cisco Systems, Inc.
    Model: FPR9K-SUP
    HW Revision: 0
    Serial (SN): JAD1234ABCD
    Perf: N/A
    Power State: Online
    Presence: Equipped
    Thermal Status: N/A
    Voltage Status: N/A
 
*** Output continues ***
 
firepower /chassis #

For additional information, refer to the Cisco Firepower 4100/9300 FXOS Command Reference.

Visual Inspection

The chassis serial number for the Firepower 4100 Series appliance is located on the bottom surface of the chassis.

The chassis serial number for the Firepower 9300 appliance is located on the pullout asset card on the front panel, on the side of the chassis and on the top of the Supervisor module.

Refer to the Serial Number Validation section in order to verify your FPR9300 and FPR4100 Series serial number(s).