Field Notice: FN - 70457 - Firepower 1000 Series Security Appliances - Some Units Shipped with an Incomplete Installation of Firepower Software - Workaround Provided
Available Languages
Notice
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
07-Oct-19 |
Initial Release |
1.1 |
09-Oct-19 |
Changed "ASA" to "Firepower" in SNV Section |
1.2 |
30-Apr-21 |
Updated the Serial Number Validation Section and Added the How to Identify Affected Products Section |
Products Affected
| Affected Product ID | Comments |
|---|---|
FPR1010-NGFW-K9 |
|
FPR1120-NGFW-K9 |
|
FPR1140-NGFW-K9 |
Defect Information
| Defect ID | Headline |
|---|---|
| CSCvr55912 | Erase config command removes FTD installation on WM |
Problem Description
Some Firepower 1000 Series security appliances were shipped with an incomplete installation of Firepower software.
Background
Due to a manufacturing test script error, Firepower 1000 Series security appliances manufactured prior to 2019-07-22 were shipped with an incomplete installation of Firepower software. Affected units will automatically initiate a reinstallation of the Firepower software image during initial boot up that requires approximately 20 to 30 minutes to complete.
Problem Symptom
Affected units will boot up properly and automatically initiate a reinstallation of the Firepower software image during initial boot up. This process requires approximately 20 to 30 minutes to complete and is indicated by a blinking green SYS LED on the front panel. The SYS LED will become solid green when the software installation and boot up procedure are complete.
Indication of the software installation during boot up is provided over the console port. An example of the console port output is shown here:
Threat Defense System: CMD=-install, CSP-ID=cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUJ445TQ1, FLAG='' System begins installation ... .... Starting nscd... mkdir: created directory '/var/run/nscd' [ OK ] Starting , please wait......complete. cleaning up *.TMM and *.TMD files Firstboot detected, executing scripts Executing S01virtual-machine-reconfigure [ OK ] Executing S01z_copy_startup-config [ OK ] Executing S02aws-pull-cfg [ OK ] Executing S02configure_onbox [ OK ] Executing S04fix-httpd.sh [ OK ] Executing S05set-default-ipv4.pl [ OK ] Executing S06addusers [ OK ] Executing S07uuid-init [ OK ] Executing S08configure_mysql [ OK ] ************ Attention ********* Initializing the configuration database. Depending on available system resources (CPU, memory, and disk), this may take 30 minutes or more to complete. ************ Attention ********* Executing S09database-init [ OK ] Executing S11database-populate [ OK ] ...
The unit might fail to boot up properly if the software installation is interrupted by power cycling the unit while the SYS LED is blinking. Indication of a failed software installation is provided over the console port. An example of the console port output is shown here:
Executing S09database-init stat: cannot stat '/var/log/firstboot.S09database-init': No such file or directory backing up existing firstboot.S09database-init '/ngfw/var/log/firstboot.S09database-init' -> '/ngfw/var/log/firstboot[FAILED]base-init.' Executing S11database-populate [FAILED] Executing S12install_infodb DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603.
Workaround/Solution
No additional action is required if the Firepower 1000 Series unit completes the automatic reinstallation of the Firepower software image during initial boot up.
Additional action is required if the unit indicates a failed software installation. Follow this procedure in order to initiate a reinstallation of the software image. The Cisco FXOS Troubleshooting Guide can also be referenced for additional information.
Log into FXOS and complete these steps in order to initiate a software reinstallation:
- Log in with these credentials: Admin/Admin123.
- Enter connect local-mgmt.
- Enter erase configuration.
- Enter yes to confirm erasing the configuration and rebooting the device.
An example of the console port output is shown here:
Cisco FPR Series Security Appliance firepower login: admin Password: Last login: Fri Sep 20 14:38:56 UTC 2019 on ttyS0 Successful login attempts for user 'admin' : 1 ... firepower# connect local-mgmt firepower(local-mgmt)# firepower(local-mgmt)# erase configuration All configurations will be erased and system will reboot. Are you sure? (yes/no):yes Removing all the configuration. Please wait.... Configurations are cleaned up. Rebooting.... deleting files under /opt/cisco/config, /opt/cisco/csp/, /opt/cisco/platform/logs, /var/data/core /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/root1/ngfw/Volume': Device or resource busy /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/root1/ngfw/usr/local/sf': Device or resource busy /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/root1/ngfw/var': Device or resource busy /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/Volume/6.4.0/lib/db/ngfw.db': Directory not empty /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/Volume/6.4.0/lib/mysql': Device or resource busy /bin/rm: cannot remove '/opt/cisco/csp/applications/cisco-ftd.6.4.0.102__ftd_001_JMX2326G2AUUEEPIB1/app_data/Volume/6.4.0/perl5': Device or resource busy cannot remove '/opt/cisco/csp/applica2019-09-23 18:15:42 logmonitor[18270]: syslog-ng not running. starting it. Stopping all devices.
Note that error messages might appear on the screen, but you can still continue to enter commands in order to complete the software installation procedure as shown in this example:
Cisco FPR Series Security Appliance firepower login: Cisco FPR Series Security Appliance DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. firepower login: adminDB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. Password: DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. Last login: Mon Sep 23 18:47:47 UTC 2019 on ttyS0 Successful login attempts for user 'admin' : 2 DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. Cisco Firepower Extensible Operating System (FX-OS) Software TAC support: http://www.cisco.com/tac Copyright (c) 2009-2019, Cisco Systems, Inc. All rights reserved. .... firepower# firepower# DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. firepower# firepower# DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. firepower# firepower# connect localDB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. firepower(local-mgmt)# DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. firepower(local-mgmt)# erase configuration All configurations will be erased and system will reboot. Are you sure? (yes/no):yes DB error - will retry: Cannot connect to DB at /usr/local/sf/lib/perl/5.10.1/SF/SFDBI.pm line 603. Removing all the configuration. Please wait.... Configurations are cleaned up. Rebooting....
How To Identify Affected Products
CLI
Enter the show inventory command in order to obtain the chassis serial number of the appliance.
Visual Inspection of the Firepower Security Appliance
The serial number label is located on the bottom surface of the Firepower 1000 Series security appliance. The serial number can also be referenced on the Sales Order documentation.
Serial Number Validation
This field notice provides the ability to determine if the serial number(s) of a device is impacted by this issue. In order to verify your serial number(s), enter it in the Serial Number Validation tool at https://snvui.cisco.com/snv/FN70457.
For More Information
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Receive Email Notification For New Field Notices
My Notifications—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.
Feedback
Unleash the Power of TAC's Virtual Assistance