Guidelines, and Best Practices for Firmware Upgrades
Before you upgrade the firmware for any endpoint in a Cisco UCS domain, consider the following guidelines, best practices, and limitations:
Configuration Changes and Settings that Can Impact Upgrades
Depending on the configuration of your Cisco UCS domain, the upgrade process may require you to make additional changes.
Default Maintenance Policy Should be Configured for User Acknowledgment
The default maintenance policy is configured to immediately reboot the server when disruptive changes are made to the service profile, such as server firmware upgrades through a host maintenance policy. We recommend that you change the reboot policy setting in the default maintenance policy to user acknowledgment to avoid unexpected disruption of server traffic.
When you configure the reboot policy in the default maintenance policy to user acknowledgment, the list of disruptive changes are listed with the pending activities. You can then control when the servers are rebooted.
Overlapping FCoE VLAN IDs and Ethernet VLAN IDs Are No Longer Allowed with Cisco UCS Release 2.0 and Higher
![]() Caution |
In Cisco UCS 1.4 and earlier releases, Ethernet VLANs and FCoE VLANs could have overlapping VLAN IDs. However, starting with Cisco UCS release 2.0, overlapping VLAN IDs are not allowed. If Cisco UCS Manager detects overlapping VLAN IDs during an upgrade, it raises a critical fault. If you do not reconfigure your VLAN IDs, Cisco UCS Manager raises a critical fault and drops Ethernet traffic from the overlapped VLANs. Therefore, we recommend that you ensure there are no overlapping Ethernet and FCoE VLAN IDs before you upgrade to Cisco UCS Release 3.1 and later releases. Be aware that when an uplink trunk is configured with VLAN ID 1 defined and set as the native VLAN, changing the Ethernet VLAN 1 ID to another value can cause network disruption and flapping on the fabric interconnects, resulting in an HA event that introduces a large amount of traffic and makes services temporarily unavailable. |
For a new installation of Cisco UCS Release 3.1 and later releases, the default VLAN IDs are as follows:
-
The default Ethernet VLAN ID is 1.
-
The default FCoE VLAN ID is 4048.
![]() Note |
If a Cisco UCS domain uses one of the default VLAN IDs, which results in overlapping VLANs, you can change one or more of the default VLAN IDs to any VLAN ID that is not used or reserved. From release 2.0 and higher, VLANs with IDs from 4043 to 4047 are reserved. |
VSANs with IDs in the Reserved Range are not Operational
A VSAN with an ID in the reserved range is not operational after an upgrade. Make sure that none of the VSANs configured in Cisco UCS Manager are in these reserved ranges:
-
If you plan to use FC switch mode in a Cisco UCS domain, do not configure VSANs with an ID in the range from 3040 to 4078.
-
If you plan to use FC end-host mode in a Cisco UCS domain, do not configure VSANs with an ID in the range from 3840 to 4079.
If a VSAN has an ID in the reserved range, change that VSAN ID to any VSAN ID that is not used or reserved.
Hardware-Related Guidelines for Firmware Upgrades
The hardware in a Cisco UCS domain can impact how you upgrade. Before you upgrade any endpoint, consider the following guidelines and limitations:
No Server or Chassis Maintenance
![]() Caution |
Do not remove the hardware that contains the endpoint or perform any maintenance on it until the update process completes. If the hardware is removed or otherwise unavailable due to maintenance, the firmware update fails. This failure might corrupt the backup partition. You cannot update the firmware on an endpoint with a corrupted backup partition. |
Avoid Replacing RAID-Configured Hard Disks During or Prior to Upgrade
During or prior to Cisco UCS infrastructure and server firmware upgrades:
-
Do not remove, insert or replace any local storage hard disks or SSDs in the servers.
-
Ensure that no storage operations are running, including Rebuild, Association, Copyback, BGI, and so on.
Always Upgrade Third-Party Adapters through a Host Firmware Package
You cannot upgrade third-party adapters directly at the endpoints. You must upgrade the firmware on those adapters through a host firmware package.
Configure the Fabric Interconnects
The clustered fabric interconnects provide data path redundancy by design. However, to ensure that data traffic is not disrupted, you must configure redundant Ethernet and storage (FC/FCoE) interfaces within the service profile. You must also ensure that the corresponding Operating System is configured correctly to handle one fabric path outage.
For a standalone configuration with a single fabric interconnect, you can minimize the disruption to data traffic when you perform a direct firmware upgrade of the endpoints. However, you must reboot the fabric interconnect to complete the upgrade and, therefore, cannot avoid disrupting traffic.
Firmware- and Software-Related Guidelines for Upgrades
Before you upgrade any endpoint, consider the following guidelines and limitations:
Determine the Appropriate Type of Firmware Upgrade for Each Endpoint
Some endpoints, such as Cisco adapters and the server CIMC, can be upgraded through either a direct firmware upgrade or a firmware package included in a service profile. The configuration of a Cisco UCS domain determines how you upgrade these endpoints. If the service profiles associated with the servers include a host firmware package, upgrade the adapters for those servers through the firmware package.
Upgrades of an adapter through a firmware package in the service profile associated with the server take precedence over direct firmware upgrades. You cannot directly upgrade an endpoint if the service profile associated with the server includes a firmware package. To perform a direct upgrade, you must remove the firmware package from the service profile.
Do Not Activate All Endpoints Simultaneously in Cisco UCS Manager GUI
If you use Cisco UCS Manager GUI to update the firmware, do not select ALL from the Filter drop-down list in the Activate Firmware dialog box to activate all endpoints simultaneously. Many firmware releases and patches have dependencies that require the endpoints to be activated in a specific order for the firmware update to succeed. This order can change depending upon the contents of the release or patch. Activating all endpoints does not guarantee that the updates occur in the required order, and can disrupt communications between the endpoints and the fabric interconnects and Cisco UCS Manager. For information about the dependencies in a specific release or patch, see the release notes provided with that release or patch.
Determine Available Bootflash and Workspace Partition
The bootflash partition is dedicated solely to firmware images managed by Cisco UCS Manager. To initiate upgrade or downgrade, at least 20 percent of the bootflash partition must be available. When the bootflash partition exceeds 70 percent, faults are raised, but Auto Install proceeds. When the bootflash partition exceeds 80 percent, faults are raised and Auto Install does not proceed.
The workspace partition on the fabric interconnect stores tech support files, core files, and the debug plugin. To initiate upgrade or downgrade, at least 20 percent of the workspace partition must be available.
Determine the Impact of Activation for Adapters and I/O Modules
During a direct upgrade, you should configure Set Startup Version Only for an adapter. With this setting, the activated firmware moves into the pending-next-boot state, and the server is not immediately rebooted. The activated firmware does not become the running version of firmware on the adapter until the server is rebooted. You cannot configure Set Startup Version Only for an adapter in the host firmware package.
If a server is not associated with a service profile, the activated firmware remains in the pending-next-boot state. Cisco UCS Manager does not reboot the endpoints or activate the firmware until the server is associated with a service profile. If necessary, you can manually reboot or reset an unassociated server to activate the firmware.
When you configure Set Startup Version Only for an I/O module, the I/O module is rebooted when the fabric interconnect in its data patch is rebooted. If you do not configure Set Startup Version Only for an I/O module, the I/O module reboots and disrupts traffic. In addition, if Cisco UCS Manager detects a protocol and firmware version mismatch between the fabric interconnect and the I/O module, Cisco UCS Manager automatically updates the I/O module with the firmware version that matches the firmware in the fabric interconnect and then activates the firmware and reboots the I/O module again.Disable Call Home before Upgrading to Avoid Unnecessary Alerts (Optional)
When you upgrade a Cisco UCS domain, Cisco UCS Manager restarts the components to complete the upgrade process. This restart causes events that are identical to the service disruptions and component failures that trigger Call Home alerts to be sent. If you do not disable Call Home before you begin the upgrade, alerts will be generated by the upgrade-related component, restarts and notifications will be sent out based on your Call Home configuration.
Fabric Interconnect Traffic Evacuation
Fabric interconnect traffic evacuation, introduced in Release 2.2(4), is the ability to evacuate all traffic that flows through a fabric interconnect from all servers attached to it through an IOM or FEX, while upgrading a system.
Upgrading the subordinate fabric interconnect in a system disrupts the traffic that is active on the fabric interconnect. This traffic fails over to the primary fabric interconnect.
![]() Important |
|
You can perform fabric evacuation as follows during the manual upgrade process:
-
Stop all the traffic that is active through a fabric interconnect by configuring Admin Evac Mode as On.
-
For vNICs configured with failover, verify that the traffic has failed over by using Cisco UCS Manager or tools such as vCenter.
-
Upgrade the subordinate fabric interconnect.
-
Restart all the stopped traffic flows by configuring Admin Evac Mode as Off.
-
Change the cluster lead to the subordinate fabric interconnect.
-
Repeat steps 1 to 4 and upgrade the other fabric interconnect.
Fabric Evacuation with Auto Install
Starting with Cisco UCS Manager Release 3.1(3), you can use fabric evacuation during Auto Install. While initiating Auto Install, when you enable fabric evacuation and then begin Auto Install, the following sequence of events occur:
-
The subordinate fabric interconnect (FI-B) is evacuated and activated.
-
Failover occurs and the primary fabric interconnect (FI-A) becomes the subordinate fabric interconnect. FI-B now becomes the cluster lead.
-
FI-A is now evacuated and activated.
If you use fabric evacuation with Auto Install, and fabric evacuation was enabled on the fabric interconnect before Auto Install, fabric evacuation is disabled after Auto Install is complete.
Ensure that you do not initiate Auto Install with fabric evacuation enabled on the primary fabric interconnect. If fabric evacuation was manually enabled on the primary fabric interconnect before Auto Install, it must be manually disabled before initiating Auto Install.
![]() Note |
|
Configuring Fabric Interconnect Traffic Evacuation
You can use the steps detailed here, or click Play on this video (http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-manager/videos/3-1/enable_and_disable_fi_traffic_evacuation.html) to watch how to enable and disable fabric interconnect traffic evacuation.
Procedure
Step 1 |
In the Navigation pane, click Equipment. |
Step 2 |
Expand . |
Step 3 |
In the Work pane, click the General tab. |
Step 4 |
In the Actions area of the General tab, click Configure Evacuation. |
Step 5 |
To configure evacuation of the traffic through the specified fabric interconnect, click one of the following radio buttons in the Admin Evac Mode field:
|
Step 6 |
(Optional) To evacuate the traffic through a fabric interconnect irrespective of its current evacuation state, check the Force check box. |
Step 7 |
Click Apply.
|
Step 8 |
Click OK to confirm fabric interconnect traffic evacuation and continue. |
Secure Firmware Update
Cisco UCS Manager, Release 3.1(2) introduces secure firmware update, which enables you to update the adapter firmware securely for third-party Intel network and storage adapters. Only server administrators can upgrade or downgrade firmware for the adapters. OS administrators with root privileges are not allowed to downgrade the adapter firmware.
The following Cisco UCS servers support secure firmware update:
-
Cisco UCS C460 M4 Server
-
Cisco UCS C240 M4 Server and Cisco UCS C240 M5 Server
-
Cisco UCS C220 M4 Server and Cisco UCS C220 M5 Server
-
Cisco UCS B200 M4 Server and Cisco UCS B200 M5 Server
-
Cisco UCS B480 M5 Server and Cisco UCS C480 M5 Server
Secure Firmware Update Supported Network Adapters and Storage Disks
Supported Storage Disks on Cisco Blade Servers
The following Intel NVMe storage disks support secure firmware update on Cisco UCS B200 M5 Server and Cisco UCS B480 M5 Server.
NVMe Storage Disks |
---|
UCSC-NVMEHW-H800 |
UCSC-NVMEHW-H1600 |
UCSC-NVMEHW-H3200 |
UCSC-NVMEHW-H6400 |
UCSC-NVMEHW-H7680 |
The following Intel NVMe storage disks support secure firmware update on a Cisco UCS B200 M4 server that has the UCSB-LSTOR-PT storage controller.
Storage Disks |
---|
UCS-PCI25-8003 |
UCS-PCI25-16003 |
UCS-PCI25-40010 |
UCS-PCI25-80010 |
![]() Note |
Secure firmware update is not supported on a Cisco UCS B200 M4 server for the following:
|
Supported Network Adapters and Storage Disks on Cisco Rack Servers
The following NVMe storage disks support secure firmware update on Cisco UCS C220 M5 Server, Cisco UCS C240 M5 Server, and Cisco UCS C480 M5 Server servers:
NVMe Storage Disks |
---|
UCSC-NVMEHW-H800 |
UCSC-NVMEHW-H1600 |
UCSC-NVMEHW-H3200 |
UCSC-NVMEHW-H6400 |
UCSC-NVMEHW-H7680 |
UCSC-NVME-H16003 to UCSC-F-H16003 |
UCSC-NVME-H32003 |
UCSC-NVME-H38401 |
UCSC-NVME-H64003 |
UCSC-NVME-H76801 |
The following Intel network adapters support secure firmware update on Cisco UCS C460, C240, and C220 M4 servers:
Network Adapters |
---|
UCSC-PCIE-IQ10GF |
UCSC-PCIE-ID10GF |
UCSC-PCIE-ID40GF |
NVMe Storage Disks |
Description |
---|---|
UCS-PCI25-8003 |
P3600 2.5" |
UCS-PCI25-16003 |
P3600 2.5" |
UCS-PCI25-40010 |
P3700 2.5" |
UCS-PCI25-80010 |
P3700 2.5" |
UCSC-F-I80010 |
P3700 HHHL |
UCSC-F-I160010 |
P3700 HHHL |
UCSC-F-I20003 |
P3600 HHHL |
Guidelines for Secure Firmware Support on Cisco UCS Servers
Cisco UCS Manager Release 3.1(2) introduces support for secure firmware update. For Cisco UCS M5 servers, secure firmware update is introduced in Cisco UCS Manager Release 3.2(2).
![]() Important |
Ensure that CIMC is running Version 2.0(13) or later and Cisco UCS Manager is running Release 3.1(2) or later releases. Secure firmware update cannot be done when the CIMC is running a version earlier than 2.0(13) and Cisco UCS Manager is running a release earlier than Release 3.1(2). |
Guidelines for Blade Servers
For secure firmware update on Cisco UCS B200 M4 , B200 M5, and B480 M5 servers, do the following:
-
For Cisco UCS B200 M4 servers, upgrade the Cisco UCS Manager infrastructure software bundle and B-Series server software bundle to Cisco UCS Manager Release 3.1(2) or a later release. For Cisco UCS M5 servers, upgrade to Cisco UCS Manager Release 3.2(2) or a later release.
-
Install the UCSB-LSTOR-PT storage controller and insert the NVMe disks on a Cisco UCS B200 M4, B200 M5, or B480 M5 server.
-
Reacknowledge the server. Refer to the Reacknowledging a Blade Server section in the Cisco UCS Manager Infrastructure Management Guide, Release 3.2.
Note
Ensure that server discovery does not fail and the NVMe disks are identified by CIMC and BIOS. After the server is associated to the service profile with the default host firmware package, Auto Install is triggered. NVMe disks can be updated with the latest firmware during Auto Install.
Cisco UCS Manager, Release 3.2(1) supports NVMe boot.
Guidelines for Rack Servers
For secure firmware update on Cisco UCS C460, C240, C220 M4 and M5 servers and C480 M5 servers, do the following:
-
For the supported Cisco UCS M4 servers, upgrade the Cisco UCS Manager infrastructure software bundle and C-Series server software bundle to Cisco UCS Manager Release 3.1(2) or a later release. For Cisco UCS M5 servers, upgrade to Cisco UCS Manager Release 3.2(2) or a later release.
-
Reacknowledge the Cisco UCS servers. Refer to the Reacknowledging a Rack Server section in the Cisco UCS Manager Infrastructure Management Guide, Release 3.2.
Note
Ensure that server discovery does not fail and the NVMe disks are identified by CIMC and BIOS. After the server is associated to the service profile with the default host firmware package, Auto Install is triggered. NVMe disks can be updated with the latest firmware during Auto Install.
Cisco UCS Manager, Release 3.2(1) supports NVMe boot.
Cautions, and Guidelines for Upgrading with Auto Install
Before you use Auto Install to upgrade the firmware for any endpoint in a Cisco UCS domain, consider the following cautions, guidelines, and limitations:
![]() Note |
These guidelines are specific to Auto Install and are in addition to those listed in Guidelines, and Best Practices for Firmware Upgrades. |
State of the Endpoints
Before you begin an upgrade, all affected endpoints must be as follows:
-
For a cluster configuration, verify that the high availability status of the fabric interconnects shows that both are up and running.
-
For a standalone configuration, verify that the Overall Status of the fabric interconnect is Operable.
-
For all endpoints to be upgraded, verify that they are in an Operable state.
-
For all servers to be upgraded, verify that all the servers have been discovered and that discovery did not fail. Install Server Firmware will fail if any server endpoints cannot be upgraded.
-
For each server to be upgraded, check the running firmware version on the storage controller and local disks, and verify that they are in the Ready state.
Recommendations for the Default Host Firmware Policy
After you upgrade Cisco UCS Manager, a new host firmware policy named "default" is created, and is assigned to all service profiles that did not already include a host firmware policy. The default host firmware policy is blank. It does not contain any firmware entries for any components. This default policy is also configured for an immediate reboot rather than waiting for user acknowledgment before rebooting the servers.
During the upgrade of server firmware, you can modify the default host firmware policy to add firmware for the blade and rack-mount servers in the Cisco UCS domain. To complete the upgrade, all servers must be rebooted.
Every service profile that is assigned to the default host firmware policy reboots the associated server according to the maintenance policy included in the service profile. If the maintenance policy is set to immediate reboot, you cannot cancel the upgrade or prevent the servers from rebooting after you complete the configuration in the Install Server Firmware wizard. We recommend that you verify the maintenance policy associated with these service profiles to ensure that they are set for a timed reboot or for user acknowledgment.
![]() Note |
If you are upgrading from a release prior to 2.1(2a), you may be impacted by CSCup57496. After manually upgrading the CIMC and associating a service profile, remove the Management Firmware pack to activate the firmware of CIMC. For more information, please refer to https://tools.cisco.com/bugsearch/bug/CSCup57496. This is not applicable to Cisco UCS Mini. |
Time, Date, and Time Zone on Fabric Interconnects Must Be Identical
To ensure that the fabric interconnects in a cluster configuration are in sync, you must ensure that they are configured for the same date, time, and time zone. We recommend that you configure an NTP server and the correct time zone in both fabric interconnects. If the date, time or time zone in the fabric interconnects are out of sync, the Auto Install might fail.
Cannot Upgrade Infrastructure and Server Firmware Simultaneously
You cannot upgrade the infrastructure firmware at the same time as you upgrade server firmware. We recommend that you upgrade the infrastructure firmware first and then upgrade the server firmware. Do not begin the server firmware upgrade until the infrastructure firmware upgrade is completed.
Required Privileges
Users must have the following privileges to upgrade endpoints with Auto Install:
Privileges | Upgrade Tasks User Can Perform |
---|---|
admin |
|
Service profile compute (ls-compute) |
Run Install Server Firmware |
Service profile server policy (ls-server-policy) |
Add, delete, and modify host firmware packages |
Service profile config policy (ls-config-policy) |
Add, delete, and modify host firmware packages |
Impact of Host Firmware Packages on Install Server Firmware
Because Install Server Firmware uses host firmware packages to upgrade the servers, you do not have to upgrade all servers in a Cisco UCS domain to the same firmware versions. However, all servers which have associated service profiles that include the host firmware packages you selected when you configured Install Server Firmware are upgraded to the firmware versions in the specified software bundles.
Effect of Using Install Server Firmware on Servers Whose Service Profiles Do Not Include a Host Firmware Package
If you use Install Server Firmware to upgrade server endpoints on servers that have associated service profiles without host firmware packages, Install Server Firmware uses the default host firmware package to upgrade the servers. You can only update the default host firmware package through Install Server Firmware.
If you want to upgrade the CIMC or adapters in a server with an associated service profile that has previously been updated through the default host firmware package in Install Server Firmware, you must use one of the following methods:
-
Use Install Server Firmware to modify the default host firmware package and then upgrade the server through Install Server Firmware.
-
Create a new host firmware package policy, assign it to the service profile associated with the server, and then upgrade the server through that host firmware package policy.
-
Disassociate the service profile from the server and then directly upgrade the server endpoints.
Upgrading Server Firmware on Newly Added Servers
If you add a server to a Cisco UCS domain after you run Install Server Firmware, the firmware on the new server is not automatically upgraded by Install Server Firmware. If you want to upgrade the firmware on a newly added server to the firmware version used when you last ran Install Server Firmware, you must manually upgrade the endpoints to upgrade the firmware on that server. Install Server Firmware requires a change in firmware version each time. You cannot rerun Install Server Firmware to upgrade servers to the same firmware version.
![]() Note |
After you finish the upgrade, you can use the Firmware Auto Sync Server policy in Cisco UCS Manager to automatically update newly discovered servers. |