Using Fabric Evacuation in UCS during Firmware Upgrades

Introduction

Cisco UCS Manager 2.2(4) introduces Fabric Evacuation, which is the ability to gracefully suspend traffic that flows through a Fabric Interconnect from all servers attached to it through an IOM or FEX while upgrading a system.

Prerequisites

• UCSM Version = 2.2(4) or higher

Components Used

• UCSM 3.1
• Fabric Interconnect 6248
• IOM/FEX 2204 / 2208  

Network Diagram

Upgrading the secondary Fabric Interconnect in a system disrupts the traffic that is active on the Fabric Interconnect. This traffic automatically fails over to the primary Fabric Interconnect.

Fabric Evaculation is used to confirm traffic will indeed fail over to the primary Fabric Interconnect prior to beginning an upgrade on the secondary Fabric Interconnect.

 

Configurations

You can follow the procedure as indicate to configure Fabric Failover:

1. To achieve this, Click on "Configure Evacuation" on the Fabric Interconnect.



2. Change the Admin Evac Mode to "On". 



 3. A confirmation message highlighting the result of enabling Fabric Evacuation appears.



4. Click OK to confirm evacuation of traffic from the fabric.  You will observe vNICs configured with Fabric Failover will repin traffic to the alternate fabric. Confirm in the host operating system that traffic is flowing on the alternate fabric.

5. At this time proceed with upgrading the secondary Fabric Interconnect.

6. When the upgrade is complete restart traffic flows on the evacuted fabric.

7. Turn Admin Evacuation Off



8. Acknowledge the Reset warning.



9. Verify the Oper Evac Mode changes from On to Off.



10. Status after Evacuation is complete.



11. Change the cluster lead to the secondary Fabric Interconnect

12. Repeat the above steps and upgrade the other Fabric Interconnect.

Verify

Check the traffic flowing through the Fabric Interconnect (FI) before Fabric Evacuation. You can check if from IOM stats

• Before the evacuation you would see the HIF(Host Interface) interfaces and the Transmit (Tx) and Receive (Rx) stats.



• After the evacuation you should not see HIF stats from the FI where you performed evacuation. Instead you should only see the NIF(Network Interface) Tx and Rx stats as seen below 



Troubleshoot

At the time of this writing, there is no alert thrown on the UCSM when this feature is turned on.

An easy way to tell if this feature is turned on is to have a look at the status of the IOM backplane ports, If all of them report "admin down" then you'd want to check if this feature is enabled.



For ease of troubeshooting defect CSCuy49191 is logged for an alert to be raised in UCSM when this feature is turned on. 

If you need to double check in the logs, look for an output that says:

Creation Time: 2016-10-05T22:10:05.121
User: admin
Session ID: web_11801_A
ID: 18706522
Action: Modification
Description: Switch attributes modified
Affected Object: sys/switch-B
Trigger: Admin
Modified Properties: adminEvacState(Old:fill, New:drain)

This will confirm that all the backplane ports (Host Interfaces aka HIF's) on the IOM are in an "Admin Down" state beacuse of fabric evacuation being turned on.

Note	Fabric evacuation is supported only with the following: Manual install  Cluster configuration