Upgrading Cisco UCS Hardware

This chapter includes the following sections:

Upgrading Fabric Interconnects

Fabric Interconnect Upgrade Considerations

Be sure the following prerequisites are met before beginning any procedures in this section:


Caution

All software and firmware on all components must be upgraded to the latest software release and build available before attempting an upgrade. UCS software version 2.0 is the bare minimum version for the Cisco UCS 6248 UP. The new Cisco UCS 6200 series fabric interconnects must be loaded with the same build version that is on the Cisco UCS 6100 series fabric interconnect it will replace.

Caution

If you intend to implement the fabric port channel feature available in UCS software version 2.0 after performing the hardware upgrade, you may need to rearrange the cabling between I/O module and Fabric Interconnect before performing the feature configuration. Fabric port channel will require that all physical links from a given I/O module physically connect to a contiguous block of ports (1-8 or 9-16, and so on). See http://www.cisco.com/en/US/products/ps10281/products_installation_and_configuration_guides_list.html. This feature requires a UCS 2200 Series I/O Module connecting to a UCS 6200 Series Fabric Interconnect.

  • Upgrading the fabric interconnect should be done before upgrading to a new IO Module or virtual interface card.
  • Do not attempt to implement new software features from the new UCS software version until all required hardware is installed.
  • Changes to the topology like number of server or uplink connections should be performed after the fabric interconnect upgrade is complete.
  • Make a detailed record of the cabling between IO Modules and fabric interconnects using the provided table. You will need to preserve the physical port mapping to maintain the server pinning already configured and minimize down time. See Table 1.
  • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and IO Modules.
  • Standalone installations should expect down time. Upgrading a fabric interconnect is inherently traffic disruptive.
  • A best practice would be to perform a full configuration and software backup before performing this hardware upgrade.

Port Mapping for Upgrades

The upgrade described here is primarily for upgrading a Cisco UCS 6120 fabric interconnect to a Cisco UCS 6248. If you have a Cisco UCS 6140 fabric interconnect that has 48 or less physical ports connected, you will also be able to utilize these steps, the same principles will apply though you will have to manually map any ports connected to slot 3 on a UCS 6140 as the UCS 6248 has no slot 3. The same considerations will also apply when upgrading a Cisco UCS 6140 fabric interconnect to a Cisco UCS 6296, though as the 6296 has enough slots that no manual re-mapping will be necessary.

Fixed Ports

On the UCS 6120 fabric interconnect, the fixed ports on slot 1 are all ethernet ports, which might be further configured as Server ports or uplink Ethernet ports. On the UCS 6248 fabric interconnect, you can separate the 32 physical ports in slot one into two contiguous pools, low numbered ports being Ethernet ports and high numbered ports being Fibre Channel ports. When upgrading to a UCS 6248, the UCS 6120 port mappings can be directly carried over on fixed ports with no reconfiguration required. When upgrading a UCS 6140 to a UCS 6296. the port mappings can be directly carried over on fixed ports with no reconfiguration required.


Note

Since a UCS 6140 has 40 ports on slot 1 and a UCS 6248 has 32 ports, any ports currently configured on ports 1/33 to 1/40 of the UCS 6140 will have to be moved during the upgrade process.


Caution

If you ever need to change the pool sizes for slot 1 this will require a reboot of the fabric interconnect which can lead to a service disruption. If you ever need to change the pool sizes for slot 2 this will require a reset of the expansion module in slot 2. To minimize disruption, plan to have at least a few Ethernet uplink and Fibre Channel uplink ports configured on Slot 1. Implement this fail safe after the upgrade is complete and the system re-stabilizes.
Figure 1. Cisco UCS 6248 Port Numbering



1

Slot 1, 32 fixed universal ports

2

Slot 1, 16 expansion universal ports

Expansion Ports

On the UCS 6120 fabric interconnect, the expansion slot port types depend on which of four possible expansion modules are installed. On the UCS 6248 fabric interconnect, you can separate the 16 physical ports in an expansion slot into two pools, low numbered ports being Ethernet ports and high numbered ports being Fibre Channel, see the Cisco UCS Manager configuration guides. When upgrading to a UCS 6248 or UCS 6296, you may need to pre- plan and re-configure expansion module port mappings before powering up the new interconnect.


Note

Any expansion ports in slot 3 of a UCS 6140 fabric interconnect will need to have their configuration deleted and their use re-mapped to ports on slots 1 or 2 of the new UCS 6248. Upgrading a UCS 6140 to a UCS 6296 is the preferred path as this manual re-mapping will note be necessary.

  • If you have an N10-E0080 or N10-E0060 Fibre channel expansion module in your UCS 6120, the existing fibre channel configurations can simply carry over to the new UCS 6248, though this may not be appropriate for all situations as it will mean that all ports in slot two are fibre channel. If you want to reconfigure ports 2/1 to 2/8 to use ports 2/9 to 2/16, do so after the upgrade is completed.
    Figure 2. Cisco UCS 6120 Port Numbering when Configured with the N10-E0080 Expansion Module

    A

    Slot 1, ports 1 through 8: 10-Gigabit or 1 Gigabit Ethernet capable ports

    C

    Slot 2 Fibre Channel ports 1 through 8

    B

    Slot 1, ports 1 through 20: 10-Gigabit Ethernet ports

    		   
    Figure 3. Cisco UCS 6120 Port Numbering when Configured with the N10-E0060 Expansion Module

    A

    Slot 1, ports 1 through 8: 10-Gigabit or 1 Gigabit Ethernet capable ports

    C

    Slot 2 Fibre Channel ports 1 through 6

    B

    Slot 1, ports 1 through 20: 10-Gigabit Ethernet ports

    		   
  • If you have an N10-E0600 Fibre channel expansion module in your UCS 6120, the existing Ethernet configurations can simply carry over to the UCS 6248, with ports 2/7 to 2/16 potentially configurable for either Ethernet or FC.
    Figure 4. Cisco UCS 6120 Port Numbering when Configured with the N10-E0600 Expansion Module

    A

    Slot 1, ports 1 through 8: 10-Gigabit or 1 Gigabit Ethernet capable unencrypted ports

    C

    Slot 2 10-Gigabit Ethernet ports 1 through 6

    B

    Slot 1, ports 1 through 20: 10-Gigabit Ethernet ports

    		   
  • If you have an N10-E0440 expansion module in your UCS 6120, the simplest direct mapping would be to allow Ethernet ports 2/1 to 2/4 on the UCS 6248 to carry over, then remap the fibre channel ports to new port numbers on either slot 1 or slot 2.
    Figure 5. Cisco UCS 6120 Port Numbering when Configured with the N10-E0440 Expansion Module

    A

    Slot 1, ports 1 through 8: 10-Gigabit or 1 Gigabit Ethernet capable ports

    C

    Slot 2 10-Gigabit Ethernet ports 1 through 4

    B

    Slot 1, ports 1 through 20: 10-Gigabit Ethernet ports

    D

    Slot 2 Fibre Channel ports 1 through 4

Fabric Interconnect Port Connection Record

Table 1 Fabric Interconnect Port Connection Record

Fabric Interconnect A/B

Connected to

Slot

Port

Chassis

IOM

Port

LAN or SAN Pin Group

Port Channel Group

Connection Notes

1

1

           

2

            

3

            

4

            

5

            

6

            

7

            

8

            

9

            

10

            

11

           

12

           

13

           

14

           

15

           

16

           

17

           

18

           

19

           

20

           

21

           

22

            

23

           

24

            

25

           

26

           

27

            

28

            

29

           

30

            

31

           

32

           

2

1

           

2

            

3

           

4

            

5

           

6

           

7

            

8

           

9

           

10

            

11

           

12

           

13

           

14

           

15

           

16

           

Upgrading a Fabric Interconnect Cluster


Caution

If your UCS 6120 fabric interconnect uses a N10-E0440 expansion module, or if you are migrating a UCS 6140 with an expansion module in slot 3 to a UCS 6248, pre-plan their deletion and re- mapping following the guidelines in the previous section on port maps.

Unless otherwise noted, for more information about how to perform configuration procedures in Cisco UCS Manager for a particular step, see the Cisco UCS Manager configuration guide for the appropriate release.

Procedure
    Step 1   Mount the replacement fabric interconnects into either the same rack or an adjacent rack.

    Refer to the Cisco UCS 6200 Series Installation Guide for details.
    Step 2   Using either the UCS manager CLI or GUI, verify the state (subordinate or active) of the fabric interconnects.

    See Verifying the High Availability Status and Roles of a Cluster Configuration.
    Step 3   Back up the software configuration information and the UCS Manager software.
    Step 4   Disable the server ports on the subordinate fabric interconnect.
    Step 5   Power down the subordinate fabric interconnect by unplugging it from the power source.

    If you are monitoring the upgrade using a KVM session, you may need to reconnect the KVM session when you power down the fabric interconnect.

    Step 6   Disconnect the cables from the chassis IO Modules or fabric extenders to the subordinate fabric interconnect ports in slot 1 on the old fabric interconnect.
    Step 7   Connect these cables into the corresponding ports on slot 1 of one of the new Cisco UCS 6248 UP fabric interconnects, using the connection records to preserve the port mapping and the configured server pinning.

    See Fabric Interconnect Port Connection Record.

    See Fabric Interconnect Upgrade Considerations.
    Step 8   Disconnect the L1/L2, M1 management, and Console cables on the old fabric interconnect.

    The ports for these connections are on the opposite side of the interconnect, so if your cables are just barely long enough to connect two rack-adjacent UCS 6120 interconnects you will probably need new cables.
    Step 9   Connect the M1 management, and Console cables to the new Cisco UCS 6248 UP.
    Step 10   Connect the L1/L2 cables that were disconnected onto the new Cisco UCS 6248 UP.

    L1 connects to L1, L2 connects to L2.

    Step 11   Disconnect the Ethernet or FC cables from slot 2 of the old fabric interconnect.
    Step 12   Connect the Ethernet or FC cables to the corresponding ports in slot 2 of the new Cisco UCS 6248 UP.

    Some may go to slot 1, depending on the mappings planned out earlier in the process.
    Step 13   Connect the power to the new Cisco UCS 6248 UP, it will automatically boot and run POST tests.
    Important:

    Directly connect the console cable to a terminal and observe the boot sequence. you should at some point see the Basic System Configuration Dialog, where you will configure the switch as a subordinate interconnect. If you do not see this dialog, you either have different builds of software on your old primary and new subordinate, or the new subordinate has previously been part of a cluster and will need to have all configuration information wiped before it can be added to a cluster as a subordinate. In either case, immediately disconnect the L1 and L2 connections and complete the bringup as a standalone fabric interconnect, then correct the issue before proceeding further.

    Step 14   (Optional)Remap UCS 6100 fabric interconnect FC ports 2/1 to 2/4 on a N10-E0440 expansion module or any slot 3 ports onto the new fabric interconnect expansion module.
    1. Use UCS Manager to delete the ports on the subordinate fabric interconnect that you will need to move within the configuration.
    2. For each port you have just deleted, create new ports on either slot 1 or slot 2. These ports must use the same port type definitions as the old ports, but will use different port numbers.
    3. For recently moved Ethernet server ports, reconfigure the associated service profile to use the new port number for the appropriate LAN pin group.
    4. For recently moved uplink Ethernet ports, reconfigure the port channel settings to use the new ports.
    5. For recently moved uplink FC ports, reconfigure the associated service profile SAN pin group to use the new ports.
    6. Re-acknowledge chassis for blade servers and fabric extender for rack servers.

      This will be disruptive to traffic, but is necessary in this specific scenario.
    Step 15   The new subordinate fabric interconnect will automatically synchronize the configuration and database/state information from the primary fabric interconnect.

    Synchronization between primary and subordinate fabric interconnects can take several minutes.

    The port configuration is copied from the subordinate switch to the new hardware.

    Step 16   Verify that the data path is ready.

    See Verifying that the Data Path is Ready.

    Make sure all faults are resolved before proceeding.

    1. Verify and if necessary reconfigure the SAN pin group for FC ports in the associated service profile.
    2. Verify and if necessary reconfigure the LAN pin group for Ethernet ports in the associated service profile.
    3. Verify and if necessary reconfigure the port channel for uplink Ethernet ports.
    Step 17   Enable the server ports that had been disabled in Step 4.
    1. If you have changed port mappings, you may need to reacknowledge the chassis or rack server connected to the subordinate fabric interconnect.
    2. Verify and if necessary reconfigure Ethernet ports as server ports.
    Step 18   Promote the subordinate fabric interconnect to active, and repeat the process on the second Cisco UCS 6248 UP.

    Cable the second new fabric interconnect identically to the first, and allow the reconfiguration done to be applied to the second new fabric interconnect as well.

    See Forcing a Fabric Interconnect Failover.

    Upgrading I/O Modules

    I/O Module Upgrade Considerations

    Be sure the following prerequisites are met before beginning any procedures in this section:

    All software and firmware on all components must be upgraded to the same software release available before attempting an upgrade. This release must support the hardware you will be adding. UCS software version 2.0(1) is the bare minimum version for the UCS 2208XP I/O Module. UCS software version 2.0(2) is the bare minimum version for the UCS 2204XP I/O Module.

    • Do not attempt to implement new software features from the new UCS software version until all required hardware is installed.
    • Make a detailed record of the cabling between I/O Modules and fabric interconnects. You will need to preserve the physical port mapping to maintain the server pinning already configured and minimize down time.
    • For a cluster configuration, both fabric interconnects must have symmetrical connection topologies between fabric interconnect and I/O module.
    • Standalone installations should expect down time. Upgrading an I/O module will be inherently traffic disruptive.
    • When performing hardware upgrades to implement features new to UCS software version 2.0, upgrade the fabric interconnect, then the I/O module, then the adapter in the blade server.
    • Cisco I/O modules only support one, two, four, and (UCS 2208 only) eight link topologies between I/O module and fabric interconnect. If there is a link failure on one of the links, UCS falls back to the next largest possible topology with regards to blade to fabric port mapping. If this ever happens, re-acknowledge the chassis, and manually re-map the fabric ports. It is recommended during the replacement of a fabric interconnect that you have all ports connected to the fabric interconnect before the configuration sync.
    • A best practice would be to perform a full configuration and database/state information backup before performing this hardware upgrade.
    • If you intend to implement the fabric port channel feature available in UCS software version 2.0, you may need to rearrange the cabling between I/O module and fabric interconnect before performing the feature configuration. Fabric port channel will require that all physical links from a given I/O module physically connect to a contiguous block of ports (1-8 or 9-16, and so on). This feature requires a UCS 2200 Series I/O Module connecting to a UCS 6200 Series Fabric Interconnect. See the Cisco UCS Manager configuration guides.

    I/O Module Port Connection Record

    Table 2 I/O Module Port Connection Record

    I/O Module

    Connected to

    Number

    Port

    Fabric Interconnect A or B

    Slot

    Port

    Connection Notes

    1

    1

    		       

    2

    		        

    3

    		        

    4

    		        

    5

    		        

    6

    		        

    7

    		        

    8

    		        

    2

    1

    		       

    2

    		        

    3

    		       

    4

    		        

    5

    		       

    6

    		       

    7

    		        

    8

    		       

    Upgrading an I/O Module

    Procedure
      Step 1   Connect to the cluster’s fabric interconnect, and verify the state (subordinate or active) of the fabric interconnects.
      Step 2   Make a detailed record of the port mapping of the connections between the fabric interconnects and the I/O modules.
      Step 3   Disconnect the 10GbE cables connecting the chassis I/O module to the subordinate fabric interconnect on the I/O module side.
      Step 4   Loosen the captive screws on the old I/O module’s levers.
      Step 5   Pull the levers outward to unseat the old I/O module.
      Step 6   Make sure that the two levers at the front of the new I/O module are pulled open.
      Step 7   Slide the new I/O module into the I/O module slot, ensuring that the new I/O module is fully seated.
      Step 8   Close the levers and tighten the captive screw on each lever.
      Step 9   Connect the 10GbE cables into the appropriate ports on the new Cisco UCS 2208UP I/O Module or Cisco UCS 2204UP I/O Module, using the connection records to preserve the port mapping and the configured server pinning.
      Step 10   If you have changed port mappings, you may need to reacknowledge the I/O Module, fabric extender, or rack server connected to the subordinate fabric interconnect.

      This action will only be necessary if you have changed port mappings, but it will be disruptive if it is needed. See the Cisco UCS Manager configuration guides.
      Step 11   Promote the subordinate fabric interconnect to active, and repeat steps 2 to 9 on the second Cisco UCS 2208 UP or Cisco UCS 2204UP I/O Module.

      Upgrading Adapter Cards

      Adapter Card Upgrade Considerations

      Be sure the following prerequisites are met before beginning any procedures in this section.

      All software and firmware on all components must be upgraded to the latest software release available before attempting an upgrade. UCS software version 2.0(1) is the bare minimum version for the Cisco UCS 6248 UP, the UCS 2208 IO Module, and the Cisco UCS Virtual Interface Card 1280. UCS software version 2.0(2) is the bare minimum version for the Cisco UCS 6296 UP and the UCS 2204 IO Module.

      • Do not attempt to implement new software features from the new UCS software version until all required hardware is installed.
      • You will be able to physically install a Cisco UCS Virtual Interface Card 1280 without first installing a UCS 2208 I/O Module or UCS 2204 I/O Module and additional connections to a fabric interconnect, but to do so would not allow you to take full advantage of the additional performance the Cisco UCS Virtual Interface Card 1280 provides compared to the UCS M81KR. The preferred order is to upgrade the Fabric interconnect, then the I/O Module, and finally the Cisco UCS Virtual Interface Card 1280.
      • Blade servers allowing more than one Adapter Card may mix the Cisco UCS Virtual Interface Card 1280 with the UCS M81KR, the M72KR-E, or the M72KR-Q. Dual Cisco UCS Virtual Interface Card 1280 adapters are also supported.
      • A best practice would be to perform a full configuration and software backup before performing this hardware upgrade.
      • If you intend to implement the fabric port channel feature available in UCS software version 2.0, you may need to rearrange the cabling between IOM and Fabric Interconnect before performing the feature configuration. Fabric port channel will require that all physical links from a given I/O module physically connect to a contiguous block of ports (1-8 or 9-16, and so on). This feature requires a UCS 2200 Series I/O Module connecting to a UCS 6200 Series Fabric Interconnect. See the Fabric Port Channel sections in the configuration guides for more details.

      Upgrading an Adapter Card

      Procedure
        Step 1   Use the locator button in the UCS Manager GUI to confirm which server you are upgrading.
        Step 2   Decommission and remove the blade server using the adapter you wish to upgrade.
        1. Decommission the server as described in Decomissioning the Server or by using the power button on the front panel.
        2. Completely loosen the captive screws on the front of the blade.
        3. Remove the blade from the chassis by pulling the ejector levers on the blade until it unseats the extended memory blade server.
        4. Slide the blade part of the way out of the chassis, and place your other hand under the blade to support its weight.
        5. Once removed, place the blade on an antistatic mat or antistatic foam if you are not immediately reinstalling it into another slot.
        Step 3   Remove the top cover and remove the old Adapter Card.
        1. Press and hold the button down.
        2. (Most models) Press and hold the top cover release button down. While holding the back end of the cover, pull the cover back and then up.
        3. (B230) Press and hold the top cover release button down. While holding the back end of the cover, pull the cover forward. The face plate is meant to be removed along with the top cover.
        4. Loosen the three captive screws attaching the Adapter Card to the motherboard. Remove the adapter connector from the motherboard connector and pull straight up. Be careful not to damage the connectors.
        5. Remove the adapter connector from the motherboard connector and pull straight up. Be careful not to damage the connectors. You may need to gently rock the card from side to side to get the connector to unseat.
        Step 4   Install the new Adapter card.
        1. Position the adapter board connector above the mother board connector and align the three adapter captive screws to the posts on the motherboard.
        2. Firmly press the adapter connector into the motherboard connector. If the seating is bad, it may cause the network connection LED to stay amber when the server is restarted.
        3. Tighten the three captive screws.
        Step 5   Replace the top cover and reinsert the blade server in the chassis.
        Step 6   Recommission the server by reacknowledging the server slot.
        Step 7   When upgrading a Virtual Interface Card, configurations created for the older card will carry over to the new hardware. Additional configuration is now possible, but no re-configuration is required.

        Upgrading Integrated Rack-Mount Servers

        Required Order of Steps for Integrating Cisco UCS Rack-Mount Servers

        After you upgrade the firmware for the existing components, you can integrate one or more Cisco UCS rack-mount servers. When you integrate rack-mount servers, you must perform the steps in the following order:

        1. If you have not already done so, configure the rack server discovery policy in Cisco UCS Manager.
        2. Follow the instructions in the appropriate rack-mount server installation guide for installing and integrating a rack-mount server in a system managed by Cisco UCS Manager.
        3. Wait for Cisco UCS Manager to discover the new server. If server discovery does not begin within a few minutes, acknowledge the server.

        Upgrades of Integrated Rack-Mount Servers

        Cisco UCS, Release 2.0(2) does not support integration with rack-mount servers through Cisco Nexus 2248 FEXes. If you upgrade a Cisco UCS domain that includes integrated rack-mount servers to Release 2.0(2) or later, you must to upgrade your setup to include Cisco Nexus 2232 FEXes.

        For more information, see the appropriate rack-mount server installation guide for migrating a rack-mount server integration to Cisco UCS, Release 2.0(2).