- Replacing Hard Drives or Solid State Drives
- Replacing Fan Modules
- Replacing a Server Node
- Replacing a Disk Expansion Tray
- Replacing a System I/O Controller (SIOC)
- Replacing a Power Supply
- Replacing DIMMs Inside a Server Node
- Replacing CPUs and Heatsinks Inside a Server Node
- Replacing a Storage Controller Card Inside the Server Node
- Replacing an SSD Inside the Server Node (C3X60 M4 or S3260 M5 Only)
- Replacing an RTC Battery Inside the Server Node
- Replacing an Internal USB Drive Inside the Server Node (C3X60 M3 Only)
- Installing a Trusted Platform Module (TPM) Inside the Server Node
- Replacing an I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing a PCIe Card Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing a Storage Controller Card Inside the I/O Expander (C3X60 M4 Server Nodes Only)
- Replacing an NVMe SSD Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing an RTC Battery Inside the S3260 SIOC
- Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only)
- Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only)
- Replacing an S3260 Chassis
Maintaining the System
This chapter describes how to use LEDs to diagnose system problems and how to install or replace supported hardware components:
Status LEDs and Buttons
This section describes the location and meaning of LEDs and buttons and includes the following topics:
Front-Panel LEDs
Figure 3-1 shows the front-panel LEDs. Table 3-1 defines the front-panel LED states.
|
|
||
|
|
||
|
|
||
|
|
|
|
|
---|---|---|
– Power supply redundancy is lost (power supply unplugged or failed). – At least one DIMM is faulty. – At least one drive in a RAID configuration failed. |
||
|
||
Use these LEDs to indicate the location of a failing drive. Then open the system cover to find exactly which drive is failing by looking at the LEDs on the drive trays.
See Figure 3-8 for an example. In this example, the red LED indicates that the failing drive is in the right half of the internal drive compartment, in row 3. |
Rear-Panel LEDs and Buttons
Figure 3-2 shows the rear-panel LEDs and buttons for a S3260 system. This example is shown with a C3X60 M4 server node and an optional four-drive expansion module. Table 3-2 defines the rear-panel LED states.
Figure 3-2 S3260 System Rear-Panel LEDs and Buttons
If your system has an optional drive expander module, it has one module status LED:
|
|
|
|
|
---|---|---|---|---|
Internal Diagnostic LEDs
Diagnostic LEDs in the Main Chassis
The diagnostic LEDs inside the main chassis compartments can be viewed while the system is powered on. See Figure 3-3 for the locations of these internal LEDs. See Table 3-4 for definitions of the LED states.
Figure 3-3 Internal Diagnostic LED Locations in the Main Chassis
|
|
||
|
|
|
|
|
---|---|---|
Diagnostic LEDs in the Server Node
For the locations of the internal diagnostic LEDs inside the server node, see the service note for your server node:
Preparing for System Component Installation or Replacement
This section describes how to prepare for component installation, and it includes the following topics:
- Required Equipment For Maintenance Procedures
- Shutting Down and Powering Off the S3260 System
- Shutting Down an Individual Server Node
- Removing Chassis and Component Covers
Required Equipment For Maintenance Procedures
The following equipment is used to perform the procedures in this chapter:
- Number 1 Phillips-head screwdriver (for SIOC cover screws)
- 1/4-inch or equivalent flat-head screwdriver (for separating the optional I/O expander from the server node)
- Electrostatic discharge (ESD) strap or other grounding equipment such as a grounded mat
- The tools required for CPU replacement are supplied with a CPU spare.
Shutting Down and Powering Off the S3260 System
You can invoke a graceful shutdown or a hard shutdown of the entire S3260 system by using either the Cisco IMC interface or the system power button on the front panel.
You can also shut down an individual server node rather than the entire system. See Shutting Down an Individual Server Node.
To use the system power button to shut down the chassis, follow these steps:
Step 1 Check the color of the System Power Status LED (see Front-Panel LEDs).
- Green—At least one server node is powered on. Go to Step 2.
- Amber—Both server nodes are powered off. Go to Step 3.
Step 2 Invoke either a graceful shutdown or a hard shutdown:
- Graceful shutdown—Press and release the Power button. The operating system on the server nodes performs a graceful shutdown of both server nodes.
- Emergency shutdown—Press and hold the Power button for 4 seconds to force the power off both server nodes.
Step 3 Disconnect power cords from all power supplies in your system to completely remove AC power and power off the system chassis.
Shutting Down an Individual Server Node
You can invoke a graceful shutdown or a hard shutdown of a server node by using either the Cisco Integrated Management Controller (Cisco IMC) interface, or the power button that is on the face of the server node.
Shutting Down a Server Node By Using the Cisco IMC GUI
To use the Cisco IMC GUI to shut down the server node, follow these steps:
Step 1 Use a browser and the management IP address of the system to log in to the Cisco IMC GUI.
Step 2 In the Navigation pane, click the Chassis menu.
Step 3 In the Chassis menu, click Summary.
Step 4 In the toolbar above the work pane, click the Host Power link.
The Server Power Management dialog opens. This dialog lists all servers that are present in the system.
Step 5 In the Server Power Management dialog, select one of the following buttons for the server that you want to shut down:
- Shut Down—Performs a graceful shutdown of the operating system.
- Power Off—Powers off the chosen server, even if tasks are running on that server.
It is safe to remove the server node from the chassis when the Chassis Status pane shows the Power State as Off for the server node that you are removing.
The physical power button on the server node face also turns amber when it is safe to remove the server node from the chassis.
Shutting Down a Server Node By Using the Power Button on the Server Node
To use the physical server node power button to shut down the server node only, follow these steps:
Step 1 Check the color of the server node power status LED:
- Green—The server node is powered on. Go to step Step 2
- Amber—the server node is powered off. It is safe to remove the server node from the chassis.
Step 2 Invoke either a graceful shutdown or a hard shutdown:
- Graceful shutdown—Press and release the Power button. The software performs a graceful shutdown of the server node.
- Emergency shutdown—Press and hold the Power button for 4 seconds to force the power off the server node.
When the server node power button turns amber, it is safe to remove the server node from the chassis.
Removing Chassis and Component Covers
Opening the Main Chassis Top Covers
This system has three hinged top covers on the main chassis. Opening these covers gives access to the internal-drives compartment and the fan module compartment.
Note The internal drives and cooling fans in the system are hot-swappable and are accessed by opening the top covers. When you rack and cable the system, be sure to allow enough slack in the power and other cables so that the system can be pulled out on the slide rails far enough to allow clearance for opening the top covers.
Step 1 Open the left or right internal-drive compartment cover to access the hot-swappable internal drives:
a. For either the right or left side cover, pull the latch release buttons on both latches toward the outer edges of the chassis. This causes the spring-loaded latches to pop up.
b. With both latches open, swing open the hinged cover from the center toward the outside.
c. To secure the cover, close it down flat and then push both latches flat until they click and lock.
Step 2 Open the fan compartment cover to access the hot-swappable fan modules:
a. Push both latch-buttons toward the center.
b. While holding both latch-buttons, open the hinged cover from the center toward the rear.
c. To secure the cover, hold both latch-buttons while you close the cover flat. Release the latch-buttons.
Figure 3-4 Opening the Main Chassis Top Covers
|
|
||
|
|
Removing a Server Node Cover
To remove a server node cover, see the service note for your server node:
Removing an I/O Expander Cover (C3X60 M4 and S3260 M5 Server Nodes Only)
To remove an I/O expander cover, see the C3X60 M4 server node service note:
Removing an I/O Expander From a Node (C3X60 M4 and S3260 M5 Server Nodes Only)
The I/O expander attaches to the top of the server node. To remove an I/O expander from a server node so that you can access the components inside the server node, see the server node service note:
Removing the System I/O Controller (SIOC) Cover
Note You do not have to slide the system out of the rack to remove the SIOC from the rear of the system.
Step 1 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 2 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged levers to evenly disengage the SIOC from its backplane connector.
b. Pull the SIOC from the system and set it on an antistatic work surface.
a. Use a #1 Phillips-head screwdriver to remove the screws that secure the cover. The version 02 SIOC cover has four securing screws. The Version 03 SIOC cover has six securing screws. See Figure 3-5 or Figure 3-6.
b. Lift the cover straight up off the SIOC.
Step 4 Reinstall the SIOC cover:
a. Set the cover back in place.
b. Replace the four screws that secure the cover.
Step 5 Reinstall the SIOC in the system:
a. Push the SIOC into its bay until it stops against the internal midplane.
b. Close the two levers on the SIOC to fully engage the SIOC connector with its midplane.
c. Tighten the thumbscrew on the SIOC levers.
Step 6 Power on the server node that you powered off in the first step.
Figure 3-5 Removing the SIOC Cover (Version 02 SIOC, UCS-S3260-SIOC)
|
|
Figure 3-6 Removing the SIOC Cover (Version 03 SIOC, UCS-S3260-PCISIOC)
|
|
||
|
|
Installing or Replacing System Components
Warning Blank faceplates and cover panels serve three important functions: they prevent exposure to hazardous voltages and currents inside the chassis; they contain electromagnetic interference (EMI) that might disrupt other equipment; and they direct the flow of cooling air through the chassis. Do not operate the system unless all cards, faceplates, front covers, and rear covers are in place. Statement 1029
– Replacing HDDs or SSDs in the Internal Drive Compartment
– Replacing Hard Drives in the Optional Disk Expansion Tray
– Replacing SAS/SATA Solid State Drives in the Rear Panel Bays
- Replacing Fan Modules
- Replacing a Server Node
- Replacing a Disk Expansion Tray
- Replacing a System I/O Controller (SIOC)
- Replacing a Power Supply
- Replacing DIMMs Inside a Server Node
- Replacing CPUs and Heatsinks Inside a Server Node
- Replacing a Storage Controller Card Inside the Server Node
- Replacing an SSD Inside the Server Node (C3X60 M4 or S3260 M5 Only)
- Replacing an RTC Battery Inside the Server Node
- Replacing an Internal USB Drive Inside the Server Node (C3X60 M3 Only)
- Installing a Trusted Platform Module (TPM) Inside the Server Node
- Replacing an I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing a PCIe Card Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing a Storage Controller Card Inside the I/O Expander (C3X60 M4 Server Nodes Only)
- Replacing an NVMe SSD Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
- Replacing an RTC Battery Inside the S3260 SIOC
- Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only)
- Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only)
Replacing Hard Drives or Solid State Drives
4K Sector Format Drives Considerations
- You must boot 4K sector format drives in UEFI mode, not legacy mode. See Setting Up Booting in UEFI Mode in the BIOS Setup Utility or Setting Up Booting in UEFI Mode in the Cisco IMC GUI.
- Do not configure 4K sector format and 512-byte sector format drives as part of the same RAID volume.
- Operating system support on 4K sector drives is as follows: Windows: Win2012 and Win2012R2; Linux: RHEL 6.5, 6.6, 6.7, 7.0, 7.2; SLES 11 SP3, and SLES 12.
ESXi/Vmware is not supported. - The 6 TB drives have 4096-byte sectors. VMware ESXi does not support this capability and therefore will not work with 6 TB drives.
Setting Up Booting in UEFI Mode in the BIOS Setup Utility
Step 1 Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.
Step 2 Select Compute and then Server 1.
Step 3 Click Launch KVM to open a virtual KVM window for the server node.
Step 4 In the Launch KVM dialog, select Server 1 and click Launch.
Step 5 Reboot server node 1. Watch the KVM window for the prompt to press F2.
Step 6 Enter the BIOS setup utility by pressing the F2 key when prompted during bootup.
Step 7 Go to the Boot Options tab.
Step 8 Set UEFI Boot Options to Enabled.
Step 9 Under Boot Option Priorities, set your OS installation media (such as a virtual DVD) as your
Boot Option #1.
Step 10 Go to the Advanced tab.
Step 11 Select LOM and PCIe Slot Configuration.
Step 12 Set the PCIe Slot ID: HBA Option ROM to UEFI Only.
Step 13 Press F10 to save changes and exit the BIOS setup utility. Allow the server to reboot.
Step 14 After the server reboots and the OS installs, verify the installation:
a. Enter the BIOS setup utility by pressing the F2 key when prompted during bootup.
b. Go to the Boot Options tab.
c. Under Boot Option Priorities, verify that the OS you installed is listed as your Boot Option #1.
Setting Up Booting in UEFI Mode in the Cisco IMC GUI
Step 1 Use a web browser and the management IP address of the system to log into the Cisco IMC GUI management interface.
Step 2 Select Compute and then Server 1.
Step 4 Under BIOS Properties, set Configured Boot Order to UEFI.
Step 6 Click Configure Boot Order.
Step 7 Select the Advanced tab.
Step 9 In the Add Local Disk dialog, enter the information for the 4K sector format drive. Enter a name and specify Slot M.
Step 11 Click Add Virtual Media.
Step 12 In the Add Virtual Media dialog, enter a name for your OS installation virtual media.
Step 15 Click Launch KVM to open a virtual KVM window for the server node.
Step 16 In the Launch KVM dialog, select Server 1 and click Launch.
Step 17 Activate virtual media. Pull down the Virtual Media menu on the KVM window and select Activate Virtual Devices.
Step 18 Reboot the server node.
Step 19 Press F6 during the boot to enter the boot device menu.
Step 20 Select UEFI: Cisco vKVM-Mapped vDVD and press Enter.
Step 21 Proceed with the installation of your OS.
After the OS installs and the system reboots, your OS is listed as a boot option.
Replacing HDDs or SSDs in the Internal Drive Compartment
Internal Drive Population Guidelines
The system has 56 internal drive bays in the main chassis. Figure 3-7 shows the internal drive bay numbering. When populating internal drives and for optimal results please adhere to the following guidelines:
- Populate HDDs starting from the lowest-numbered bays to the highest. Populate row 1 – 14, then row 15 – 28, and so on.
- Populate internal SSDs starting from the highest-numbered bays to the lowest. Populate bay 56, then bay 55, and so on.
For example, a system with 36 HDDs and 2 SSDs should have the HDDs in bays 1 – 36 and the SSDs in bays 56 and 55.
- The four colored boxes shown in Figure 3-7 represent the four power groups in which power is distributed to the drive bays.
Figure 3-7 Internal Drive Bay Numbering
Identifying a Faulty Internal Drive
The system has internal-drive fault LEDs on the right-front handle (see Figure 3-1). Use these LEDs to get an indication of the location of a failing drive.
Step 1 Observe the internal-drive fault LEDs on the right-front handle.
- The two columns of LEDs correspond to the two halves of the internal drive compartment (under either the right- or left-side top cover).
- The four numbered rows of LEDs correspond to the four horizontal rows of drive bays (14 drive bays in each row).
See Figure 3-8 for an example. In this example, the amber LED indicates that the failing drive is in the right half of the internal drive compartment, in row 3.
Step 2 Open the right- or left-side cover and look at the fault LEDs on the drive trays.
A solid amber fault LED indicates a failed drive.
Figure 3-8 Internal-Drive Status LED Example
|
Internal-drive fault LEDs on right-front handle of system, indicating faulty drive in right side of row 3 |
|
|
|
|
Replacing Internal Drives
Note Cisco PID UCSC-C3X60-HD4TB (Toshiba model MG03SCA400) has been EOLed and should be replaced if any problems occur. This Cisco PID now uses a newer HGST model.
Note SAS HDDs and SSDs are hot-swappable and can be replaced without removing power from the system.
Step 1 Slide the system out the front of the rack far enough so that you can open the top cover.
Step 2 Identify a failing drive as described in Identifying a Faulty Internal Drive
Step 3 Open the internal-drive compartment cover.
Step 4 Remove the faulty drive:
a. Press the release button on the drive carrier. The drive lever pops up.
b. Lift the drive lever to the fully open, 90-degree position, then lift the drive straight up out of its bay.
Spare drives are already installed in a carrier, so it is not necessary to remove the old drive from its carrier.
Note Observe the drive population guidelines in Internal Drive Population Guidelines.
a. Align the new drive with the empty bay. Orient the drive so that its connector aligns with the connector on the board.
b. Lower the drive until it touches the board connector and the drive lever begins to close.
c. Press the drive lever down flat until it clicks and locks.
Step 6 Close the chassis cover and push the system back into the rack.
Figure 3-9 Internal Drive Carrier Features
|
|
||
|
|
Replacing Hard Drives in the Optional Disk Expansion Tray
Disk Expansion Tray Drives Population Guidelines
The optional disk expansion tray can hold up to four 3.5-inch hard drives,. Drive numbering is shown in Figure 3-10. When populating these drives, follow these guidelines.
Figure 3-10 Disk Expansion Tray and Solid State Drive Numbering
Identifying a Faulty Disk Expansion Tray Drive
Each drive has a fault LED that lights solid amber to indicate a failing drive.
Replacing Drive Expander Module Drives
Note SAS/SATA drives are hot-swappable and can be replaced without removing power from the system.
Step 1 Identify a failing drive as described in Identifying a Faulty Disk Expansion Tray Drive.
Step 2 Remove the faulty drive:
a. Press the release button on the drive carrier. The drive lever pops up.
b. Lift the drive lever to the fully open position, then pull the drive straight up out of its bay.
Spare drives are already installed in a carrier, so it is not necessary to remove the old drive from its carrier.
a. Align the new drive with the empty bay and then push the drive in until it touches the board connector and the drive lever begins to close.
b. Press the drive lever down flat until it clicks and locks.
Figure 3-11 Drive Expander Module Drive Carrier Features
|
|
||
|
|
Replacing SAS/SATA Solid State Drives in the Rear Panel Bays
Rear-Panel SAS/SATA Solid State Drive Population Guidelines
There are four bays for SAS/SATA SSDs in the rear panel. Drive numbering is shown in Figure 3-12. When populating these drives, follow these guidelines.
- Server 1 SSD1 and SSD2 can be managed by server node 1. See also Management Architecture.
- Server 2 SSD1 and SSD1 can be managed by server node 2.
- Populate drives starting with the lowest-numbered bay to the highest within each server-controlled pair.
- If a pair of SSDs is not present, keep the blanking panel in the empty two SSD bays to ensure proper air flow (see Figure 3-12).
Identifying a Faulty Solid State Drive
Each solid state drive bay has a fault LED that lights solid amber to indicate a failing drive (see Rear-Panel LEDs and Buttons).
Replacing Solid State Drives
Note SAS/SATA solid state drives are hot-swappable and can be replaced without removing power from the system.
Step 1 Remove a faulty solid state drive:
a. Grasp and pinch the release latch toward the center.
b. Pull the solid state drive straight out of the bay.
Step 2 Install a new solid state drive:
Note Observe the drive population guidelines in Rear-Panel SAS/SATA Solid State Drive Population Guidelines.
a. Align the new drive with the empty bay (with the label facing up) and then push the drive in until it touches the board connector.
b. Grasp and pinch the release latch toward the center while you push the drive fully into the bay, and then release the release latch.
Figure 3-12 Solid State Drive Bay Features
|
|
||
|
|
Replacing Fan Modules
Each fan module contains two fans. See Figure 3-13 for the fan numbering.
You do not have to shut down or power off the system to replace fan modules because they are hot-swappable. Do not operate the system with a fan module removed for more than one minute.
Tip Each fan module has a fault LED that lights amber if the fan module fails.
Step 1 Slide the system out the front of the rack far enough so that you can open the fan compartment cover.
Step 2 Open the fan compartment cover as described in Opening the Main Chassis Top Covers.
a. Grasp the two latches on the top of the fan and pinch them toward the center.
b. Lift the fan module straight out of the bay.
Step 4 Install a new fan module:
Note The arrow on the fan module that indicates the air flow direction should point to the rear of the system.
a. Align the fan module with the bay so that the connector on the bottom of the fan module is aligned with the socket on the floor of the chassis.
b. Lower the fan module until it touches the socket, then push down firmly until the latch locks.
Step 5 Close the fan compartment cover and then push the system back into the rack.
Figure 3-13 Fan Modules (Top View)
|
|
||
|
|
Replacing a Server Node
The system can support one or two server nodes.
- Cisco IMC releases earlier than 2.0(13): If your S3260 system has only one server node, it must be installed in bay 1.
- Cisco IMC releases 2.0(13) and later: If your S3260 system has only one server node, it can be installed in either server bay.
Note Whichever bay a server node is installed to, it must have a corresponding SIOC. That is, a server node in bay 1 must be paired with a SIOC in SIOC slot 1; a server node in bay 2 must be paired with a SIOC in SIOC bay 2. See Overview of Cisco UCS S3260 Architecture for more information.
To replace a server node, including exporting and importing its configuration, see the service note for your server node:
Replacing a Disk Expansion Tray
The system can support one optional disk expansion tray, in server bay 2 only.
Tip The tray contains one status LED (see Figure 3-14 for the location and Table 3-3 for definitions of the LED states).
Figure 3-14 Disk Expansion Tray External Features (Four-Drive Version Shown)
|
|
Note The disk expansion tray is hot-swappable, which means that you can remove it without shutting down system power.
The disk expansion tray is accessed from the rear of the system, so you do not have to pull the system out from the rack.
Step 1 Remove a disk expansion tray from the system:
a. Grasp the two ejector levers and pinch their latches to release the levers (see Figure 3-14).
b. Rotate both levers to the outside at the same time to evenly disengage the tray from the midplane connectors.
c. Pull the tray straight out from the system.
Step 2 Remove any drives from the old tray and move them to your new disk expansion tray. Install each drive to the same position that it occupied in the old tray.
Step 3 Install a new disk expansion tray (server bay 2 only):
a. With the two ejector levers open, align the new tray with the empty bay.
b. Push the tray into the bay until it engages with the midplane connectors.
c. Rotate both ejector levers toward the center until they lay flat and their latches lock into the rear of the tray.
Replacing a System I/O Controller (SIOC)
The system can support up to two system I/O controllers (SIOCs).
For information about the management architecture and the chassis management controller (CMC) that is in each SIOC, see Management Architecture.
This section contains the following topics:
- Replacing a SIOC in a Single-SIOC System
- Replacing a SIOC in a Dual-SIOC System
- Setting SIOC QSFP Port Speed
- Updating SIOC Adapter Firmware in a System With Dual SIOCs But Only One Server Node
- Migrating From Version 02 SIOC to Version 03 SIOC
Note If you are replacing a Version 02 SIOC (UCSC-S3260-SIOC) with a Version 03 SIOC (UCS-S3260-PCISIOC), use the procedure in Migrating From Version 02 SIOC to Version 03 SIOC. Version 03 SIOCs require Cisco IMC 4.0(1n) or later and for UCS Manager-controlled systems, Cisco UCS Manager 4.0(1n) or later.
Note The Version 03 SIOC is supported only with S3260 M5 or later server nodes.
Note Do not mix different versions of SIOCs in the same system.
Note If your system has two Version 03 SIOCs, they must have identical adapter cards. Do not mix adapter cards between Version 03 SIOCs in a system.
Note A Version 01 SIOC from a Cisco C3160 system cannot be installed in a Cisco S3260 system.
Note If you move a SIOC from one chassis to another, the SIOC’s CMC configuration is treated as incompatible and is automatically deleted. The CMC will sync with the active CMC configuration.
Figure 3-15 SIOC External Mechanical Features (Version 03 SIOC shown)
|
|
||
|
|
Replacing a SIOC in a Single-SIOC System
You do not have to slide the system out of the rack to remove the SIOC from the rear of the system. The SIOC can be replaced without powering off the chassis.
Step 1 Export the CMC configuration from the SIOC.
Step 2 Export the adapter configuration from the SIOC.
Step 3 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 4 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.
b. Pull the SIOC from the system.
Step 5 Version 03 SIOC Only : If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:
a. Remove the cover from the existing Version 03 SIOC as described in Removing the System I/O Controller (SIOC) Cover.
b. Remove the adapter card as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).
c. Remove the NVMe SSDs as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).
d. Remove the cover from the replacement Version 03 SIOC.
e. Install the adapter card to the replacement SIOC as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).
f. Install the NVME SSDs to the replacement SIOC as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).
g. Install the cover to the replacement SIOC.
a. Push the SIOC into its bay until it stops against the internal backplane.
b. Close the two ejector levers on the SIOC to fully engage its connector with the midplane connector.
c. Tighten the thumbscrew on the SIOC ejector levers.
Step 7 Wait for the CMC in the new SIOC to finish booting.
Step 8 Import the CMC configuration that you exported earlier to the new SIOC’s CMC.
Step 9 Import the adapter configuration that you exported earlier to the new SIOC’s CMC.
Step 10 Power on the server node that you powered off earlier.
Step 11 Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.
See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions on updating the firmware.
Note If you are using Cisco Card NIC mode, which uses the SIOC’s uplink interfaces to manage the system, you might need to configure the uplink’s port speed to either 4x10 Gbps or 40 Gbps to match your network installation. See Setting SIOC QSFP Port Speed.
Replacing a SIOC in a Dual-SIOC System
You do not have to slide the system out of the rack to remove the SIOC from the rear of the system. The SIOC can be replaced without powering off the chassis.
Step 1 Set the CMC in the SIOC as the standby CMC, if it is not already.
Step 2 Export the adapter configuration from the SIOC.
Step 3 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 4 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.
b. Pull the SIOC from the system.
Step 5 Version 03 SIOC Only: If desired, transfer the adapter card and any NVMe SSDs from the existing Version 03 SIOC to the new Version 03 SIOC:
a. Remove the cover from the old Version 03 SIOC as described in Removing the System I/O Controller (SIOC) Cover.
b. Remove the adapter card as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).
c. Remove the NVMe SSDs as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).
d. Remove the cover from the new Version 03 SIOC.
e. Install the adapter card to the replacement SIOC as described in Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only).
f. Install the NVME SSDs to the replacement SIOC as described in Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only).
g. Install the cover to the replacement SIOC.
a. Push the SIOC into its bay until it stops against the internal backplane.
b. Close the two ejector levers on the SIOC to fully engage the SIOC connector with the midplane.
c. Tighten the thumbscrew on the SIOC ejector levers.
Step 7 Wait for the CMC in the new SIOC to finish booting.
Step 8 Power on the server node that you powered off earlier.
The configuration in the active CMC is automatically synched with the standby CMC in the SIOC that you just installed.
Step 9 Update the SIOC firmware using the Host Upgrade Utility. This will ensure that the firmware version on the SIOC is compatible with the latest system firmware.
See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions on updating the firmware.
Note If you are using Cisco Card NIC mode, which uses the SIOC’s uplink interfaces to manage the system, you might need to configure the uplink’s port speed to either 4x10 Gbps or 40 Gbps to match your network installation. See Setting SIOC QSFP Port Speed.
Step 10 Import the adapter configuration that you exported earlier to the new SIOC’s CMC.
Setting SIOC QSFP Port Speed
Optional: Set the port speeds for the SIOC QSFP ports:
Step 1 Connect a keyboard and console to the system or log in remotely and view a virtual KVM window.
Step 2 Reboot the system and press F8 when prompted to launch the Cisco IMC Configuration Utility.
Step 3 After the first utility screen is displayed, press F1 twice to go to the third utility screen, which has the settings for Adapter Port Speeds.
Step 4 Set the desired port speeds. “Adapter-1” is SIOC 1; “Adapter-2” is SIOC 2, if present.
Step 5 Press F10 to save your changes and exit the utility.
Updating SIOC Adapter Firmware in a System With Dual SIOCs But Only One Server Node
The recommended method for updating SIOC adapter firmware is to run the Cisco Host Upgrade Utility (HUU) on the server node that is associated with the SIOC. However, if a system has only one server node and two SIOCs, this method does not work to upgrade firmware on the SIOC that is not associated with a server node (SIOC 1 is associated with a server node in bay 1; SIOC 2 is associated with a server node in bay 2).
In this case, you must manually upgrade the adapter firmware on the un-associated SIOC:
Using the Cisco IMC GUI Interface to Update SIOC Adapter Firmware
Step 1 Install the SIOC adapter firmware update:
a. In the Admin menu, click Firmware Management.
b. In the Component Column, select Adapter-SIOC1 or Adapter-SIOC2.
c. Click Update. The Update Firmware dialog opens.
d. Browse to select the firmware for the SIOC adapter.
e. Click Install Firmware to begin download and installation.
a. In the Component Column, select Adapter-SIOC1 or Adapter-SIOC2.
Note You must reset the adapter to make the activated firmware version the running version. Resetting the adapter also causes the host to reset.
Step 3 Reset the adapter in the SIOC:
a. In the Navigation pane, click the Networking menu.
b. Click the Adapter Card tab. The General tab appears.
c. Select Adapter-SIOC1 or Adapter-SIOC2.
d. In the Actions area of the General tab, click Reset and then click Yes to confirm.
The adapter and host reset. The activated firmware becomes the running version.
Using the Cisco IMC PMCLI Interface to Update SIOC Adapter Firmware
Step 1 Update and activate the adapter firmware by using the following commands:
server1/chassis#
update-adapter-fw tftp nnn.nnn.nnn.nnn /fw-image.bin activate 1
Note You must reset the adapter to make the activated firmware version the running version. Resetting the adapter also causes the host to reset.
server/chassis#
adapter-reset <SIOC1 or SIOC2>
server/chassis#
adapter-reset SIOC1
The adapter and host reset. The activated firmware becomes the running version.
Replacing a Power Supply
The system requires four power supplies, which are redundant as 3+1.
Note Do not mix power supply types in the server. Both power supplies must be identical.
This section contains these procedures:
- Replacing an AC Power Supply
- Installing a DC Power Supply (First-Time Installation)
- Replacing a DC Power Supply
- See also Power Specifications for power supply specifications.
- See also Rear-Panel LEDs and Buttons for information about power supply LEDs.
Replacing an AC Power Supply
To replace or install an AC power supply, follow these steps:
Note You do not have to power off the system to replace up to one power supply because they are redundant as 3+1.
Step 1 Remove the power supply that you are replacing or a blank panel from an empty bay (see Figure 3-16):
a. Remove the power cord from the power supply that you are replacing.
b. Grasp the power supply handle while pinching the release lever toward the handle.
c. Pull the power supply out of the bay.
Step 2 Install a new power supply:
a. Grasp the power supply handle and insert the new power supply into the empty bay.
b. Push the power supply into the bay until the release lever locks.
c. Connect the power cord to the new power supply.
d. If you powered off the system, press and hold the system Power button for four seconds to return the system to main power mode.
Figure 3-16 Removing and Replacing Power Supplies
|
|
Installing a DC Power Supply (First-Time Installation)
Warning A readily accessible two-poled disconnect device must be incorporated in the fixed wiring. Statement 1022
Warning This product requires short-circuit (overcurrent) protection, to be provided as part of the building installation. Install only in accordance with national and local wiring regulations. Statement 1045
Warning Installation of the equipment must comply with local and national electrical codes. Statement 1074
Step 1 Turn off the DC power source from your facility’s circuit breaker to avoid electric shock hazard.
Note The required DC input cable is Cisco part CAB-48DC-40A-8AWG. This 3-meter cable has a 3-pin connector on one end that is keyed to the DC input socket on the power supply, as shown in Figure 3-17. The other end of the cable has no connector so that you can wire it to your facility’s DC power.
Step 2 Wire the non-terminated end of the cable to your facility’s DC power input source.
Step 3 Connect the terminated end of the cable to the socket on the power supply. The connector is keyed so that the wires align for correct polarity and ground, as shown in Figure 3-17.
Step 4 Return DC power from your facility’s circuit breaker.
Figure 3-17 1050 W, –48 VDC Power Supply and Cable
|
|
See Rear-Panel LEDs and Buttons for details. |
|
|
|
Replacing a DC Power Supply
Note You do not have to power off the system to replace up to one power supply because they are redundant as 3+1.
Step 1 Remove the power supply that you are replacing or a blank panel from an empty bay:
a. Remove the CAB-48DC-40A-8AWG keyed cable connector from the power supply that you are replacing.
b. Grasp the power supply handle while pinching the release lever toward the handle.
c. Pull the power supply out of the bay.
Step 2 Install a new power supply:
a. Grasp the power supply handle and insert the new power supply into the empty bay.
b. Push the power supply into the bay until the release lever locks.
c. Connect the CAB-48DC-40A-8AWG keyed cable connector to the new power supply.
d. If you powered off the system, press and hold the system Power button for four seconds to return the system to main power mode.
Replacing DIMMs Inside a Server Node
To replace DIMMs inside a server node, see the service note for your server node:
Replacing CPUs and Heatsinks Inside a Server Node
To replace CPUs inside a server node, see the service note for your server node:
Replacing a Storage Controller Card Inside the Server Node
To replace a storage controller card inside a server node, see the service note for your server node:
Replacing an SSD Inside the Server Node (C3X60 M4 or S3260 M5 Only)
To replace a solid state drive (SSD) inside a C3X60 M4 or S3260 M5 server node, see the service note for your server node:
Replacing an RTC Battery Inside the Server Node
To replace an RTC battery inside a server node, see the service note for your server node:
Replacing an Internal USB Drive Inside the Server Node (C3X60 M3 Only)
For instructions on replacing an internal USB drive inside a C3X60 M3 server node, and for enabling or disabling the USB port, see the C3X60 M3 service note:
Installing a Trusted Platform Module (TPM) Inside the Server Node
To install a TPM inside a server node, see the service note for your server node:
Replacing an I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
The C3X60 M4 or S3260 M5 server node might have an optional I/O expander attached to its top. To replace an I/O expander, see the service note for your server node:
Replacing a PCIe Card Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
To replace a PCIe card inside an optional I/O expander, see the service note for your server node:
Replacing a Storage Controller Card Inside the I/O Expander (C3X60 M4 Server Nodes Only)
To replace a storage controller card inside an optional I/O expander, see the service note for the Cisco UCS C3000 M4 server node:
Note The storage controllers that are supported with the S3260 M5 server node are not supported in the I/O expander.)
Replacing an NVMe SSD Inside the I/O Expander (C3X60 M4 or S3260 M5 Server Nodes Only)
To replace a PCIe card inside an optional I/O expander, see the service note for your server node:
Replacing an RTC Battery Inside the S3260 SIOC
The real-time clock (RTC) battery retains settings when the SIOC is disconnected from power. The battery type in the SIOC is CR1632. Cisco supports the industry-standard CR1632 battery, which can be purchased from most electronic stores.
Note When the RTC battery is removed or it completely loses power, settings that were stored in the CMC of the SIOC are lost. You must reconfigure the CMC settings after installing a new battery.
Step 1 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 2 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.
b. Pull the SIOC from the system.
Step 3 Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.
Step 4 Gently pry under the battery to lift it from its socket on the SIOC board.
Note The battery socket is in different locations in version 02 and Version 03 SIOCs. See the following figures.
Step 5 Insert the new battery into the socket and then press down until it sits flat.
Step 6 Replace the cover to the SIOC.
Step 7 Replace the SIOC to the system:
a. Push the SIOC into its bay until it stops against the internal midplane.
b. Close the two levers on the SIOC to fully engage the SIOC connector with its backplane.
c. Tighten the thumbscrew on the SIOC levers.
Step 8 Power on the server node that you powered off earlier.
Step 9 Reconfigure the settings of the CMC in this SIOC.
Figure 3-18 RTC Battery Location Inside the Version 02 S3260 SIOC (UCS-S3260-SIOC)
|
Battery socket on version 02 SIOC board, |
|
Figure 3-19 RTC Battery Location Inside the Version 03 S3260 SIOC (UCS-S3260-PCISIOC)
|
Battery socket on Version 03 SIOC board, |
|
Replacing a PCIe Adapter Card Inside the S3260 SIOC (Version 03 SIOC Only)
The Version 03 SIOC uses a removable PCIe adapter card that plugs into a horizontal socket. (The version 02 SIOC uses an embedded chip and so does not have a removable adapter card.)
Cisco VIC Cards Supported in Version 03 SIOC (UCSC-S3260-PCISIOC)
At this time, the following Cisco Virtual Interface Cards (VICs) are supported in the Version 03 SIOC:
Note If your S3260 system has two Version 03 SIOCs, the adapter card in each must be identical. Do not mix adapter cards between Version 03 SIOCs.
Step 1 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2)
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 2 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.
b. Pull the SIOC from the system.
Step 3 Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.
Step 4 Remove an existing adapter card:
a. Remove the SIOC side panel that is closest to the adapter card. Turn the SIOC upside down and remove the three screws that secure the side panel from the underside of the SIOC.
c. Slide the adapter card horizontally to disconnect its edge connector from the socket.
Step 5 Install a new adapter card:
a. Set the card in place on the SIOC floor. Align the card’s edge connector with the socket.
b. Gently slide the card into the socket, pushing evenly on both ends of the card. Stop when the card is fully in the socket and its rear-panel tab sits flat.
c. Return the side panel to the SIOC. Set the side panel in place so that its edge sits on top of the SIOC floor and its three screw holes align with those on the SIOC.
d. Turn the SIOC upside down and install the three screws that secure the side panel.
Step 6 Turn the SIOC over and reinstall the SIOC cover. Install the six screws that secure the cover.
Step 7 Replace the SIOC to the system:
a. Push the SIOC into its bay until it stops against the internal midplane.
b. Close the two levers on the SIOC to fully engage the SIOC connector with its backplane.
c. Tighten the thumbscrew on the SIOC levers.
Step 8 Power on the server node that you powered off earlier.
Figure 3-20 Adapter Card in Version 03 SIOC (UCS-S3260-PCISIOC)
|
|
Location of three screws that secure the side panel (access from the underside of the SIOC) |
|
|
|
Replacing an NVMe SSD Inside the S3260 SIOC (Version 03 SIOC Only)
SIOC NVMe SSD Population Rules
The Version 03 SIOC can contain up to two 2.5-inch NVMe SSDs in a sled assembly. You must remove the sled when installing or replacing the SSDs.
Each NVMe SSD slot in a SIOC is numbered for the server that supports it (SERVER 1 NVME or SERVER 2 NVME).
- Populate an NVMe SSD in SERVER 1 NVME only if you have a server node in upper server bay 1.
- Populate an NVME SSD in SERVER 2 NVME only if you have a server node in lower server bay 2.
Note If your system configuration has an M5 server node in lower server bay 2 and an I/O expander in upper server bay 1, you cannot populate an NVMe SSD in the SERVER 1 NVME slot in either SIOC. See Figure 3-21and Figure 3-22 for illustrations of the system connectivity and labeling.
Figure 3-21 NVMe SSD Connectivity in Version 03 SIOC (UCS-S3260-PCISIOC)
Procedure
Step 1 Power off the connected server nodes based on the scenario:
a. If there is one or no NVMe in the SIOC you are removing, power off the server node that is paired with the SIOC that you are removing (SIOC 1 is paired with server node1; SIOC 2 is paired with server node 2).
b. If there are two NVMes in the SIOC you are removing, power off both server nodes.
See Shutting Down an Individual Server Node.
Step 2 Remove the SIOC from the system:
a. Loosen the single captive thumbscrew on the SIOC and then open its two hinged ejector levers to evenly disengage the SIOC from its midplane connector.
b. Pull the SIOC from the system.
Step 3 Remove the SIOC cover as described in Removing the System I/O Controller (SIOC) Cover.
Note The NVMe SSD sled (UCS-S3260-NVMSLD2) in the Version 03 SIOC is not interchangeable with the NVMe SSD sled (UCS-S3260-NVMSLD1) in the S3260 M5 server node.
Step 4 Remove the NVMe SSD sled assembly from the SIOC.
Loosen the two captive thumbscrews that secure the sled to the SIOC floor and then pull the sled so that the connectors on any installed SSDs disengage from the two sockets on the SIOC board.
Figure 3-22 NVMe SSDs in Version 03 SIOC (UCS-S3260-PCISIOC)
|
|
Populate this slot only when you have a server node in lower server bay 2. |
|
|
Populate this slot only when you have a server node in upper server bay 1. |
|
Step 5 Remove the three screws that secure an SSD to the sled. Two screws enter from the outside edge and one screw enters from the underside of the sled.
Figure 3-23 NVMe SSD Securing Screws on Sled
|
Screws that secure NVME 1 SERVER SSD to the sled assembly (three) |
|
Screws that secure NVME 2 SERVER SSD to the sled assembly (three) |
Step 6 Install a new NVMe SSD:
Note Before installing, see the SIOC NVMe SSD Population Rules.
a. Install the new SSD to the sled. Set the SSD in place so that the connector end faces away from the bracket.
b. Install the 3 screws that secure the SSD to the sled.
c. Set the sled into the SIOC so that the connectors on the two SSDs align with the two sockets on the SIOC board.
d. Slide the SSD connectors into the sockets as you tilt the sled down flat.
e. Tighten the two thumbscrews on the sled assembly to secure it to the SIOC board standoffs.
Step 7 Reinstall the SIOC cover.
Step 8 Replace the SIOC to the system:
a. Push the SIOC into its bay until it stops against the internal midplane.
b. Close the two levers on the SIOC to fully engage the SIOC connector with its backplane.
c. Tighten the thumbscrew on the SIOC levers.
Step 9 Power on the server node that you powered off earlier.
Replacing an S3260 Chassis
This procedure is for replacing an S3260 chassis and transferring all components to the new chassis.
Step 1 Export the component configurations (CMC, BMC, VIC adapters) and save them to a local computer using the import-export command mode.
Refer to the Cisco UCS Integrated Management Controller CLI Configuration Guide for S3260 Storage Servers. See the chapter, Server Utilities and the section, Exporting and Importing the Cisco IMC and BMC Configuration.
Step 2 Perform a graceful shutdown of all server nodes.
See Shutting Down an Individual Server Node for more information.
Step 3 Disconnect all power cables from all power supplies to fully remove power from the chassis.
Step 4 Remove each server node and install each to the same bay position in the new chassis.
If a server node has an I/O expander attached, move the node and expander attached together.
See the service note for your server node version for more information: Server Node Service Notes.
Step 5 If the system has a disk expansion tray in place of a server node, remove it with its disks in place and install the tray with disks to the same bay position in the new chassis.
See Replacing a Disk Expansion Tray for more information.
Step 6 Remove each SIOC and install each to the same bay position in the new chassis.
See Replacing a System I/O Controller (SIOC) for more information.
Step 7 Remove each rear-panel boot SSD and install each to the same bay position in the new chassis.
See Replacing SAS/SATA Solid State Drives in the Rear Panel Bays for more information.
Step 8 Remove each power supply and install each to the same bay position in the new chassis.
See Replacing a Power Supply for more information.
Step 9 Open the main chassis top cover on the old and new chassis.
See Opening the Main Chassis Top Covers for more information.
Step 10 Remove each top-loading hard drive and install each to the same bay position in the new chassis.
See Replacing HDDs or SSDs in the Internal Drive Compartment for more information.
Step 11 Open the fan compartment cover on the old and new chassis.
Step 12 Remove each cooling fan and install each to the same bay position in the new chassis.
See Replacing Fan Modules for more information.
Step 13 If you have not done so already, remove the empty chassis from your rack.
Step 14 Close all covers on the new chassis.
Step 15 Install the new chassis to your rack.
See Installing the System in a Rack for instructions.
Note When a SIOC is moved to a new chassis, the CMC settings are reset to the factory defaults and so you must perform initial setup as you would for a new system.
Step 16 Power on, connect cables to the system, and log in to the Cisco IMC Configuration Utility as described in Connecting the System and Opening the Setup Utility.
Step 17 Configure the network settings for the system as described in Setting Up the System Using the Cisco IMC Configuration Utility.
- You can use a DHCP server to set IP addresses or set them statically using the procedure in Setting Static CMC and BMC Internal IP Addresses.
- For more information about the IP addresses in the system (up to five), see System IP Addresses.
Step 18 After the system has connection to the network, use the Host Upgrade Utility to update the system to your desired level and ensure that all components in the system are running compatible software levels.
See the Cisco Host Upgrade Utility User Guide For S3260 Storage Servers for instructions.
Step 19 Import the component configurations that you exported in step 1.
Refer to the Cisco UCS Integrated Management Controller CLI Configuration Guide for S3260 Storage Servers. See the chapter Server Utilities and the section Exporting and Importing the Cisco IMC and BMC Configuration.
Step 20 Verify that all configurations imported successfully.
Step 21 Verify that chassis health is optimal.
Service Headers on the Server Node Board
The server node includes headers that you can jumper for certain service functions.
For information about the service headers, see the service note for your server node: