See the “Troubleshooting E-Series Server or NCE Access Issues” section in the “Firmware Management” chapter in the CLI Configuration Guide for Cisco UCS E-Series Servers and the Cisco UCS E-Series Network Compute Engine Integrated Management Controller, Release 3.2,x

To resolve this problem, do the following:

• Use IPMI to reboot CIMC. If using IPMI does not resolve the problem, power cycle the E-Series Server.
• Verify that there is an SD card in the SD0 card slot. The SD card in the SD0 card slot contains the CIMC software and should always be present.

Note	SD card is applicable to M1 and M2 servers. M3 servers have a dedicated SSD flash disk based on eMMC technology. This eMMC storage is not a user replaceable component. This replaces the SD card functionality that was used on M1 and M2 servers.

The below figure shows the location of the SD0 card slot in a single-wide E-Series Server.

The below figure shows the location of the dual SD card slots in a double-wide E-Series Server.

Note	The SD0 card is the lower card, located in the bottom SD card slot.

• Power down the E-Series Server to check whether the SD card has come out of the slot. If it has, reinsert the SD card. Do the following:
  • If the E-Series Server is installed in a Cisco 3900 series router, use the hw-module sm slot oir-stop command to power down the server, reinsert the SD card, and then use the hw-module sm slot oir-start command to start the server.
  • If the E-Series Server is installed in a Cisco 2900 series router, power cycle the router.

Note	Do not remove the SD card when the system is in operation.

If you are running CIMC version 2.2.x, first upgrade to version 2.3.2 and then upgrade to 3.1.x.

If you have already upgraded to CIMC version 3.1.x, type *** to get into the recovery shell and then install the image 3.1.x from the recovery shell.

To resolve this problem, verify the following:

• The FTP server on which you want to download the host image is running.
• The FTP server path to the image files is correct.

To resolve this problem, clear the server's BIOS CMOS memory using the CIMC GUI or the CIMC CLI:

Due to hardware differences between the ISR G2 and ISR 4K platforms, the BIOS setting for Power Restore Policy is not applicable on ISR 4K platforms. Instead, the CIMC automatically enacts a “Restore Last State” policy on ISR 4K.

Note	On ISR 4K platforms, Power Restore Policy settings will be configurable in CIMC beginning with the Release 3.1.3. On ISR G2 platforms, Power Restore Policy settings are still configured through the BIOS settings in CIMC.

Make sure that you configure boot order through the CIMC, and save the changes (“commit” in CLI or “save changes” in GUI). Do not configure boot order through the BIOS setup (<F2>) menu. If you configure boot order through the BIOS setup, the CIMC configuration will overwrite the boot order configuration.

See the “Troubleshooting E-Series Server or NCE Access Issues” section in the “Firmware Management” chapter in the CLI Configuration Guide for Cisco UCS E-Series Servers and the Cisco UCS E-Series Network Compute Engine Integrated Management Controller, Release 3.2,x.

See the “Troubleshooting E-Series Server or NCE Access Issues” section in the “Firmware Management” chapter in the CLI Configuration Guide for Cisco UCS E-Series Servers and the Cisco UCS E-Series Network Compute Engine Integrated Management Controller, Release 3.2,x.

See the “Troubleshooting E-Series Server or NCE Access Issues” section in the “Firmware Management” chapter in the CLI Configuration Guide for Cisco UCS E-Series Servers and the Cisco UCS E-Series Network Compute Engine Integrated Management Controller, Release 3.2,x.

Use the Cisco Host Upgrade Utility to verify and upgrade to the latest firmware image. See the Host Upgrade Utility Guide for Cisco UCS E-Series Servers and the Cisco UCS E-Series Network Compute Engine.

Workaround for ESXi versions 6.7U1,6.7U2,6.0U2, and 6.0U3:

To make VMNIC interface ordering start with the lowest MAC address, follow these steps:

1. Enable Shell and SSH in ESXi.

2. SSH into ESXi.

3. Use esxcli network nic list command to display VMNIC number and its corresponding MAC address.

4. Use localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list command to display the Bus address and VMNIC number mappings.

5. Use localcli command to remap VMNIC number to Bus address that has the lowest MAC address.

6. Reboot ESXi.

7. SSH into ESXi and verify the changes.

This example shows how to display VMNIC number and its MAC address:

~ # esxcli network nic list
Name    PCI Device     Driver  Link  Speed  Duplex  MAC Address         MTU  Description
------  -------------  ------  ----  -----  ------  -----------------  ----  -------------------------------------------------------
vmnic0  0000:04:00.0  ixgben  Up            Up            1000  Full    00:f6:63:b9:65:d6  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic1  0000:04:00.1  ixgben  Up            Up            1000  Full    00:f6:63:b9:65:d7  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic2  0000:09:00.0  tg3     Up            Up            1000  Full    00:f6:63:b9:65:d4  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic3  0000:09:00.1  tg3     Up            Up            1000  Full    00:f6:63:b9:65:d5  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet

This example shows how to display Bus address and VMNIC name mapping:

~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list
Bus type  Bus address          Alias
------------------------------------
pci       p0000:09:00.1        vmnic3
pci       p0000:09:00.0        vmnic2
pci       p0000:07:00.0        vmhba0
pci       p0000:04:00.1        vmnic1
pci       p0000:04:00.0        vmnic0
logical   pci#p0000:04:00.1#0  vmnic1
logical   pci#p0000:04:00.0#0  vmnic0
logical   pci#p0000:07:00.0#0  vmhba0

This example shows how to remap VMNIC number to Bus address that has the lowest MAC address:

[root@localhost:~] localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic0 --bus-address p0000:09:00.0 --bus-type pci
[root@localhost:~]  localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic1 --bus-address p0000:09:00.1 --bus-type pci
[root@localhost:~]  localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic2 --bus-address p0000:04:00.0 --bus-type pci
[root@localhost:~] localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic3 --bus-address p0000:04:00.1 --bus-type pci
[root@localhost:~]  localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic2 --bus-address pci#p0000:04:00.0#0 --bus-type logical
[root@localhost:~] localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic3 --bus-address pci#p0000:04:00.1#0 --bus-type logical
[root@localhost:~] reboot

After a reboot, the VMNIC interface number begins with the lowest MAC address. This example shows the output after the reboot:

~ # esxcli network nic list
Name    PCI Device     Driver  Link  Speed  Duplex  MAC Address         MTU  Description
vmnic0  0000:09:00.0  tg3     Up            Up            1000  Full    00:f6:63:b9:65:d4  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic1  0000:09:00.1  tg3     Up            Up            1000  Full    00:f6:63:b9:65:d5  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic2  0000:04:00.0  ixgben  Up            Up            1000  Full    00:f6:63:b9:65:d6  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic3  0000:04:00.1  ixgben  Up            Up            1000  Full    00:f6:63:b9:65:d7  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T

Workaround for ESXi versions 6.5,6.5U1, and 6.5U2:

To make VMNIC interface ordering start with the lowest MAC address, follow these steps:

1. Enable Shell and SSH in ESXi.

2. SSH into ESXi.

3. Use esxcli network nic list command to display VMNIC number and its corresponding MAC address.

4. Use localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list command to display the Bus address and VMNIC number mappings.

5. Use localcli command to remap VMNIC number to Bus address that has the lowest MAC address.

6. Reboot ESXi.

7. SSH into ESXi and verify the changes.

This example shows how to display VMNIC number and its MAC address:

~ # esxcli network nic list
Name    PCI Device     Driver  Link  Speed  Duplex  MAC Address         MTU  Description
------  -------------  ------  ----  -----  ------  -----------------  ----  -------------------------------------------------------
vmnic0  0000:004:00.0  ixgbe   Up     1000  Full    a8:9d:21:fc:61:12  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic1  0000:004:00.1  ixgbe   Up     1000  Full    a8:9d:21:fc:61:13  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic2  0000:008:00.0  tg3     Up     1000  Full    a8:9d:21:fc:61:10  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic3  0000:008:00.1  tg3     Up     1000  Full    a8:9d:21:fc:61:11  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet

This example shows how to display Bus address and VMNIC name mapping:

~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list
Bus type  Bus address          Alias
------------------------------------
pci       p0000:06:00.0        vmhba0
pci       p0000:08:00.0        vmnic2
pci       p0000:08:00.1        vmnic3
pci       p0000:04:00.1        vmnic1
pci       p0000:04:00.0        vmnic0
logical   pci#p0000:06:00.0#0  vmhba0

This example shows how to remap VMNIC number to Bus address that has the lowest MAC address:

~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic0 --bus-address p0000:08:00.0 --bus-type pci
~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic1 --bus-address p0000:08:00.1 --bus-type pci
~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic2 --bus-address p0000:04:00.0 --bus-type pci
~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias store --alias vmnic3 --bus-address p0000:04:00.1 --bus-type pci
~ # reboot

After a reboot, the VMNIC interface number begins with the lowest MAC address. This example shows the output after the reboot:

~ # esxcli network nic list
Name    PCI Device     Driver  Link  Speed  Duplex  MAC Address         MTU  Description
------  -------------  ------  ----  -----  ------  -----------------  ----  -------------------------------------------------------
vmnic0  0000:008:00.0  tg3     Up     1000  Full    a8:9d:21:fc:61:10  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic1  0000:008:00.1  tg3     Up     1000  Full    a8:9d:21:fc:61:11  1500  Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic2  0000:004:00.0  ixgbe   Up     1000  Full    a8:9d:21:fc:61:12  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
vmnic3  0000:004:00.1  ixgbe   Up     1000  Full    a8:9d:21:fc:61:13  1500  Intel(R) Ethernet Connection X552/X557-AT 10GBASE-T
~ # localcli --plugin-dir /usr/lib/vmware/esxcli/int/ deviceInternal alias list
Bus type  Bus address          Alias
------------------------------------
pci       p0000:06:00.0        vmhba0
pci       p0000:08:00.0        vmnic0
pci       p0000:08:00.1        vmnic1
pci       p0000:04:00.1        vmnic3
pci       p0000:04:00.0        vmnic2
logical   pci#p0000:06:00.0#0  vmhba0
~ #

To Enable ixgbe Driver

After you install or upgrade the ESXi version to specific builds of 6.5 and 6.7 in the UCS-E M3 modules, either the Intel 10Gig NICs (external port) disappear or they remain in downstate. To fix this issue, follow these steps:

# esxcfg-module-d ixgben
# esxcfg-module-e ixgbe
# Reboot the host

This problem occurs if you are using more than 32 GB of RAM with VMware vSphere Hypervisor ™ 5.x. To resolve this problem, reduce the RAM to 32 GB or less, or upgrade your license to FL-SRE-V-HOSTVC.

This problem occurs if you had first installed VMware, and then installed Microsoft Windows. To resolve this problem, do the following:

1. Change the state of the physical drive to JBOD. See the “Changing the Physical Drive State” section in the configuration guides for the version of the CIMC that you are using.
2. Configure RAID to clear partitions. See the “Configuring RAID Using the CIMC GUI” section in the configuration guides for the version of the CIMC that you are using.

The configuration guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-and-configuration-guides-list.html.

During the Microsoft Windows installation, the “Where do you want to install Windows” dialog box appears (see figure below, which displays partitions that were previously created by VMware. This dialog box also displays the following warning message:

Warning: Windows Cannot be Installed to Disk x
 Partition x

To resolve this problem, clear the old partitions by clicking Drive options (advanced) , and then resume with the Microsoft Windows installation.

This problem occurs if you did not provide your full domain name followed by your username. To resolve this problem, provide your full domain name followed by your username. For example:

CIMC login: cert.cisco.com\adadmin
Password: 
PING 172.19.159.52 (172.19.159.52): 56 data bytes
64 bytes from 172.19.159.52: seq=0 ttl=128 time=0.000 ms
--- 172.19.159.52 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.000/0.000/0.000 ms

Note

The supported Cisco IOS software version on E-Series Servers is 15.2(4)M and later versions. For details, see the “Verifying the Router, E-Series Server, and Cisco IOS Software Version Compatibility” section in the getting started guide for the version of the CIMC that you are using. The getting started guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-guides-list.html.

To determine the Cisco IOS version, use the show version or the show inventory command from the router:

Router> show version
Cisco CISCO3945-CHASSIS (revision 1.0) with C3900-SPE150/K9 with 1048576K/63488K bytes of memory.
1 cisco UCSE Module(s)
Router>
 show inventory
NAME: “UCSE Server Module on Slot 4", DESCR: “UCSE Server Module”
PID: UCS-E140D-M1/K9   , VID: V00 , SN: FOC16161F5E

To determine the type of E-Series Server that is installed in the router and in which slot it is installed, use the show diag command from the router:

Router>
 show diag
Slot 4:
UCSE Double Wide Module Port adapter, 2 ports
Port adapter is analyzed 
Port adapter insertion time 4d18h ago
Product (FRU) Number     : UCS-E140D-M1/K9 

You can determine the hardware version of the E-Series Server or the NCE from the CIMC GUI or from the CIMC CLI:

You can determine the BIOS version of the E-Series Server or the NCE from the CIMC GUI or from the CIMC CLI.

You can determine the CIM version installed on the E-Series Server or the NCE from the CIMC GUI or from the CIMC CLI.

This problem occurs if Lock IOS Configuration Changes is enabled in CIMC. Use the CIMC GUI or the CIMC CLI to resolve this problem.

To resolve this problem, after making changes to the configuration, click Refresh .

This problem occurs under the following conditions:

• If you assigned a static IP address for CIMC, and then left the values of DHCP Enabled and DNS-use-DHCP as Yes . To resolve this problem, change the value to No , and then assign the static IP address:

Server /cimc/network # set dns-use-dhcp no
Server /cimc/network *# set dhcp-enabled no
Server /cimc/network *# set v4-addr 192.168.100.78
	
Server /cimc/network *# commit

• If you made configuration changes in one scope, and then tried to commit those changes from another scope.

The commit command must be used to commit changes that are made within the same scope. If you try to use the commit command to submit changes made in a different scope, an error message displays. To resolve this problem, redo and recommit the changes in the same scope.

To resolve this problem, do the following:

• Make sure that the DIMMs are installed and are of the same capacity. Do the following:
  • In the Navigation pane, click the Server tab.
  • On the Server tab, click Inventory. The Inventory page appears.
  • Click the Memory tab. The Capacity column in the Memory Details area allows you to determine the capacity of the DIMMs and whether the DIMM is installed or not.

Verify that the value in the Capacity column for the DIMMs is the same.

If the DIMMs are installed, the Capacity column displays a numerical value; otherwise, it displays Not Installed .

• Check whether there are memory related system event logs. Do the following:
  • In the Navigation pane, click the Server tab.
  • On the Server tab, click System Event Log. The System Event Log page appears.
  • In the Description column, look for events that start with FRU_RAM xxx .
• Check if the DIMM is supported on that server model.
  • Single-Wide E-Series Server—Supports DDR3 1333MHz VLP UDIMM 1.5 V, 4 GB, and 8 GB
  • Double-Wide E-Series Server—Supports DDR3 1333 MHz RDIMM 1.35 V, 4 GB, 8 GB and 16 GB
  • Double-wide with PCIE support—Supports DDR3 1333 MHz RDIMM 1.35 V, 4 GB, 8 GB, and 16 GB
  • EHWIC E-Series NCE—Supports DDR3 1333 MHz SODIMM, 4 GB, 8 GB, and 16GB
• Check if the DIMM is correctly installed in the server slot, otherwise, remove and reinstall the DIMM.

To resolve this problem, do the following:

• Verify that the USB Port to which the CD/DVD device is connected to, is enabled. See Front Panel USB Port Does Not Work.
• Verify that the power current used by the CD/DVD device does not exceed 700mA. If it exceeds 700mA, the USB port to which the CD/DVD device is connected to, might not stay stable. The USB port might go on or off; or might get disabled. To resolve this problem, try connecting the device to an externally-powered USB hub.
• Make sure that the CD/DVD ROM device is configured as the first boot device in the boot order table. For procedure, See Configuring the Boot Order Using the BIOS Setup Menu.
• Reboot the E-Series Server.

If you try to launch vKVM through Proxy, you get a Connection Failed error message. To resolve this problem, disable Proxy.

If Broadcom FCOE configuration does not launch when you press CTRl-S, reboot the E-Series Server and try again. For more information about configuring FCOE, see Broadcom NetXtreme II Network Adapter User Guide.

Ignore this message.

This is a known problem. To work around this problem, connect a physical mouse and keyboard.

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

Use the LSI WebBIOS under the following conditions:

• When the LSI warning message appears at OPROM, which requires drive configuration.
• To configure RAID arrays with SED drives.
• To change or remove security keys and pass phrase for SED.
• To configure foreign SED drives.
• To clear RAID configuration before installing a new operating system.

Note	This error message is applicable to E-Series Servers and the SM E-Series NCE. This error message is not applicable to the EHWIC E-Series NCE.

Symptom—When you enter the show inventory command, the server is not recognized and you get the following error message:

Router> show inventory
NAME: “Unknown on Slot 4", DESCR: “Unknown”
PID: UCS-E140D-M1/K9   , VID: V00 , SN: FOC16161F5E

This problem occurs if you installed an unsupported version of Cisco IOS software. The supported Cisco IOS software version on E-Series Servers is 15.2(4)M and later versions. For details, see the “Verifying the Router, E-Series Server, and Cisco IOS Software Version Compatibility” section in the getting started guide for the version of the CIMC that you are using. The getting started guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-guides-list.html

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

Symptom—After the rebuild, reconstruction, and consistency checks trigger, the RAID operation does not proceed and remains at 0%, and the elapsed time displays 0 seconds.

This problem occurs because RAID operations rely on the memory of the host operating system to function properly. To resolve this problem, verify if the operating system has booted. If it has not booted, boot the operating system.

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

Symptom—When you click Create to configure RAID, you notice that no drives are available in the Select Drives area.

To resolve this problem, make sure that the state of the drive is unconfigured good. See “Changing the Physical Drive State” section in the configuration guides for the version of the CIMC that you are using.

The configuration guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-and-configuration-guides-list.html

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

Symptom—CIMC GUI does not show the RAID array that was originally present in the system.

This problem can occur if during bootup, at LSI OpROM, you concurrently entered wrong or empty pass phrase three times. Because of the “BIOS Continue on Error” feature, the booting process continues, but the drives cannot be accessed.

To resolve this problem, do the following:

1. Reboot the system, and then enter the correct pass phrase.
2. From LSI WebBIOS, configure the new pass phrase or remove security.

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

This problem can occur if during bootup, at LSI OpROM, you concurrently entered wrong or empty pass phrase three times. Because of the “BIOS Continue on Error” feature, the booting process continues, but the drives cannot be accessed.

To resolve this problem, do the following:

1. Reboot the system, and then enter the correct pass phrase.
2. From LSI WebBIOS, configure the new pass phrase or remove security.

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

This problem can occur if during bootup, at LSI OpROM, you concurrently entered wrong or empty pass phrase three times. Because of the “BIOS Continue on Error” feature, the booting process continues, but the drives cannot be accessed.

To resolve this problem, do the following:

1. Reboot the system, and then enter the correct pass phrase.
2. From LSI WebBIOS, configure the new pass phrase or remove security.

Note	This section is applicable to E-Series Servers and the SM E-Series NCE. This section is not applicable to the EHWIC E-Series NCE.

This problem can occur if during bootup, at LSI OpROM, you concurrently entered wrong or empty pass phrase three times. Because of the “BIOS Continue on Error” feature, the booting process continues, but the drives cannot be accessed.

To resolve this problem, do the following:

1. Reboot the system, and then enter the correct pass phrase.
2. From LSI WebBIOS, configure the new pass phrase or remove security.

See the “Diagnostic Tests” chapter in the configuration guides for the version of the CIMC that you are using. The configuration guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-and-configuration-guides-list.html

Symptom—When you enter the show diag command, the server is not recognized and you get the following error message:

Router> show diag
Unknown (type 1889) Port adapter
Port adapter is disabled
Product (FRU) Number     : UCS-E140D-M1/K9 

This problem occurs if you installed an unsupported version of Cisco IOS software. The supported Cisco IOS software version on E-Series Servers is 15.2(4)M and later versions. For details, see the “Verifying the Router, E-Series Server, and Cisco IOS Software Version Compatibility” section in the getting started guide for the version of the CIMC that you are using. The getting started guides are available at: http://www.cisco.com/c/en/us/support/servers-unified-computing/ucs-e-series-servers/products-installation-guides-list.html

From the CIMC CLI, enter the following commands:

Server# scope cimc
Server /cimc
 # scope tech-suppor
t
Server /cimc/tech-support #
 set tftp-ip tftp_server_ip_address Server /cimc/tech-support *# set path /user/user1/supportfile
Server /cimc/tech-support *#
 commit
S
erver /cimc/tech-support #
 start
Server /cimc/tech-support #
 show detail