Chapter 1. Introduction

- Introduction

Introduction

HyperFlex Stretched Cluster enables you to deploy an Active-Active disaster avoidance solution for mission critical workloads requiring high uptime (near zero Recovery Time Objective) and no data loss (zero Recovery Point Objective). A stretched cluster is a single cluster with geographically distributed nodes. Both sides of the cluster act as primary for certain user VMs. The data for these VMs is replicated synchronously on the other site. Stretched clusters enable you to access the entire cluster even if one of the sites were to completely go down. Typically these sites are connected with a low latency, dedicated, high-speed link between them.

This document describes the deployment process for stretched clusters, gives information about monitoring stretched clusters, and discusses guidelines to follow when deploying stretched clusters.

Chapter 2. Preinstallation Checklist

- Preinstallation Checklist

- Stretch Witness

- Deploying the Witness Node

- Changing the Witness VM Password

- Network IP Addressing

Preinstallation Checklist

Network Requirements

- 10 Gbps dedicated, 5-ms Round-Trip Time (RTT) latency between the two active sites is required.

- 100 Mbps, 200 ms-RTT worst case latency for 16 kilobyte packet sizes between the active sites and witness site is required.

- Existing fabric interconnects are supported, provided the fabric interconnects support M5 servers.

- User VMs should be capable of vMotioning to any site, without impacting external network connectivity to these VMs.

- Similar to regular HX Clusters, two separate IP subnets are required—Both over Stretch L2. One subnet for data traffic and one for management traffic, with the management subnet reachable from vCenter and the witness node.

- Static IP address assigned to Witness VM, can be changed only with cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM.

Network Topology

Witness Requirements

- Configure and enable NTP on all servers.

- An independent third witness site is required.

- A separate witness is required for each cluster.

- Both the main sites must have connectivity to the third witness site with a minimum bandwidth of 100 MBPS, 200 ms-RTT worst case latency for 16 kilobyte packet sizes.

- Site must have the capability to deploy and run Open Virtualization Format (OVF) image.

Network Latency Requirements for the Witness VM

- The HyperFlex Stretched Cluster solution requires that the Witness VM be located in a third site to ensure that a storage site failure does not affect the Witness VM.

- The solution can support a Witness bandwidth as low as 100 Mbps, 200 ms-RTT worst case latency for 16 kilobyte packet sizes.

- Latency to the witness impacts site failure times, and it is recommended that for larger clusters with significant load and data, to have RTT times in the order of 10ms or lower.

Fabric Interconnect and Node Requirements

- Symmetric configuration is required across both sites.

- There must be a minimum of two converged nodes on each site.

- A maximum of 16 converged nodes on each site is supported. Ensure that both sites have the same number of converged nodes.

- There must be a redundant fabric interconnect configuration on each site.

- Converged nodes have to be M5 nodes.

- Ensure that the Fabric Interconnect pair is of the same model in the same domain.

- Compute-only nodes are supported.

VMware Requirements

- VMware Enterprise Plus edition with HyperFlex Stretched Cluster is highly recommended to ensure proper failover behavior and guarantee high performance during normal operations. While it is possible to run Stretched Cluster without VMware Enterprise Plus edition, the advanced DRS features such as site affinity will not be available, negating some of the intended operational functions of Stretched Cluster.

- Use a single vCenter for both sites.

- The vCenter can be a VM running at the same site as the witness.

- Nested vCenter is not supported in Stretched Cluster.

- The vCenter must be configured independently for High Availability, as required.

UCS Manager Requirements

- Two separate, manually synchronized fabric interconnect domains are required.

VLAN Requirements

- IP addresses for nodes on both sites are required.

- Stretched VLANs across both sites are required.

Stretch Witness

Required only when deploying HyperFlex Stretched Cluster.

Port Number

Service/Protocol

Source

Port Destinations

Essential Information

2181

2888

3888

Zookeeper/TCP

Witness

Each CVM Node

Bidirectional, management addresses

8180

Exhibitor (Zookeeper lifecycle)/TCP

Witness

Each CVM Node

Bidirectional, management addresses

80

HTTP/TCP

Witness

Each CVM Node

Potential future requirement

443

HTTPS/TCP

Witness

Each CVM Node

Potential future requirement

Deploying the Witness Node

Important

- A HyperFlex witness node is mandatory in a stretched cluster environment to achieve quorum in case of total failure in any of the sites or when the network link between the sites encounters a failure.

- In case of accidental deletion or loss of Witness VM, to replace the Witness VM please contact Cisco TAC.

- The Witness VM requires a Static IP address that cannot be changed without cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM.

Note

If the IP address is requested from a DHCP server please verify that the clustering service is running after deploying the Witness VM, perform the following steps:

- Check for the clustering service by running the following command after every reboot of the Witness VM:

service exhibitor status

- If the output does not show the service as running, then restart exhibitor using the following command after ensuring that the witness does have an IP address:

service exhibitor restart

The following procedure details the steps to follow, for deploying a witness virtual node on a physical ESXi host.

Before you begin

Attention

- HyperFlex Witness node version 1.0.2 is supported from 3.5(1a) and later releases.

- The Witness VM must be deployed on an ESXi server that has sufficient hosting capability. The Witness VM requires 4 vCPUs, 8GB of memory, and 40 GB of disk space.

- Ensure that the virtual network on this ESXi host is reachable from both the stretched cluster sites.

- Download the HyperFlex witness node on to your desktop or host that runs vSphere Web Client from Download Software.

Example:
HyperFlex-Witness-1.0.2.ova

- High Availability is optional for the witness node.

Procedure

Step 1

Log in to vSphere Web Client. Choose the ESXi server where the witness should be deployed. Right-click the ESXi host and select Deploy OVF Template.

Step 2

Browse and select the HyperFlex-Witness.ova file. Click Next.

Step 3

Specify a unique name for the witness node in the Virtual Machine Name field. Select a location for the virtual machine from the drop-down list. Click Next.

Step 4

From the Select a compute resource drop-down list, choose the ESXi host where you want to deploy the witness node. Click Next.

Step 5

In the Review details pane, verify the template details. Click Next.

Step 6

In the Select Storage pane, do the following:

Field

Description

Select virtual disk format drop-down list

- Thick Provision Lazy Zeroed

- Thick Provision Eager Zeroed

- Thin Provision

VM Storage Policy drop-down list

Datastore Default

Select the datastore where the virtual machine will reside. Ensure that this datastore has at least 40 GB of available free space. Click Next.

Step 7

In the Select Networks pane, select a Destination Network port group, where the witness VM has to connect. Click Next.

Step 8

On the Customize Template page, complete the fields that are relevant for your configuration. If no values are entered, the VM uses DHCP server provided network configuration parameters.

Field

Description

Static IP Address field

The IP address for Witness VM.

The Static IP address can be changed only with cluster redeployment. If DHCP server is used to define the network configuration, the IP address needs to be dedicated to Witness VM.

Leave blank if DHCP is desired.

Netmask field

The netmask or prefix for this interface.

Leave blank if DHCP is desired.

Default Gateway field

The default gateway address for this VM.

Leave blank if DHCP is desired.

DNS field

The domain name servers for this VM (comma separated).

Leave blank if DHCP is desired.

NTP field

NTP servers for this VM (comma separated) to sync time.

Leave blank if DHCP is desired.

Click Next.

Step 9

On the Ready to complete page, verify all the details entered. Click Finish.

Step 10

Repeat this process for each cluster.

What to do next

After successfully deploying the witness node, you can proceed to installing your stretched cluster. When prompted enter the IP address of this witness node on the IP Address page when creating a HyperFlex stretched cluster. The witness node is automatically used during configuration of the stretched cluster.

Changing the Witness VM Password

After successful deployment of witness VM, you must change the default password.

Before you begin

Download and deploy the witness VM.

Procedure

Step 1

Log in to the witness VM using SSH.

$ ssh root@<IP address of witness VM>

Step 2

Enter the default password.

Step 3

Enter the passwd command and change the password.

$ passwd
Changing password for user admin.

Step 4

Log out from the witness VM.

What to do next

Log in to the witness VM with the new password.

Network IP Addressing

IP addresses for HyperFlex Stretched Cluster need to be allocated from the appropriate subnets and VLANs to be used.

Important

- Ensure that the Data and Management Networks are on different subnets for a successful installation.

- Ensure that the IP addressing is in the same subnet for the same components (Management Network IP addresses, Data Network IP addresses).

Example: Stretched Cluster IP Addressing

Storage Cluster Management IP address

10.10.10.128

Storage Cluster Data IP address

192.168.10.160

Subnet mask IP address

255.255.255.0

Subnet mask IP address

255.255.255.0

Default gateway IP address

10.10.10.1

Default gateway IP address

192.168.10.1

Example: Stretched Cluster IP Addressing for Site A

Management Network IP Addresses

(must be routable)

Data Network IP Addresses

(does not have to be routable)

ESXi Hostname*

Hypervisor Management Network

Storage Controller Management Network

Hypervisor Data Network (Not Required for Cisco Intersight)

Storage Controller Data Network (Not Required for Cisco Intersight)

Server 1:

10.10.10.2

10.10.10.32

192.168.10.2

192.168.10.32

Server 2:

10.10.10.3

10.10.10.33

192.168.10.3

192.168.10.33

Server 3:

10.10.10.4

10.10.10.34

192.168.10.4

192.168.10.3

Server 4:

10.10.10.5

10.10.10.35

192.168.10.5

192.168.10.35

Server 5:

10.10.10.6

10.10.10.36

192.168.10.6

192.168.10.36

Example: Stretched Cluster IP Addressing for Site B

Management Network IP Addresses

(must be routable)

Data Network IP Addresses

(does not have to be routable)

ESXi Hostname*

Hypervisor Management Network

Storage Controller Management Network

Hypervisor Data Network (Not Required for Cisco Intersight)

Storage Controller Data Network (Not Required for Cisco Intersight)

Server 1:

10.10.10.64

10.10.10.96

192.168.10.64

192.168.10.96

Server 2:

10.10.10.65

10.10.10.97

192.168.10.65

192.168.10.97

Server 3:

10.10.10.66

10.10.10.98

192.168.10.66

192.168.10.98

Server 4:

10.10.10.67

10.10.10.99

192.168.10.67

192.168.10.99

Server 5:

10.10.10.68

10.10.10.100

192.168.10.68

192.168.10.100

Chapter 3. Guidelines and Limitations

- Guidelines

- Limitations

Guidelines

Consider the following guidelines when creating a HyperFlex Stretched Cluster:

- HXDP Enterprise Edition licensing is required to use the HyperFlex Stretched Cluster feature.

- vSphere Enterprise Plus licensing is required for full featured HyperFlex Stretched Cluster functionalities like VM load balancing and VM migration.

- Ensure that DRS is enabled.

- Supported Stretched Cluster scale:

- There must be a minimum of 2 nodes on each site. The minimum overall cluster size across both sites is 4.

- Small Form Factor (SFF). The maximum cluster size across both sites is 64. The maximum converged node count is 16 per site. The compute to converged node ratio can be 2:1, while the maximum limit of nodes per site must not exceed 32. For example, you can have 11 converged and 21 compute nodes per site.

- Large Form Factor (LFF). The maximum cluster size is 48. The maximum converged node count is 8 per site. The compute to converged node ratio can be 2:1, while the maximum limit of nodes per site must not exceed 24. For example, you can have 8 converged and 16 compute nodes per site.

- There must be a redundant fabric interconnect configuration on each site.

- There must be symmetric cluster configuration across both sites. The number of nodes and the model of HX nodes should be the same on both sites.

- VMs are placed correctly within site affinity only if VMware HA and DRS are enabled before VM creation. Otherwise, the correct placement of VM for proper affinity is not guaranteed.

- HyperFlex Native Replication is supported between Stretched Clusters, and between Stretched Clusters and standard clusters.

- From HyperFlex release 3.0, the Stretched Clusters running on HyperFlex release 3.0 or later can be upgraded to the current release. For more details, see Stretched Clusters Upgrade.

- It is a best practice to have two datastores one per site, with the respective affinity.

Limitations

Consider the following limitations when creating a HyperFlex Stretched Cluster:

- Self Encrypting Drives (SEDs) are not supported. However, VM based third-party software encryption is supported.

- Overlay networking and L3 protocols are not supported. L2 adjacency is required for the data and management networks.

- Auto failback of VMs after site failure is not supported. You must manually migrate the failover VMs to the site.

- Stretched Cluster is not supported on Hyper-V platform.

- Online rolling upgrades are supported only for the HX Data Platform. Cisco UCS Manager upgrades and VMware ESXi upgrades must be performed manually on one node at a time or performed offline.

- Upgrade from standalone cluster to Stretched Cluster configuration is not supported.

- Stretched Cluster is supported only on M5 nodes. M4/M5 mixed cluster is not supported.

- All NVMe converged nodes are not supported.

- Hardware Acceleration Engine (HX-PCIE-OFFLOAD-1) is not supported.

- Shared Witness VM is not supported for Stretched Cluster deployments.

Chapter 4. Installation

- Installation Overview

- Create the Stretched Cluster Sites

- Create Your HyperFlex Stretched Cluster

- Configuring VMware vCenter High Availability Settings for Stretched Cluster

Installation Overview

The following installation workflow summarizes the steps that are involved in creating a Stretched Cluster, using the HX Data Platform Installer.

Follow this workflow during installation:

Step

Description

Reference

Create the Stretched Cluster sites—Run the Configure Site workflow for Site A and Site B.

Log in to the HX Data Platform Installer. Enter UCS Manager credentials and Hypervisor credentials for both the sites.

Enter Credentials

Configure the server ports and associate HyperFlex servers.

Associate HyperFlex Servers

Configure VLAN, MAC Pool, 'hx-ext-mgmt' IP Pool for Out-of-Band CIMC, inband CIMC, iSCSi Storage, and FC Storage.

Configure UCS Manager

Configure the Hypervisor.

Configure Hypervisor

Download and deploy the Witness VM.

Note

A witness VM is mandatory in a HyperFlex Stretched Cluster environment.

Deploying the Witness Node

Create your HyperFlex Stretched Cluster—Run the Create Stretch Cluster workflow.

Enter UCS Manager Credentials for Site A and Site B, and vCenter credentials.

Enter Credentials

Configure the server ports and associate HyperFlex servers.

Associate HyperFlex Servers

Configure IP addresses.

Configure IP Addresses

Deploy the HyperFlex Stretched Cluster.

Deploy a HyperFlex Stretched Cluster

Create the Stretched Cluster Sites

Before you begin

- Complete the prerequisites. Refer to the Preinstallation Checklist for more details.

- Download and deploy the Witness VM for each cluster.

Use the following procedure to install and configure a Cisco HyperFlex Stretched Cluster using the Cisco HX Data Platform Installer.

Procedure

Step 1

Log in to the Cisco HX Data Platform Installer.

Step 2

On the Workflow page, from the Create Cluster drop-down list, select Stretch Cluster. Click Continue.

Step 3

On the Credentials page, enter UCS Manager and Hypervisor credentials. See Enter Credentials for more details.

Step 4

On the Server Selection page, configure the server ports and associate HyperFlex servers. See Associate HyperFlex Servers for more details.

Step 5

On the UCSM Configuration page, configure VLAN, MAC Pool, 'hx-ext-mgmt' IP Pool for Out-of-Band CIMC, iSCSi Storage, and FC Storage. See Configure UCS Manager for more details.

Step 6

On the Hypervisor Configuration page, configure the subnet mask, gateway, and complete the hypervisor settings. See Configure Hypervisor for more details.

Step 7

Click Configure Site to begin cluster creation. The Progress page displays the progress of various configuration tasks.

Caution

Do not skip validation warnings.

See the Warnings section for more details.

What to do next

Using the HX Data Platform Installer, proceed to create your HyperFlex stretch cluster. See Create Your HyperFlex Stretched Cluster for more details.

- Enter Credentials

- Associate HyperFlex Servers

- Configure UCS Manager

- Configure Hypervisor

Enter Credentials

Enter UCS Manager credentials and hypervisor credentials for both the sites across the Stretched Cluster.

Before you begin

- Complete the installation prerequisites. Refer to the Preinstallation Checklist for more details.

- Download and deploy the Witness VM.

Procedure

Step 1

Log in to the HX Data Platform Installer with root user credentials. For first time log in, you will be prompted to change the default password set in the factory.

1. In a browser, enter the URL for the VM where HX Data Platform Installer was installed.

2. Enter the following login credentials.

- Username: root

- Password: Cisco123

3. Read the EULA, select the I accept the terms and conditions checkbox, and click Login.

Step 2

In the Change factory default password screen, complete the following fields, and click Change Password & Login.

Field

Description

New password

Enter a new hypervisor password.

Re-enter new password

Re-enter the new hypervisor password.

Step 3

On the Workflow page, from the Create Cluster drop-down list, select Stretch Cluster. Click Continue.

Step 4

On the Credentials page, select Configure Site.

To perform cluster creation, you can import a JSON configuration file with the required configuration data. The following two steps are optional if importing a JSON file, otherwise you can input data into the required fields manually.

Note

For a first-time installation, contact your Cisco representative to procure the factory preinstallation JSON file.

1. Click Select a file and choose your JSON file to load the configuration. Select Use Configuration.

2. An Overwrite Imported Values dialog box displays if your imported values for Cisco UCS Manager are different. Select Use Discovered Values.

Step 5

Enter the following credentials for UCS Manager:

UCS Manager Credentials for this site

Field

Description

UCS Manager Host Name field

Enter the UCS Manager FQDN or IP address.

For example, 10.193.211.120.

UCS Manager User Name field

Enter the administrative level username.

For example, <admin>.

Password field

Enter the administrative level password.

For example, <root>.

Site Name field

Specify a unique site name.

Step 6

Enter the following credentials for the Hypervisor:

Hypervisor Credentials

Field

Description

Admin User Name field

Enter the administrative username.

For example, <admin> username.

The username is root for factory nodes.

The hypervisor on this node uses the factory default password check box

Select the check box, if you are changing the factory default password for the hypervisor.

To enter the current hypervisor password, uncheck this check box. You can now enter the hypervisor password in the Enter current hypervisor password field.

New Password

Create a new password for the hypervisor.

Important

You are required to change the factory default password.

Confirm New Password

Reenter the new password for the hypervisor.

Step 7

Click Continue to begin associating HyperFlex Servers. See Associate HyperFlex Servers.

Associate HyperFlex Servers

On the Server Selection page, the Configuration pane on the right displays a detailed list of the Credentials used. The Server Selection page displays a list of unassociated HX servers under the Unassociated tab, and the list of discovered servers under the Associated tab.

Field

Description

Locator LED column

Turn on to locate a server.

Server Name column

Name that is assigned to the server.

Status column

- Inaccessible

- Ok

Model column

Displays the server model.

Serial column

Displays the serial number of the server.

Service Profile column [Only for Associated Servers]

Service profile that is assigned to the server.

Actions drop-down list

- Launch KVM Console—Choose this option to launch the KVM Console directly from the HX Data Platform Installer.

- Disassociate Server—Choose this option to remove a service profile from that server.

Before you begin

Ensure that you completed entering UCS Manager, vCenter, and Hypervisor credentials on the Credentials page. See Enter Credentials.

Procedure

Step 1

Click the Configure Server Ports button to discover any new HX nodes. In the Configure Server Ports dialog box, list all ports to be configured as server ports. Click Configure.

Note

Typically, the server ports are configured in Cisco UCS Manager before you start the configuration.

Step 2

Select the servers under the Unassociated tab to include in the HyperFlex cluster.

If HX servers do not appear in this list, check Cisco UCS Manager and ensure that they have been discovered.

Note

If there are no unassociated servers, the following error message is displayed:

No unassociated servers found. Login to UCS Manager and ensure server ports are enabled.

Step 3

Click Continue to configure UCS Manager. See Configure UCS Manager .

Configure UCS Manager

On the UCSM Configuration page, you can configure VLAN, MAC Pool, 'hx-ext-mgmt' IP Pool for Out-of-Band CIMC, iSCSi Storage, and FC Storage.

Before you begin

Associate servers on the HyperFlex cluster. See Associate HyperFlex Servers.

Procedure

Step 1

In the VLAN Configuration section, complete the following fields:

Note

Use separate subnet and VLANs for each of the following networks.

Field

Description

VLAN for Hypervisor and HyperFlex management

VLAN Name field

hx-inband-mgmt

VLAN ID field

Default—3091

VLAN for HyperFlex storage traffic

VLAN Name field

hx-storage-data

VLAN ID field

Default—3092

VLAN for VM vMotion

VLAN Name field

hx-vmotion

VLAN ID field

Default—3093

VLAN for VM Network

VLAN Name field

vm-network

VLAN ID(s) field

Default—3094

A comma-separated list of guest VLANs.

Step 2

In the MAC Pool section, configure the MAC Pool Prefix by adding in two more hex characters (0-F).

Note

Select a prefix that is not used with any other MAC address pool across all UCS domains.

Example:
00:25:B5:A0.

Step 3

In the 'hx-ext-mgmt' IP Pool for Out-of-Band CIMC section, complete the following fields:

Field

Description

IP Blocks field

The range of IP addresses designated for the HyperFlex nodes. The IP addresses can be a comma-separated list of values for the guest VLANs.

For example, 10.193.211.124-127, 10.193.211.158-163.

Subnet Mask field

Set the subnet to the appropriate level to limit and control IP addresses.

For example, 255.255.0.0.

Gateway field

Enter the IP address.

For example, 10.193.0.1.

Step 4

If you want to add external storage, configure iSCSI Storage by completing the following fields:

Field

Description

Enable iSCSI Storage check box

Select to configure iSCSI storage.

VLAN A Name field

Name of the VLAN associated with the iSCSI vNIC, on the primary fabric interconnect (FI-A).

VLAN A ID field

ID of the VLAN associated with the iSCSI vNIC, on the primary fabric interconnect (FI-A).

VLAN B Name field

Name of the VLAN associated with the iSCSI vNIC, on the subordinate fabric interconnect (FI-B).

VLAN B ID field

ID of the VLAN associated with the iSCSI vNIC, on the subordinate fabric interconnect (FI-A).

Step 5

If you want to add external storage, configure FC Storage by completing the following fields:

Field

Description

Enable FC Storage check box

Select to enable FC Storage.

WWxN Pool field

A WWN pool that contains both WW node names and WW port names. For each fabric interconnect, a WWxN pool is created for WWPN and WWNN.

VSAN A Name field

The name of the VSAN for the primary fabric interconnect (FI-A).

Default—hx-ext-storage-fc-a.

VSAN A ID field

The unique identifier assigned to the network for the primary fabric interconnect (FI-A).

Caution

Do not enter VSAN IDs that are currently used on the UCS or HyperFlex system. If you enter an existing VSAN ID in the installer which utilizes UCS zoning, zoning will be disabled in your existing environment for that VSAN ID.

VSAN B Name field

The name of the VSAN for the subordinate fabric interconnect (FI-B).

Default—hx-ext-storage-fc-b.

VSAN B ID field

The unique identifier assigned to the network for the subordinate fabric interconnect (FI-B).

Caution

Do not enter VSAN IDs that are currently used on the UCS or HyperFlex system. If you enter an existing VSAN ID in the installer which utilizes UCS zoning, zoning will be disabled in your existing environment for that VSAN ID.

Step 6

In the Advanced section, do the following:

Field

Description

UCS Server Firmware Version drop-down list

Select the UCS server firmware version to associate with the HX servers from the drop-down list. The UCS firmware version must match the UCSM version. See the latest Cisco HX Data Platform Release Notes for more details.

For example, 3.2(1d).

HyperFlex Cluster Name field

Specify a user-defined name. The HyperFlex cluster name is applied to a group of HX Servers in a given cluster. The HyperFlex cluster name adds a label to service profiles for easier identification.

Org Name field

Specify a unique Org Name to ensure isolation of the HyperFlex environment from the rest of the UCS domain.

Step 7

Click Continue to configure the Hypervisor. See Configure Hypervisor.

Configure Hypervisor

Note

Review the VLAN, MAC pool, and IP address pool information on the Hypervisor Configuration page, in the Configuration pane. These VLAN IDs may be changed by your environment. By default, the HX Data Platform Installer sets the VLANs as non-native. Configure the upstream switches to accommodate the non-native VLANs by appropriately applying a trunk configuration.

Attention

You can skip configuring Hypervisor in case of a reinstall, if ESXi networking has been completed.

Before you begin

Configure VLAN, MAC Pool, and 'hx-ext-mgmt' IP Pool for Out-of-Band CIMC. If you are adding external storage, configure iSCSI Storage and FC Storage. Select the UCS Server Firmware Version and assign a name for the HyperFlex Stretched Cluster. See Configure UCS Manager.

Procedure

Step 1

In the Configure Common Hypervisor Settings section, complete the following fields:

Field

Description

Subnet Mask field

Set the subnet mask to the appropriate level to limit and control IP addresses.

For example, 255.255.0.0.

Gateway field

IP address of gateway.

For example, 10.193.0.1.

DNS Server(s) field

IP address for the DNS Server.

Note

- If you do not have a DNS server, do not enter a hostname in any of the fields on the Cluster Configuration page of the HX Data Platform Installer. Use only static IP addresses and hostnames for all ESXi hosts.

- If you are providing more than one DNS server, check carefully to ensure that both DNS servers are correctly entered, separated by a comma.

Step 2

On the Hypervisor Settings section, select Make IP Addresses and Hostnames Sequential to make the IP addresses sequential. Complete the following fields:

Note

You can rearrange the servers using drag and drop.

Field

Description

Name column

Name assigned to the server.

Locator LED column

Turn on to locate a server.

Serial column

Displays the serial number of the server.

Static IP Address column

Input static IP addresses and hostnames for all ESXi hosts.

Hostname column

Do not leave the hostname fields empty.

Step 3

Click Configure Site to begin cluster creation. The Progress page displays the progress of various configuration tasks.

Caution

Do not skip validation warnings.

See the Warnings section for more details.

What to do next

Run the Create Stretch Cluster workflow from the HX Data Platform Installer to create your HyperFlex Stretched Cluster. See Create Your HyperFlex Stretched Cluster for more details.

Create Your HyperFlex Stretched Cluster

Before you begin

- Complete the prerequisites. Refer to the Preinstallation Checklist for more details.

- Ensure that you run the Configure Site workflow for both sites.

- Download and deploy the Witness VM.

Use the following procedure to create a Cisco HyperFlex Stretched Cluster using the Cisco HX Data Platform Installer.

Procedure

Step 1

Log in to the Cisco HX Data Platform Installer.

Step 2

On the Workflow page, from the Create Cluster drop-down list, select Stretch Cluster. Click Continue.

Step 3

On the Credentials page, select Create Stretch Cluster. Enter UCS Manager credentials for site 1 and site 2. Enter vCenter credentials. Click Continue. See Enter Credentials for more details.

Step 4

On the Server Selection page, configure the server ports and associate HyperFlex servers. See Associate HyperFlex Servers for more details.

Step 5

On the IP Addresses page, configure the IP addresses to be applied to the storage cluster. See Configure IP Addresses for more details.

Step 6

On the Cluster Configuration page, configure and deploy the HyperFlex stretched cluster. See Deploy a HyperFlex Stretched Cluster for more details.

Step 7

Click Start to begin deploying the stretched cluster. The Progress page displays the progress of the various configuration tasks.

What to do next

Click Launch HyperFlex Connect to manage your HyperFlex Stretched Cluster.

- Enter Credentials

- Associate HyperFlex Servers

- Configure IP Addresses

- Deploy a HyperFlex Stretched Cluster

Enter Credentials

Enter UCS manager credentials for site 1 and site 2, and enter credentials for the vCenter.

Before you begin

- Complete the prerequisites. Refer to the Preinstallation Checklist for more details.

- Download and deploy the Witness VM.

- Complete configuring the HyperFlex Stretched Cluster. See Create the Stretched Cluster Sites for more details.

Procedure

Step 1

Log in to the Cisco HX Data Platform Installer.

1. In your web browser, enter the IP address or the node name for the HX Data Platform Installer VM. Click Accept or Continue to bypass any SSL certificate errors. On the Cisco HX Data Platform Installer login page, verify the HX Data Platform Installer Build ID in the lower right corner of the login screen.

2. In the login page, enter the following credentials:

Username: root

Password (Default): Cisco123

Important

Systems ship with a default password of Cisco123 that must be changed during installation. You cannot continue installation unless you specify a new user supplied password.

3. Read the EULA, check the I accept the terms and conditions check box, and click Login.

Step 2

On the Workflow page, from the Create Cluster drop-down list, select Stretch Cluster. Click Continue.

Step 3

On the Credentials page, select Create Stretch Cluster.

Step 4

Enter the following UCS Manager credentials for Site 1 and Site 2.

To perform cluster creation, you can import a JSON configuration file with the required configuration data. The following two steps are optional if importing a JSON file, otherwise you can input data into the required fields manually.

Note

For a first-time installation, contact your Cisco representative to procure the factory preinstallation JSON file.

1. Click Select a file and choose your JSON file to load the configuration. Select Use Configuration.

2. An Overwrite Imported Values dialog box displays if your imported values for Cisco UCS Manager are different. Select Use Discovered Values.

UCS Manager Credentials for Site 1

Field

Description

UCS Manager Hostname field

UCS Manager FQDN or IP address for site 1.

For example, 10.193.211.120.

User Name field

Enter the administrative level username.

For example, <admin> username.

Password field

Enter the administrative level password.

Enter the <root> password.

Site Name field

Specify a unique site name.

Org Name field

Specify a unique org name, to ensure isolation of the HyperFlex environment from the rest of the UCS domain.

UCS Manager Credentials for Site 2

Field

Description

UCS Manager Host Name field

Enter the UCS Manager FQDN or IP address for site 2.

For example, 10.193.211.120.

User Name field

Enter the administrative level username.

For example, <admin> username.

Password field

Enter the administrative level password.

Enter the <root> password.

Site Name field

Specify a unique site name.

Org Name field

Specify a unique org name. This ensures isolation of the HyperFlex environment from the rest of the UCS domain.

Step 5

Enter the following credentials for the vCenter.

vCenter Credentials

Field

Description

vCenter Server field

Enter the vCenter server FQDN or IP address.

For example, 10.193.211.120.

Note

- vCenter Server input is optional if building a nested vCenter. See the Nested vCenter TechNote for more details. A vCenter server is required before the cluster can be made operational.

- The vCenter address and credentials must have root level administrator permissions to the vCenter.

User Name field

Enter the administrative username.

For example, administrator@vsphere.local.

Admin Password field

Enter the administrative level password.

Enter the <root> password.

Step 6

Click Continue to begin associating HyperFlex servers. See Associate HyperFlex Servers.

Associate HyperFlex Servers

On the Server Selection page, the Configuration pane on the right displays a detailed list of the Credentials used. The Server Selection page displays a list of unassociated HX servers under the Unassociated tab, and the list of discovered servers under the Associated tab.

Field

Description

Locator LED column

Turn on to locate a server.

Server Name column

Name that is assigned to the server.

Status column

- Inaccessible—

- Ok—

Model column

Displays the server model.

Serial column

Displays the serial number of the server.

Service Profile column [Only for Associated Servers]

Service profile that is assigned to the server.

Actions column

- Launch KVM Console—Choose this option to launch the KVM Console directly from the HX Data Platform Installer.

- Disassociate Server—Choose this option to remove a service profile from that server.

Before you begin

Ensure that you completed entering UCS Manager, vCenter, and Hypervisor credentials on the Credentials page. See Enter Credentials.

Procedure

Step 1

Click the Configure Server Ports button to discover any new HX nodes. In the Configure Server Ports dialog box, list all ports to be configured as server ports. Click Configure.

Note

Typically, the server ports are configured in Cisco UCS Manager before you start the configuration.

Step 2

Select the servers under the Unassociated tab to include in the HyperFlex cluster.

If HX servers do not appear in this list, check Cisco UCS Manager and ensure that they have been discovered.

Note

If there are no unassociated servers, the following error message is displayed:

No unassociated servers found. Login to UCS Manager and ensure server ports are enabled.

Step 3

Click Continue to configure IP addresses. See Configure IP Addresses.

Configure IP Addresses

Before you begin

Associate servers on the HyperFlex cluster. See Associate HyperFlex Servers.

Procedure

Step 1

On the IP Addresses page, select Make IP Addresses Sequential to make the IP Addresses sequential.

Step 2

When you enter IP addresses in the first row for Hypervisor, Storage Controller (Management) and Hypervisor, Storage Controller (Data) columns, the HX Data Platform Installer incrementally autofills the node information for the remaining nodes.

For each HX node, enter the Hypervisor and storage controller IP addresses. For the IP addresses, specify if the network belongs to the Data Network or the Management Network.

Field

Description

Management Hypervisor field

Enter the static IP address that handles the Hypervisor management network connection between the ESXi host and the storage cluster.

Management Storage Controller field

Enter the static IP address that handles the storage controller VM management network connection between the storage controller VM and the storage cluster.

Data Hypervisor field

Enter the static IP address that handles the Hypervisor data network connection between the ESXi host and the storage cluster.

Data Storage Controller field

Enter the static IP address that handles the storage controller VM data network connection between the storage controller VM and the storage cluster.

Step 3

The IP address provided here are applied to one node in the storage cluster. In the event the node becomes unavailable the affected IP address is moved to another node in the storage cluster. All nodes must have a port configured to accept these IP addresses.

Provide the following IP addresses:

Field

Description

Management Cluster Data IP Address field

Enter the management network IP address for the HX Data Platform storage cluster.

Data Cluster IP Address field

Enter the IP address of data network for the HX Data Platform storage cluster.

Management Subnet Mask field

Enter the subnet information for your VLAN and vSwitches.

Provide the management network value. For example, 255.255.255.0.

Data Subnet Mask field

Provide the network value for the data network. For example, 255.255.255.0.

Management Gateway field

Provide the network value for your management network. For example, 10.193.0.1.

Data Gateway field

Provide the network value for your data network. For example, 10.193.0.1.

Witness IP field

Provide the IP address of the witness VM.

Step 4

Click Continue to deploy the HyperFlex stretched cluster. See Deploy a HyperFlex Stretched Cluster for more details.

Deploy a HyperFlex Stretched Cluster

On the Cluster Configuration page, for the Cisco HX Storage Cluster complete the following fields to begin deploying the HyperFlex cluster.

Before you begin

Ensure that you completed configuring IP addresses on the IP Addresses page. See Configure IP Addresses.

Procedure

Step 1

In the Cisco HX Cluster section, complete the following fields:

Field

Description

Cluster Name field

Specify a name for the HX Data Platform storage cluster.

Replication Factor drop-down list

Specify the number of redundant replicas of your data across the storage cluster. Set the replication factor to 4, for 2 copies in each site.

Step 2

In the Controller VM section, create a new password for the Administrative User of the HyperFlex cluster.

A default administrator username and password is applied to the controller VMs. The VMs are installed on all converged and compute-only nodes.

Important

- You cannot change the name of the controller VM or the controller VM’s datastore.

- Use the same password for all controller VMs. The use of different passwords is not supported.

- Provide a complex password that includes 1 uppercase character, 1 lowercase character, 1 digit, 1 special character, and a minimum of 10 characters in total.

- You can provide a user-defined password for the controller VMs and for the HX cluster to be created. For password character and format limitations, see the section on Guidelines for HX Data Platform Special Characters in the Cisco HX Data Platform Management Guide.

Step 3

In the vCenter Configuration section, complete the following fields:

Field

Description

vCenter Datacenter Name field

Enter the vCenter datacenter name for the Cisco HyperFlex cluster.

vCenter Cluster Name field

Enter the vCenter cluster name.

Step 4

In the System Services section, complete the following fields:

Field

Description

DNS Server(s) field

Enter a comma-separated list of IP addresses of each DNS server.

NTP Server(s) field

Enter a comma-separated list of IP addresses of each NTP server.

Note

All hosts must use the same NTP server, for clock synchronization between services running on the storage controller VMs and ESXi hosts.

DNS Domain Name field

Enter the DNS FQDN or IP address.

Time Zone drop-down list

Select the local time zone for the controller VM, to determine when to take scheduled snapshots. Scheduled native snapshot actions are based on this setting.

Step 5

In the Auto Support section, select Enable Connected Services to enable Auto Support and Cisco Intersight management.

Field

Description

Enable Connected Services (Recommended) check box

Select to enable Auto Support and Cisco Intersight management. Log on to HX Connect to configure these services or selectively turn them On or Off.

Send service ticket notifications to field

Enter the email address where SR notifications should be sent when triggered by Auto Support.

Step 6

In the Advanced Networking section, complete the following fields:

Note

When repurposing existing hyperflex hardware for stretched cluster, the VM Network port group may not have configured with the correct vSwitch. You must manually configure the VM network on the correct vSwitch.

Field

Description

Management VLAN Tag - Site 1 field

Enter the correct VLAN ID if using trunk ports.

Enter 0 if using access ports.

Note

Use the same Management VLAN Tag for site 1 and site 2.

Management VLAN Tag - Site 2 field

Enter the correct VLAN ID if using trunk ports.

Enter 0 if using access ports.

Note

Use the same Management VLAN Tag for site 1 and site 2.

Management vSwitch field

Default is vswitch-hx-inband-mgmt.

Note

Do not modify the name of the vSwitch.

Data VLAN Tag - Site 1 field

Enter the correct VLAN ID if using trunk ports.

Enter 0 if using access ports.

Note

Use the same Data VLAN Tag for site 1 and site 2.

Data VLAN Tag - Site 2 field

Enter the correct VLAN ID if using trunk ports.

Enter 0 if using access ports.

Note

Use the same Data VLAN Tag for site 1 and site 2.

Data vSwitch field

Default is vswitch-hx-storage-data.

Step 7

In the Advanced Configuration section, do the following:

Field

Description

Jumbo frames

Enable Jumbo Frames check box

Check to set the MTU size for the storage data network on the host vSwitches and vNICs, and each storage controller VM.

The default value is 9000.

Note

At the time of installation, you can select the MTU size as either 1500 or 9000 bytes.

Note

If you want to change the MTU size from 9000 to 1500 after installation, you must change the MTU size in Cisco UCS, Data vSwitch, VMkernel, and Controller VM Ethernet1 interface.

Disk Partitions

Clean up Disk Partitions check box

Check to remove all existing data and partitions from all nodes added to the storage cluster for manually prepared servers. Select this option to delete existing data and partitions. You must backup any data that should be retained.

Attention

Do not select this option for factory prepared systems. The disk partitions on factory prepared systems are properly configured.

Virtual Desktop (VDI)

check box

Check for VDI only environments.

Note

To change the VDI settings after the storage cluster is created, shutdown or move the resources, make changes, and restart the cluster.

Step 8

Click Start to begin deploying the HyperFlex stretched cluster. The Progress page displays the progress of various configuration tasks.

Caution

Do not skip validation warnings.

See the Warnings section for more details.

What to do next

- Some validation errors require you to go back and re-enter a parameter (for example, an invalid ESXi password, incorrect NTP server, bad SSO server, or other incorrect input). Click Re-enter Values to return to the Cluster Configuration page and resolve the issue.

- When complete, the HyperFlex servers are installed and configured. The deployed cluster status shows as Online and Healthy.

- Click Launch HyperFlex Connect to manage your HyperFlex stretched cluster.

Configuring VMware vCenter High Availability Settings for Stretched Cluster

The HyperFlex Stretched Cluster works as designed using the default installation values. This section provides the recommended settings for configuring VMware vSphere high availability for Stretched Cluster during custom configuration.

Note

Use the default value for the fields for which the settimgs are not captured in this section.

Before you begin

Create the HyperFlex Stretched Cluster sites.

Procedure

Step 1

Log on to vCenter with the vSphere Web Client.

Step 2

Select the Stretched cluster and click Configure.

Step 3

Choose Configure > vSphere Availability and click Edit.

Step 4

Check the Turn on vSphere HA check box.

Step 5

Click Failures and Responses.

Step 6

In the Failure conditions and responses screen, complete the following fields:

- Enable Host Monitoring—Check this check box to enable the host monitoring.

- Host Failure Response—From the drop-down list, choose Restart VMs.

- Response for Host Isolation—From the drop-down list, choose Power off and Restart VMs.

- Datastore with PDL—From the drop-down list, choose Power off and Restart VMs.

- Datastore with APD—From the drop-down list, choose Power off and Restart VMs (conservative).

- VM Monitoring—From the drop-down list, choose the required option for VM monitoring. By default, Disabled is displayed.

Step 7

Click Admission Control. From the Define host failover capacity by drop-down list, choose Cluster resource percentage and check the Override calculated failover capacity check box. Change the set percentage of CPU and Memory to 50%.

Step 8

Click Heartbeat Datastores. In the Heartbeat Datastores screen, click the Use datastores only from the specified list radio button and choose the Hyperflex datastotres.

Step 9

Click Advanced Options, and add the following options:

Option

Value

das.usedefaultisolationaddress

False.

das.isolationaddress0

IP address of the eth0 interface of a storage controller on site A, IP address of the stretch cluster on complimentary site A, or virtual IP address of complimentary site A Fabric Interconnect (FI).

das.isolationaddress1

IP address of the eth0 interface of a storage controller on site B, IP address of the stretch cluster on complimentary site B, or virtual IP address of complimentary site B Fabric Interconnect (FI).

Step 10

Click OK.

Chapter 5. Managing HyperFlex Stretched Clusters

- Overview

- Monitoring the Health of a Stretched Cluster

- Viewing System Information

- Create Datastore

Overview

You can view do the following for the sites and the witness node in a stretched cluster using HX Connect:

- View the Operational Status and Resiliency Status of the HyperFlex cluster on both the sites and the witness node.

- View the Functional status of the sites and the witness node, and the IP address of the witness node. View HX storage cluster system-related information, including node and disk data.

- Enter/Exit HX maintenance mode.

- Associate a datastore with one of the sites in a stretched cluster, when creating the datastore.

Monitoring the Health of a Stretched Cluster

You can view the Operational Status and Resiliency Status of the HyperFlex cluster on both the sites and the witness node in HX Connect on the Dashboard page.

Important

If you are a read-only user, you may not see all the options available in the Help. To perform most actions in HX Connect, you must have administrative privileges.

Procedure

Step 1

Log in to HX Connect.

1. Enter the HX Storage Cluster management IP address in a browser. Navigate to https://<storage-cluster-management-ip>.

2. Enter the administrative username and password.

3. Click Login.

Step 2

On the left navigation pane, click Dashboard.

Step 3

On the Dashboard you can view the following details for the HyperFlex Stretched Cluster:

Displays a status summary of your HX storage cluster for the sites across the stretched cluster.

UI Element

Essential Information

Operational Status section

Provides the functional status and application performance of the HX storage cluster on Site A and Site B.

Provides the functional status and application performance of the witness node.

Click Information () to access the HX storage cluster name and status data.

Resiliency Health section

Provides the data health status and the ability of the HX storage cluster on Site A and Site B to tolerate failures.

Click Information () to access the resiliency status, replication and failure data. This also provides information about data replication compliance, caching device failures tolerable, and device failures tolerable on each node in both the sites.

Capacity section

Displays a breakdown of the total storage versus how much storage is used or free.

Also displays the storage optimization, compression-savings, and deduplication percentages based on the data stored in the cluster.

Nodes section

Displays the number of nodes and the division of converged versus compute nodes across Site A and Site B in the Stretched Cluster.

Hovering over a node icon displays that node's name, IP address, node type, and an interactive display of disks with access to capacity, usage, serial number, and disk type data.

Performance section

Displays an HX storage cluster performance snapshot for a configurable amount of time, showing IOPS, throughput, and latency data.

For full details, see Performance Page.

Cluster Time field

System date and time for the cluster.

Several tables in HX Connect provide one or more of the following three fields that affect the content displayed in the table.

UI Element

Essential Information

Refresh field and icon

The table automatically refreshes for dynamic updates to the HX Cluster. The timestamp indicates the last time the table was refreshed.

Click the circular icon to refresh the content now.

Filter field

Display in the table only list items that match the entered filter text. The items listed in the current page of the table are automatically filtered. Nested tables are not filtered.

Type in the selection text in the Filter field.

To empty the Filter field, click the x.

To export content from other pages in the table, scroll to the bottom, click through the page numbers, and apply the filter.

Export menu

Save out a copy of the current page of table data. The table content is downloaded to the local machine in the selected file type. If the listed items are filtered, the filtered subset list is exported.

Click the down arrow to select an export file type. The file type options are: cvs, xls, and doc.

To export content from other pages in the table, scroll to the bottom, click through the page numbers, and apply the export.

Viewing System Information

On the System Information page, you can view HX storage cluster system-related information, including node and disk data. You can also Enter or Exit Maintenance Mode for the sites.

Procedure

Step 1

Log in to HX Connect.

1. Enter the HX Storage Cluster management IP address in a browser. Navigate to https://<storage-cluster-management-ip>.

2. Enter the administrative username and password.

3. Click Login.

Step 2

On the left navigation pane, select System Information.

Step 3

Under the System Overview tab you can view the following information for both the sites and the witness node:

HX Storage Cluster Configuration Data tab

Displays the basic configuration information the HX storage cluster on the stretched cluster sites.

Field

Description

HX storage cluster field

Name of this storage cluster.

HX storage cluster status field

Provides functional status of the HX storage cluster in Site A and Site B:

- Online—Cluster is ready.

- Offline—Cluster is not ready.

- Read Only—Cluster is out of space.

- Unknown—Transitional state while the cluster is coming online.

vCenter link

Secure URL to the VMware vSphere associated with this HX storage cluster. Click the link to remotely access the vSphere Web Client.

Hypervisor field

Hypervisor version installed on this HX storage cluster.

HXDP Version field

Installer package version installed on this HX storage cluster.

Data Replication Factor field

Number of the redundant data replicas stored on this HX storage cluster.

Uptime field

Length of time this HX storage cluster has been online.

Total Capacity field

Overall storage size of this cluster.

Available Capacity field

Amount of free storage in this cluster.

DNS Server(s) field

IP address for the DNS server(s) for this HX storage cluster.

NTP Server(s) field

IP address for the NTP server(s) for this HX storage cluster.

Witness IP Address field

Provides the IP address of the Witness VM.

Step 4

Under the Nodes tab, you can view the following information:

Displays data about individual nodes in this HX storage cluster. To see this information in tabular format, go to the Nodes page.

UI Element

Essential Information

Node field

Name of a node on this cluster.

Model field

Physical hardware model number of this node.

Disks field

Number of caching versus persistent disks in this node.

Node status field

- Online

- Offline

- In Maintenance

- Healthy

- Warning

HXDP Version field

Installer package version installed on this node.

Hypervisor Status field

- Online

- Offline

- In Maintenance

- In Progress

Hypervisor Address field

IP address for the management network for this HX storage cluster.

Controller Address field

IP address of the controller VM on this HX storage cluster.

Controller Status field

Status of the controller VM on this HX storage cluster.

Step 5

Under the Disks tab, you can view the following information:

For nodes with disks, an interactive display of disks is included with the following pop-up data:

Caching Disks

UI Element

Essential Information

Slot Number field

Location of the drive.

Serial Number field

Physical serial number of this disk.

Disk State field

- Ready

Capacity field

Total disk size.

Storage Usage field

Percentage of disk storage used.

Locator LED action

Activates a physical light on the host to help locate a disk; options are On and Off.

Persistent Disks

UI Element

Essential Information

Slot Number field

Location of the drive.

Serial Number field

Physical serial number of this disk.

Disk State field

- Ready

- Blacklisted

- To Be Removed

Used / Total Capacity field

Amount of the disk used versus the total disk size.

Storage Usage field

Percentage of disk storage used.

Locator LED action

Activates a physical light on the host to help locate a disk; options are On and Off.

Create Datastore

To associate a datastore with one of the sites in a stretched cluster, do the following:

Procedure

Step 1

Log in to HX Connect.

1. Enter the HX Storage Cluster management IP address in a browser. Navigate to https://<storage-cluster-management-ip>.

2. Enter the administrative username and password.

3. Click Login.

Step 2

On the left navigation page, click Datastores.

Step 3

In the work pane, click Create Datastore.

Step 4

Enter a datastore name and capacity.

UI Element

Essential Information

Datastore Name field

Enter a unique datastore name for this HX Storage Cluster.

Size field

Enter the quantity of the datastore.

Select the unit of measure. Options are: GB and TB.

Ensure it is sufficient to support the virtual machines in this HX Storage Cluster.

Block Size

Select a block size.

- 8K—Default

- 4K

Site Affinity

Choose a site from the drop-down list to associate the datastore with the site.

Step 5

Click Create Datastore.

HX Data Platform creates a datastore and mounts it on every node in this HX Storage Cluster.

Chapter 6. Troubleshooting

- Viewing Resiliency Status in HX Connect

- Troubleshooting the Network Configuration

- Increasing Datastore Capacity May Not Show as Free Space

- Troubleshooting for Site-to-Site Failover

Viewing Resiliency Status in HX Connect

Independent and Non-Simultaneous Failure Scenarios

The Dashboard page in HX Connect displays the status summary of your HX storage cluster. Click Information () to access the resiliency status. Resiliency status is the data resiliency health status and the ability of the storage cluster to tolerate failures.

To access the resiliency status in HX Connect:

1. Log in to HX Connect.

1. Enter the HX Storage Cluster management IP address in a browser. Navigate to https://<storage-cluster-management-ip>.

2. Enter the administrative username and password. Click Login.

2. On the left navigation pane, click Dashboard.

3. On the Dashboard page, you can view the resiliency status for the HyperFlex Stretched Cluster:

Color coding and icons are used to indicate various status states. Click an icon to display additional information, such as reason messages that explain what is contributing to the current state. The various Resiliency Status states are:

- Healthy—The cluster is healthy with respect to data and availability.

- Warning—Either the data or the cluster availability is being adversely affected.

- Unknown—A transitional state while the cluster is coming online.

Important

The disk failures that are mentioned below are for the converged nodes only (the witness node and the converged nodes do not have disks hosting user data).

Failure Scenario

Expected Behavior

Resiliency Status in HX Connect

One node failure

VMs will failover to the remaining nodes on the same site as long as those nodes can accommodate the VM resources.

Warning—Cluster shows status of unhealthy until the cluster recovers.

All node failure on any one site

User VMs failover to other site.

Note

For the recovery operation to succeed after a site failure, the witness node must be online.

Warning—HX Connect shows the site failure details.

Cluster shows status of unhealthy until the cluster recovers.

One disk failure

Cluster recovers after failover.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Two disk failure on a single site (one disk on each node) simultaneously

Same as one disk failure.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Same as one disk failure.

Witness failure

Cluster remains online.

No visible indication.

vCenter failure (platform impact)

Cluster remains online.

No visible indication.

Network isolation between a given site and a witness

Cluster remains online.

No visible indication.

Network isolation between sites

All user VMs from one site will fail over to the other site. The VMs from the other site will continue to run.

Note

For the recovery operation to succeed after a site failure, the witness node must be online.

Warning—HX Connect shows the site failure details.

Cluster shows status of unhealthy until the cluster recovers.

Failure Scenarios with Witness Failure

Failure Scenario

Expected Behavior

Visible Indications in HX Connect

Disk failure

Cluster remains online.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Single node failure

Cluster remains online.

Cluster becomes unhealthy.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Link failure

Cluster halts resulting in All Path Down on both sites, until either the second failure is healed or the witness is restored.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Either fabric interconnect pair failure

Cluster halts resulting in All Path Down on both sites, until either the second failure is healed or the witness is restored.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Switch failure

Cluster halts resulting in All Path Down on both sites, until either the second failure is healed or the witness is restored.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Site power failure

Cluster halts resulting in All Path Down on both sites, until either the second failure is healed or the witness is restored.

Warning—Cluster shows status of unhealthy until the cluster recovers.

Troubleshooting the Network Configuration

Do the following to verify if the network configuration prerequisites are met:

- Ensure that the nodes physically residing on different sites are pingable, once the stretched VLANs are created.

- Ensure that the Round-Trip Time (RTT) and bandwidth requirements are met across all the sites. You can use iPerf and Ping or any Cisco supported tool to measure the RTT and bandwidth.

Increasing Datastore Capacity May Not Show as Free Space

Issue:

Increasing the datastore capacity may not show as free space in the HX Connect UI and stcli.

Recommended Solution:

Resize the datastore depending on the actual space that is used by each VM.

Note

Use the maximum cluster capacity to resize the datastore. It reflects the actual used spaced of the datastore, which can be used as a hint to resize the datastore based on your requirements.

1. From HX Connect, select Datastores.

2. Select a datastore. Click Edit.

3. In the Edit Datastore dialog box, to resize the datastore, change the Size as needed.

4. Click Edit Datastore.

Troubleshooting for Site-to-Site Failover

Do the following during site-to-site failover (if needed):

- During site-to-site failure, depending on the time it takes to failover, VMs may see a transient All Paths Down (APD) event from the ESX layer. This APD is expected when the witness connection is slow.

- In some cases, you may need to manually restart the user VMs if they have timed-out during site-to-site failover.

Do not try to resolve the APD by rebooting the nodes. Depending on the size of the cluster and amount of data, the failover can take 20-30 minutes or more if the witness transaction latencies are high.

Chapter 7. Post Installation

- Running Post Install Script

- Confirm That the Installation Is Complete

- Verifying Which Site a Node Belongs To

Running Post Install Script

After the installation of a Stretched Cluster using the HX Data Platform Installer, run the post installation script to finalize the configuration and set the vMotion network up. You can also run this script at a future time if needed.

If you wish to run the post_install.py script later, you can run the script directly from any Control Virtual Machine (CVM) using the following procedure:

1. Log into a Cluster IP (CIP) through a SSH server.

2. Run the post_install script available at the following location:

/usr/share/springpath/storfs-misc/hx-scripts/post_install.py

3. Follow the prompts and enter the required information.

Confirm That the Installation Is Complete

After the installation of the stretched cluster across both sites is successful, if HX Connect does not display the correct status, run the command stcli cluster info to verify the health and status of the stretched cluster.

Example:
root@ucs-stctlvm-365-1:~#stcli cluster info
about:
vCluster:…
upgradeState: ok
cluster:
…
nodeSiteMap:
10.104.2.67: nyc01
10.104.2.99: sjc02
10.104.2.97: sjc02
10.104.2.68: nyc01

Verifying Which Site a Node Belongs To

Use the command, stcli cluster get-zone, to check which node belongs to which site.

Example:
root@ucs-stctlvm-230-1:~# stcli cluster get-zonezones:
----------------------------------------
pNodes:
----------------------------------------
state: ready
name: 10.104.49.115
----------------------------------------
state: ready
name: 10.104.49.116
----------------------------------------
zoneId: 7f2bf7811475cacc:44dd22fa3eadfd4d
numNodes: 2
----------------------------------------
pNodes:
----------------------------------------
state: ready
name: 10.104.49.113
----------------------------------------
state: ready
name: 10.104.49.114
----------------------------------------
zoneId: 422fe637cab59ec5:4b49875b5641bf8a
numNodes: 2
----------------------------------------
isClusterZoneCompliant: True
zoneType: 2
isZoneEnabled: True
numZones: 2

Full Cisco Trademarks with Software License

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

All printed copies and duplicate soft copies of this document are considered uncontrolled. See the current online version for the latest version.

Cisco has more than 200 offices worldwide. Addresses and phone numbers are listed on the Cisco website at www.cisco.com/go/offices.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1721R)