Validate APC PowerChute Network Shutdown Integration with Cisco HyperFlex Edge Cluster White Paper

 

Introduction

Today the IT industry is going through a tremendous revolution. Evolving technologies such as the Internet of Things (IoT), image processing, Artificial Intelligence Markup Language (AIML), autonomously driven vehicles, and intelligent manufacturing are growing rapidly, and massive amounts of data are being created, processed, and analyzed. This changing environment creates a unique opportunity and challenge for edge computing. A new concept evolving in the industry, the edge data center, also called the micro data center, is being employed to distribute workloads closer to the connected devices with much less latency. The edge data center gives devices faster and secure access to data and computing resources. It efficiently delivers power for computing but does not require the full power of a traditional data center and therefore has a much lower cost. Not only are customers transitioning their software and applications, but they are also transforming the hardware layer and security architecture for the underlying distributed infrastructure.

Virtualization commonly is used to solve many of the challenges of this new computing landscape. Hyperconvergence is an evolving technology that builds on many of the benefits of virtualization. Hyperconverged infrastructure (HCI) is fast to deploy and simple to manage and offers increased agility, thereby reducing the cost of daily IT operations. Hyperconverged infrastructure has quickly become a standard choice for enterprise and edge data centers.

A proven industry-leading hyperconverged solution, Cisco HyperFlex™ Edge is an optimal choice for many applications in a virtualized edge environment. Cisco HyperFlex systems let you unlock the full potential of hyperconvergence and adapt IT to the needs of any workload at any location at any scale. With Cisco HyperFlex systems, customers have the flexibility to support different types of workloads without compromising their performance requirements. It provides an excellent choice of hardware and storage for the high-performing virtual infrastructure required for many new applications.

While virtualization has changed the IT landscape, the need for power protection remains. APC by Schneider Electric PowerChute Network Shutdown (PCNS) software reduces business risk by protecting real-time data and helping ensure uptime productivity through graceful shutdown and unattended startup processes in response to prolonged power disturbances. Although Cisco HyperFlex HX Data Platform software is resilient in response to any type of planned or unplanned outage, organizations must make sure that applications running on top of the cluster are properly and gracefully shut down and subsequently restarted. Graceful shutdown is not required for data integrity for the Cisco HyperFlex storage platform, but it does complement the solution by providing the necessary power-down and power-up orchestration.

PowerChute seamlessly integrates with Cisco HyperFlex systems, supporting virtual clusters through multiple uninterruptible power supply (UPS) configuration options for VMware ESXi. The software communicates over the network with your APC UPS to gracefully shut down your physical servers and virtual machines. This protection decreases application-level inconsistencies and reduces downtime when critical events such as extended power outages occur.

The solution provides options to simplify configuration:

●      Use the intuitive step-by-step wizard.

●      Manage your hosts and UPSs from a single screen with simple drag-and-drop functionality.

●      Prioritize the shutdown and startup of virtual machines in high, medium, or low priority groups.

 

The solution also offers seamless integration:

●      PowerChute gracefully shuts down user virtual machines, Cisco HyperFlex cluster service, and controller virtual machines in the correct order before gracefully shutting down the ESXi hosts.

●      When power returns to normal, PowerChute automatically restarts Cisco HyperFlex cluster service and user virtual machines.

●      In Cisco HyperFlex Edge environments, PowerChute can be deployed as a virtual machine on local storage within the cluster.

In addition to this collaboration, Cisco and Schneider Electric have released a new edge computing solution that integrates Cisco’s HyperFlex Edge hyperconverged infrastructure with the APC NetShelter WX 6U Wall Mount Enclosure. In combination with PowerChute Network Shutdown software, this integration provides a complete solution for your Cisco HyperFlex Edge deployment.

Installation and configuration

The PowerChute virtual appliance is a virtual machine image with CentOS Linux 8.1 running PCNS software preinstalled. To deploy and configure the virtual appliance, follow these steps:

1.     Go to the APC website www.apc.com/pcns to download the software-only PowerChute open virtual appliance (OVA) file (Release 4.4 or later).

2.     Log in to the VMware vSphere vCenter web client interface for the Cisco HyperFlex Edge cluster.

3.     Right-click the Cisco HyperFlex cluster and choose Deploy OVF Template.

4.     Enter the location of the downloaded PowerChute OVA file. Click Next.

5.     Enter the name of the PowerChute virtual machine and choose the data center in which the virtual machine is deployed. Click Next.

6.     Choose the Cisco HyperFlex Edge cluster that is the intended target. Click Next.

7.     Review the required resources. Click Next.

8.     Accept the license agreement. Click Next.

9.     Choose the local data store (name starting with SpringpathDS-) on any of the Cisco HyperFlex Edge nodes used for the deployment. Select Thin Provision for the virtual disk format. Click Next.

             

10.  Choose Storage Controller Management Network on the Cisco HyperFlex Edge cluster for the PowerChute appliance administrative network. Click Next.

             

11.  On the Customize Template page, specify IP settings for the PowerChute virtual appliance such as the Domain Name System (DNS), gateway, IP address, netmask, and host name. Enter a root password. Click the box to enable Secure Shell (SSH) service. Enter a valid Internet Assigned Numbers Authority (IANA) time-zone setting: for example, Etc/UTC, America/New_York, or Europe/Dublin. Click Next.

12.  Review the deployment properties. Click Finish to start creation of the PowerChute virtual appliance.

13.  When the OVF deployment is complete, power on the virtual machine.

14.  If you chose to skip step 11 when deploying the PowerChute virtual appliance or provided incomplete parameters, the virtual appliance First-Time Configuration Wizard opens. You can complete the PowerChute basic configuration with this wizard.

15.  In your web browser, open https://<PCNS_hostname>:6547 or https://<PCNS_IP_Address>:6547 to launch the PowerChute appliance Setup Wizard. When the PowerChute Setup: Welcome page appears, click Next to follow the steps for setting up the application.

16.  For VMware Configuration, select “Host managed by vCenter Server.” Click Next to enter the vCenter information.

 

17.  To enable Cisco HyperFlex support, on the vCenter Server Details page select HyperFlex from the Hyperconverged Infrastructure Support drop-down list.

18.  On the HyperFlex Details page, enter the Cisco HyperFlex cluster IP address and the administrator’s user name and password to connect to your cluster.

19.  Click Next to validate the connection. Wait until the connection to the Cisco HyperFlex Cluster is successful. The Virtualization Settings page will appear.

20.  Configure the settings according to the needs of your specific environment for Virtual Machine and Virtual Appliance Shutdown and Startup, Host Maintenance Mode, vCenter Server VM Shutdown, Cluster Shutdown/Startup, and Controller VM Shutdown/Startup. Also choose whether to wait for all hosts to be online prior to startup.

21.  Continue to finish the PowerChute setup with Cisco HyperFlex integration.

22.  After the initial setup is complete, you will be directed to the PowerChute administrative GUI. Here you can make changes to PowerChute setup and virtualization settings. You can specify new configurations as necessary for Simple Network Management Protocol (SNMP), SSH, events, shutdown, host protection, and virtual machine prioritization settings, etc. You can also monitor the event logs from the GUI.

Figure 1 shows an example of the PowerChute virtualization settings on a Cisco HyperFlex Edge cluster.

Example of PowerChute virtualization settings

Be sure to set the proper duration values for your environment. Cisco recommends that you manually test the parameters with different events and then specify the durations to allow sufficient time for your system to shut down gracefully and to be restored in sequence successfully. To enable the Cisco HyperFlex system to fully restore all the services before restarting the cluster, you should to set the controller virtual machine startup duration for at least 5 minutes. In the PowerChute configuration file (pcnsconfig.ini), the cluster_ops_retries setting allows you to set a value (the default is 10) for retry attempts. PowerChute will re-attempt to shut down and start up the Cisco HyperFlex Cluster with the shutdown or startup duration in between each attempt until the task is successful or the retry limit has been reached.

In a PowerChute configuration, virtual machine prioritization commonly is enabled, and the virtual machines are placed in different priority groups with appropriate shutdown and startup durations. This configuration is helpful if you have multiple servers to protect and you want to extend the runtime for your higher-priority servers such as the vCenter appliance. It also helps ensure that lower-priority servers are the first to be shut down. For the embedded vCenter scenario, if virtual machine prioritization is enabled the vCenter appliance will be shut down as part of the virtual machine group to which it belongs. If the vCenter appliance virtual machine is not in any priority group, it will be shut down after the guest virtual machines. If high availability (HA) is disabled on the cluster, a shared account must be used in PowerChute that can log in to vCenter and also to each ESXi host so that it can shut down the hosts directly after the vCenter virtual machine goes down.

The PowerChute virtual appliance is deployed in the Cisco HyperFlex cluster, and it depends on the vSphere HA policy or virtual machine autostart configuration to power it back up with the host. Customers commonly have vSphere Distributed Resource Scheduler (DRS) and vSphere HA enabled in Cisco HyperFlex clusters. You need to be cautious in specifying the HA Admission Control settings because the cluster may not be restored if the resource calculation is not correct. The recommended approach is to disable HA Admission Control to avoid the cluster recovery issue. If HA is disabled on the cluster, the PowerChute appliance virtual machine must be configured to automatically start up with the system. After the PowerChute appliance virtual machine starts, the guest virtual machines will start based on the PowerChute virtual machine prioritization settings. When a zero-second startup duration is set in PowerChute for a virtual machine priority group, this effectively means that all virtual machines in that group will be skipped for virtual machine startup. Then the guest virtual machines will start up based on the vSphere autostart configuration.

Figure 2 shows an example of the vSphere autostart configuration.

Example of a VMware vSphere autostart configuration

Validation testing

Here is the list of the devices used in validating the solution described in this document (Figure 3):

●      Cisco HyperFlex Edge clusters (x 2) composed of:

◦    Cisco HyperFlex Edge HXAF-E-220M5SX All Flash converged node (x 4)

●      Cisco Nexus® 9348GC-FXP Ethernet top-of-the-rack (ToR) switch (x 2)

●      Cisco Catalyst® 9300-48P-E Ethernet ToR switch (x 2)

●      Cisco Intersight™ cloud management platform (for Cisco HyperFlex installation and management)

Validation testing devices

In this solution, four Cisco HyperFlex HXAF-E-220M5SX All-Flash rack servers are used to build two 2-node Cisco HyperFlex Edge clusters: one with an embedded vCenter appliance and another with an external vCenter appliance.

Note:      The solution is supported only on Cisco HyperFlex M5 servers that have a solid-state disk (SSD) backed data store. Cisco HyperFlex M4 servers have not been tested.

Table 1 lists the Cisco HyperFlex Edge converged node specifications.

Table 1.        Cisco HyperFlex HXAF-E-220M5SX node specifications

Table 2 lists the software versions used in the test environment described in this document.

Table 2.        Test environment software versions

 

The test environment in a Cisco HyperFlex Edge cluster was set up with one load master virtual machine, one Dynamic Host Configuration Protocol (DHCP) server virtual machine, and multiple load virtual machines on each Cisco HyperFlex HX-Series node, along with the PowerChute appliance virtual machine and vCenter appliance virtual machine for the embedded scenario (Figure 4). All load virtual machines and the DHCP server virtual machine connect to the guest virtual machine network, and the master virtual machine has two virtual network interface cards (vNICs), with one connected to the Cisco HyperFlex management network and another connected to the guest virtual machine network. The PowerChute appliance virtual machine and the vCenter appliance virtual machine are connected to the Cisco HyperFlex management network. All load virtual machines obtain IP addresses from the DHCP server. The VDBench test tool was run on the master virtual machine using command-line interface (CLI) commands to generate I/O workloads on all the load virtual machines. I/O processing was run before the cluster shut down and after the cluster was restored to verify application-level functionality across the power cycle event.

Validation testing setup

Testing was performed using the following procedure:

1.     Create a two-node Cisco HyperFlex Edge cluster using the Cisco Intersight™ platform.

2.     Deploy an embedded vCenter appliance virtual machine on a data store created in the Cisco HyperFlex cluster. Register the Cisco HyperFlex cluster with this vCenter appliance.

3.     Deploy the load master virtual machine, DHCP server virtual machine, and load virtual machines in the Cisco HyperFlex Edge cluster. Power them up and establish the workload test environment.

4.     Start a 1-hour I/O run on the load virtual machines to verify the initial performance of the cluster and the virtual machines. Wait until the I/O run completes.

5.     Deploy the PowerChute virtual appliance on local storage on one of the Cisco HyperFlex Edge nodes.

6.     Power on the PowerChute appliance virtual machine and run the setup wizard to complete the PowerChute configuration.

7.     Set appropriate values to configure shutdown events: for example, the duration for virtual machine shutdown and startup, the duration for cluster shutdown and startup, and virtual machine prioritization.

8.     Use the representational state transfer (REST) API call to simulate a UPS critical event and start the shutdown sequence. For example, the API call might look like this:

https://{{pcns_ip_address}}:6547/REST/remoteShutdown?source=UPS&reason=UPS On Battery&active=True

Note: This API call was used to simulate a power event. In a real production cluster, a Schenider Electric UPS would automatically trigger events based on local power conditions.

9.     Verify that all the virtual machines and the Cisco HyperFlex cluster shut down gracefully in the order shown in Figure 5.

Shutdown procedure

Note:      The host that runs the PowerChute virtual machine will not be placed in maintenance mode so that the PowerChute virtual machine can automatically start up after power is restored.

This completes the graceful shutdown sequence. To continue with graceful startup, continue with the following steps.

10.  From the Cisco Intersight interface (or the management interface of your choice), power on the two Cisco HyperFlex Edge nodes that were shut down.

11.  Verify that the Cisco HyperFlex cluster and all the virtual machines start up gracefully in the order shown in Figure 6.

Startup procedure

12.  After all virtual machines power up, start a 1-hour I/O run on the load virtual machines to verify the performance of the cluster and the virtual machines. Wait until the I/O run completes.

13.  Create a two-node Cisco HyperFlex Edge cluster from the Cisco Intersight interface with an existing external vCenter server appliance.

14.  Repeat steps 3 through 12 to verify that the PowerChute graceful shutdown and startup sequences run as intended.

For more information

For additional information, see the following:

●      Cisco Validated Design: Cisco HyperFlex Edge 4.0 with Cisco Intersight

●      Cisco HyperFlex products, services, and solutions: https://www.cisco.com/go/hyperflex

●      To download PowerChute Network Shutdown software: www.apc.com/pcns

●      For more information about the partnership between Cisco and APC by Schneider Electric and integrated solutions: https://www.apc.com/us/en/partners-alliances/alliances/technology-alliances/cisco.jsp