FlexPod XCS: The First Cisco Intersight Integrated System

 

Introduction

The Cisco FlexPod solution delivers an integrated architecture that incorporates compute, storage, and network design best practices to reduce IT risks by validating the integrated architecture and helping ensure compatibility among the components. Since the first FlexPod solution was delivered to the market, each of the underlaying components of the solution is managed by its own management software stacks. For example, Cisco UCS® Manager manages compute components while various NetApp ONTAP management software stacks administer storage components. Then we must rely on VMware vCenter for managing components required for virtualization. At any given point of time, end users must deal with these management software stacks separately in order to administer and manage FlexPod deployed in the data centers. This white paper describes capabilities of the Cisco Intersight™ platform for providing a common management control plane and interface across the compute, storage, network, and virtualization components of a FlexPod instance. It introduces the FlexPod XCS integrated system with Cisco Intersight, highlighting methods for integrating FlexPod components into the Cisco Intersight platform. It describes creation of a FlexPod XCS integrated system instance on the platform and provides a walkthrough showing detailed full-stack inventory visibility, dashboard and metrics, and storage alarms associated with a FlexPod instance.

This paper describes foundation for the next step of complete infrastructure lifecycle management of a FlexPod solution. Existing feature capabilities of a Cisco Intersight for FlexPod XCS integrated system are the building blocks that will act as a prelude for bringing in an integrated experience across the full stack with four dimensions: turnkey automation, workload optimization, consumption model, and hybrid cloud. This experience extends from automated provisioning to day-n operations in the FlexPod environment, including health monitoring, interoperability compliance, and connected Cisco Technical Assistance Center (TAC) for support services.

Cisco Intersight for FlexPod solutions: FlexPod XCS

The Cisco Intersight platform (https://intersight.com) is an application programming interface (API) driven, cloud-based, software-as-a-service (SaaS) hybrid cloud operations platform. It delivers intelligent automation, observability, and optimization for traditional and cloud-native applications and infrastructure. It is a new generation of global management tools for Cisco Unified Computing System™ (Cisco UCS) and Cisco HyperFlex™ platforms and other devices, including third-party devices connected to the Cisco Intersight platform. It provides a holistic and unified approach to managing distributed infrastructure, cloud services, virtualized workloads, and container orchestration platforms. As a result, you can achieve significant total cost of ownership (TCO) savings and deliver applications faster in support of new business initiatives.

The Cisco Intersight platform is a SaaS infrastructure lifecycle management platform that delivers simplified configuration, deployment, maintenance, and support. With this platform, you get all the benefits of SaaS delivery model and the full lifecycle management of Cisco Intersight connected distributed servers and third-party storage systems such as NetApp across data centers, remote sites, branch offices, and edge environments (Figure 1).

The Cisco Intersight platform is designed to be modular, so you can adopt services based on your individual requirements. The platform significantly simplifies IT operations by bridging applications with infrastructure, providing visibility and management from bare-metal servers and hypervisors to serverless applications, thereby reducing costs and mitigating risk. This unified SaaS platform uses a unified OpenAPI design that natively integrates with the third-party platforms and tools. The FlexPod XCS with Cisco Intersight takes advantage of OpenAPI design capabilities for integrating the NetApp Active IQ Unified Manager to enable storage stack management from a unified interface.

Cisco Intersight overview

FlexPod XCS is a joint solution offering from Cisco and NetApp. Its primary goal is to enable you to simplify and accelerate your hybrid cloud journeys (Figure 2). It allows you to claim and manage FlexPod instances in the Cisco Intersight platform. FlexPod XCS is the first converged infrastructure (CI) integrated system stack on the Cisco Intersight platform that allows you to create and view FlexPod instances and monitor the state of the server, storage, and virtualization components that are part of the instance. With FlexPod XCS you can create a custom dashboard with predefined widgets for visibility into important metrics associated with the running instance. Additionally, it provides a single pane of glass view of triggered alarms on compute and storage components that are part of a FlexPod instance. It enables you to create and manage the FlexPod instances across the globe using a single cloud portal—Cisco Intersight—quickly and easily. With the Cisco Intersight platform, the FlexPod components are logically grouped into compute, networking, storage, and hypervisor entities, enabling the administrator to get a single converged stack view of the FlexPod.

 

FlexPod XCS

The FlexPod XCS integrated system on Cisco Intersight enables you to create FlexPod definitions from the Cisco Intersight platform to monitor FlexPod components. It provides the ability to monitor server and storage alarms/health, storage capacity, and high-level performance and virtualization information. Getting insights from different components of FlexPod and easily navigating through them from the Cisco Intersight platform is the key for FlexPod XCS. With this ability, you also can compare different FlexPod instances in a single pane of view: for example, top-five FlexPod instances with storage usage or by server alarms. With the FlexPod XCS integrated system feature you can bring your brownfield FlexPod environment to Cisco Intersight for visibility and in the future, manage your entire lifecycle directly from the Cisco Intersight platform.

FlexPod XCS delivers four core capabilities to help you accelerate your hybrid cloud journeys (Figure 3):

●      Visibility: It provides an enhanced and unified view of FlexPod that comprises Cisco Unified Computing and NetApp storage through the Cisco Intersight platform. The new FlexPod dashboard for monitoring within Cisco Intersight provides a single, detailed view of all components and enables more intelligent resource decisions.

●      Automation: Cisco Intersight Cloud Orchestrator simplifies all critical day-to-day operations on your FlexPod infrastructure, including configuration, deployment, expansion, and consumption/optimization.

●      Hybrid cloud services: Cisco Intersight and NetApp ONTAP storage data services work closely to bring higher levels of hybrid cloud operations and cloud native services support.

●      FlexPod-as-a-service: This service enables an OpEx consumption model for you to lower initial costs and tailor purchasing options based on budgets and usages.

Important pillars of FlexPod XCS

FlexPod integrated system instance creation on Cisco Intersight platform

Cisco Intersight offers flexible deployment either as SaaS running on Intersight.com or on your premises as Cisco Intersight Virtual Appliance. The virtual appliance provides the benefits of Cisco Intersight software while allowing more flexibility for those with additional data locality and security requirements. You can deploy the Cisco Intersight connected virtual appliance software on premises to take advantage of the SaaS functions; you also can deploy the private virtual appliance on premises with further security restrictions.

Cisco Intersight Assist helps you add endpoint devices to the Cisco Intersight platform. A data center could have multiple devices that do not connect directly with the platform. Any device that is supported by the platform but does not connect directly with it will need a connection mechanism. Cisco Intersight Assist provides that connection mechanism, and helps you add devices into the platform.

The FlexPod on Cisco Intersight is available either in the cloud or on premises as shown in Table 1.

Table 1.        Cisco Intersight product feature sets

The Cisco Intersight solution automation framework is designed and developed for onboarding solutions such as FlexPod to the Cisco Intersight cloud-based management services. This framework helps in defining the integrated system and various operations that can be performed on such instances. Solution action definitions define operations such as creating, editing, and deleting a FlexPod instance and performing health checks on predefined parameters of the instance. It also allows collection of alarms and alerts originating from critical FlexPod components on a per-instance converged view. Components include Cisco UCS server and fabric interconnects, NetApp storage, VMware virtualization, and Cisco Nexus® 9000 and Cisco MDS 9000 Series Switches (in a future release).

The following prerequisites are mandatory to create a FlexPod integrated system instance on the Cisco Intersight platform:

●      Users with administrative privileges associated with an account on Cisco Intersight can create FlexPod integrated system instances.

●      Users must have an Advantage or a Premier license tier assigned to servers that are part of the FlexPod environment.

●      The FlexPod environment can either be in Cisco UCS Manager mode or Intersight managed mode. The FlexPod environment must be configured to use storage from a NetApp ONTAP cluster with Small Computer System Interface over IP (iSCSI) and Network File System (NFS) protocols. Detailed guides and references on deploying FlexPod in Cisco UCS Manager mode and Intersight managed mode are given in the “For more information" section at the end of this document. Table 2 lists supported systems and software versions for Cisco UCS Manager mode, Intersight managed mode and storage components for FlexPod XCS.

Table 2.        Supported hardware and software versions

Figure 4 outlines the FlexPod environment and targets for claim on the Cisco Intersight platform that are required to create a FlexPod integrated system instance.

FlexPod XCS components

●      Cisco Intersight Assist, NetApp Active IQ Unified Manager, Cisco Nexus 9000 Series switches and VMware vCenter must be deployed, configured, and claimed into Cisco Intersight beforehand to facilitate creating a FlexPod integrated system on Cisco Intersight. Detailed guides and references are given below at the end of this document.

●      NetApp Active IQ Unified Manager should be configured to allow REST API request processing by enabling the API gateway.

The FlexPod instance creation workflow is a simple three-step process:

1.     Verify that Cisco UCS Manager domain or Cisco Intersight managed mode domain, Cisco Nexus 9000 Series Switches, NetApp Active IQ Unified Manager with corresponding claimed ONTAP storage cluster, VMware vCenter, and Intersight Assist running on premises have been successfully deployed, claimed into Cisco Intersight and are in connected state for FlexPod instance creation.

Deployment of the FlexPod environment in Cisco UCS Manager mode or Cisco Intersight managed mode, AIQUM, vCenter, and Cisco Intersight Assist is out of the scope for this document. Please refer to the "For more information" section for guidance.

2.     Claim all mentioned components in step 1 on the Cisco Intersight platform.

To claim a new target, the following common procedure applies to the previously mentioned endpoints:

◦    Log in to Cisco Intersight with the account administrator, device administrator, or device technician role.

◦    Navigate to ADMIN > Targets > Claim a New Target.

◦    Choose Available for Claiming and select the target type you want to claim. Click Start.

◦    Enter the required details and click Claim to complete the claiming process.

3.     Create a FlexPod instance using the guided integrated system creation wizard.

Claiming Cisco Intersight Assist and Cisco UCS Manager domain

The device connector provides a secure way for connected targets to send information and receive control instructions from the Cisco Intersight portal using a secure Internet connection. Table 3 provides the format of the device ID and the device connector location.

Table 3.        Device ID and claim code format and device connector location

For a Cisco Intersight Assist claim, you will perform two actions on Admin > Targets > Claim a New Target. First select the Intersight Assist option under Platform Services. Then fill in the device ID and claim code taken from the method provided in the previous location table and the complete claim flow.

For claiming Cisco UCS Manager, select Admin > Target > Claim a New Target > Compute/Fabric > Cisco UCS Domain (UCSM Managed).

Next fill in the device ID and claim code taken from the location described in Table 3 for the Cisco UCS domain.

Claiming Cisco UCS domain in Cisco Intersight managed mode

For a Cisco UCS domain in Cisco Intersight managed mode (IMM), to claim it as the target, you must navigate through Admin > Targets > Claim a New Target > Compute / Fabric > Cisco UCS Domain (Intersight Managed). The next screen requires the device ID and claim code for either the primary or subordinate fabric interconnect running in the IMM configuration. For further details, you can follow the steps as described here: https://intersight.com/help/saas/getting_started/claim_targets#target_claim_in_intersight_managed_mode

To create a FlexPod XCS instance on the Cisco Intersight platform, you must deploy FlexPod in Cisco Intersight managed mode with the required computing policies and profiles, Cisco UCS domain profile, and NetApp storage configuration, and you must create a VMware data center with running hypervisors on the Cisco UCS nodes. Complete deployment steps can be found in this guide: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_xseries_vmware_7u2.html

Claiming Cisco Nexus 9000 Series Switches, NetApp Active IQ Unified Manager, and VMware vCenter endpoint targets

The next set of targets essential for a FlexPod instance creation consists of Cisco Nexus 9000 Series network switches, NetApp's AIQUM, and vCenter. These endpoints do not run device connectors natively in order to have them claimed directly on the Cisco Intersight platform. They require Cisco Intersight Assist to get claimed. The target types are network, storage, and hypervisors for Cisco Nexus 9000 Series Switches, AIQUM, and vCenter, respectively. For all of these target types, you need to make sure Cisco Intersight Assist can reach to these endpoints over an IP network. For the Hostname/IP Address field, you can use FQDN if their hostnames are registered with DNS. Otherwise use the IP address.

For Cisco Nexus 9000 Series Switches, select Admin > Target > Claim a New Target > Network > Cisco Nexus Switch.

For AIQUM, select Admin > Target > Claim a New Target > Storage > NetApp Active IQ Unified Manager.

While claiming the vCenter target, you must enable the datastore browsing and guest metrics options. These options help in provisioned datastore visibility and collection of virtual machine resource usage metrics.

When all the required endpoints have been claimed successfully as targets on the Cisco Intersight platform, their status should be as shown in the following screenshot.

Creating a FlexPod integrated system instance

The next step is to create a FlexPod instance with the previously created FlexPod environment running in the Cisco UCS Manager domain or a Cisco UCS domain managed by Cisco Intersight software. Users with Cisco Intersight Advantage or Premier licenses will have the integrated systems feature tab appear at the left navigation bar on their Cisco Intersight portal account. With this integrated systems feature tab you can create, delete, or edit FlexPod instances.

1.     Choose Operate > Integrated Systems from the left navigation pane.

2.     In the Integrated Systems page on the right panel, click Create Integrated System.

3.     Select FlexPod on the Select Integrated System page that comes next. Click Start.

4.     In the General screen complete the following fields:

◦    Organization: Select Default or choose the previously created organization from the drop-down menu. The FlexPod instance should then be created under the selected organization.

◦    Name: Provide a name for the FlexPod instance to be created.

◦    Set tags: Tags enable filtering of FlexPod instances for display and identification.

◦    Description (optional): This user input describes the instance.

5.     Click Next.

 

6.     Select Cisco UCS domains that are running FlexPod components on the UCS Domain Selection page. Selection of multiple domains is allowed because a FlexPod instance can be composed of more than one Cisco UCS domain. Even a mix of a Cisco UCS Manager domain and a Cisco Intersight managed domain is possible for a FlexPod instance if they are running a valid FlexPod environment that includes NetApp ONTAP storage and a VMware hypervisor.

7.     Click Next.

 

8.     On the Network Switch Selection page, select the Cisco Nexus 9000 Series Switches that are part of the FlexPod environment and previously configured to handle the network data path.

9.     Click Next.

10.  On the Storage Array Selection page, select an available storage array that is part of a previously deployed FlexPod environment.

11.  Click Next.

12.  The next page shows a summary of selected options from the previous steps. This page is provided for review so that you can make any changes necessary.

13.  Click Create.

14.  Monitor progress by clicking the Request icon at the top-right corner. It shows workflow progress, success, or failures with relevant errors, if any.

 

The create action invokes Cisco Intersight Cloud Orchestrator services to execute the FlexPod integrated system instance workflow. Successful execution results in creation of the FlexPod instance visible on the integrated systems page. You can now navigate through it for full-stack visibility and perform edit and delete actions. One important data point to note here: even though the vCenter claim is essential for FlexPod XCS, it was never selected while you were creating the instance. The system internally validates its relationship with the FlexPod instance by mapping the server hardware running the hypervisor to vCenter data center, cluster, and host objects under the virtualization inventory.

Converged inventory view, full-stack visibility, and alarms on Cisco Intersight platform

Traditionally, FlexPod environments lacked a single management dashboard for administrators that could provide visibility to all four main components of a FlexPod. There was no way to view the compute, storage, network, and virtualization resources together through a single interface. FlexPod XCS bridges this long-standing gap and aggregates various resources by presenting a consolidated inventory view and full-stack visibility on the Cisco Intersight platform. Inventory that you couldn't see includes aggregation of servers, hypervisors, virtual machines, storage data, networks, health status, and alarms associated with FlexPod resources. FlexPod XCS also checks interoperability compliance using the Cisco Hardware Compatibility List (HCL) and NetApp Interoperability Matrix Tool (IMT) on underlaying compute, storage, network, and virtualization resources.

The Cisco Intersight platform has complete knowledge of Cisco UCS server inventory data natively collected and curated. For NetApp and VMware, it uses AIQUM and vCenter device connectors with Cisco Intersight Assist to collect storage and virtualization inventory (Figure 5):

FlexPod XCS visibility stack

On completion of FlexPod instance creation, the Cisco Intersight inventory services internally tags all the required components of the FlexPod instance to aggregate collected inventory information. It then calculates and populates all summary information: in other words, total nodes, storage information (capacity/usage), virtualization information (number of virtual machines, active/suspended, number of hypervisors, etc.), alarms, and HCL status.

Cisco Intersight Inventory service also aggregates storage information from storage arrays for storage capacity/usage across all FlexPod environments that are part of a FlexPod integrated system instance. As shown in Figure 5, inventory collection of NetApp arrays occurs after the successful FlexPod instance creation, and then periodically through the NetApp AIQUM.

Alarms associated with FlexPod components help in deriving the overall health of a FlexPod integrated system instance. Alarms are updated as soon as they are raised, helping ensure that overall health is a real-time reflection and accurate at any point in time. Alarms are aggregated based on their severity. More information about alarms on the Cisco Intersight platform is available at https://intersight.com/help/saas/features/cisco_intersight/monitor#alarms.

The General tab (Figure 6) shows the high-level inventory view associated with a FlexPod instance. It has four broad categories: servers, switches, storage, and virtualization. It also shows the aggregated view of storage and server alarms with the severity level.

FlexPod XCS instance general tab for inventory view

The Inventory tab shows detailed inventory views of servers, fabric interconnects, networking, storage, and virtualization, which are part of the FlexPod instance. Figure 7 shows the FlexPod instance server inventory.

Inventory tab showing the server inventory

You can get detailed views similarly for fabric interconnects, network switches, storage, and virtualization components of the FlexPod instance. For example, inventory details for Cisco Nexus switches that are part of a FlexPod XCS integrated system instance can be viewed by choosing Integrated system > FlexPod instance > Inventory > Networking > Ethernet Switches (Figure 8).

Detailed view of Cisco Nexus switches

FlexPod XCS dashboard and metrics

The Cisco Intersight platform provides a built-in monitoring capability for a FlexPod integrated system instance under monitoring dashboards. The default FlexPod dashboard is created as part of your initiated FlexPod instance creation. This default dashboard monitors important metrics associated with the FlexPod instance. This dashboard comes with a set of nine important metrics (Figure 9):

●      FlexPod server model summary

●      FlexPod fabric interconnect firmware version summary

●      FlexPod server firmware summary

●      FlexPod storage OS version summary

●      Top-five FlexPod storage arrays by capacity usage

●      Top-five FlexPod instances by capacity usage

●      FlexPod fabric interconnect model summary

●      Top-five FlexPod volumes by capacity usage

●      FlexPod storage node model summary

FlexPod dashboard

In addition, you can create a custom dashboard to suit your needs by using predefined FlexPod widgets. One of the custom metrics widgets provides a real-time status of the top-five FlexPod fabric interconnect ports by traffic usage. This widget enables you to monitor network traffic on port-channel interfaces and individual Ethernet interfaces on fabric interconnects used in the FlexPod instance. You have the option to select a required time interval and the level of granularity for one or more Cisco UCS Manager domains (Figure 10).

Custom dashboard

Interoperability check

The FlexPod XCS integrated system interoperability feature allows users to check the compatibility of Cisco UCS servers, adapters, fabric interconnects, host operating systems, NetApp storage arrays, Cisco Nexus switches, and VMware components. This feature combines the Cisco HCL and NetApp IMT in one place with a homogeneous interface. It displays the compliance status of the qualified hardware and software components and their versions used in the previously deployed FlexPod instance running with the iSCSI Protocol configuration. In the past, users had to use multiple interfaces manually to understand their FlexPod components’ interoperability compatibility. They needed to first check the Cisco HCL to know what is compliant according to Cisco HCL and then select the same items in the NetApp’s IMT web interface to make sure they are compliant with the NetApp IMT as well. With the FlexPod XCS interoperability feature, users now have a single automated interface for checking compatibility. No more manual selection of components to check compatibility through Cisco HCL and NetApp IMT tool separately (Figure 11).

Automated compatibility check

Users can run this check based on their needs, and the results summary has last-run time stamp. A user-initiated check triggers an internal workflow consisting of tasks to collect FlexPod inventory information and get compliance data from the Cisco HCL and  NetApp IMT (Figure 12).

Interoperability check

A successful workflow run results in an interoperability status report, displayed as shown in Figure 13. This report lists a summary of results and provides a detailed view of each of the FlexPod components, such as servers, fabric interconnects, and storage. Users can view detailed status information about each of these groups by navigating the page.

Interoperability status report

The status report shows four states:

●      Not Evaluated: Components are not in the Cisco HCL or NetApp IMT.

●      Incomplete:  The collected inventory information is incomplete: for example, if computing nodes are not running any hypervisor operating system.

●      Not Approved: Components are not approved in the interoperability compliance check.

●      Approved: Components are approved based on the interoperability compliance check.

Conclusion

●      The Cisco Intersight platform is best described as a SaaS infrastructure lifecycle management solution that delivers simplified configuration, deployment, monitoring, maintenance, and support. The Cisco FlexPod integrated system delivers an integrated architecture that incorporates computing, storage, virtualization, and network design best practices to reduce IT risk through validated architecture to help ensure best practices and interop compatibility.

●      FlexPod XCS integrates the Cisco Intersight platform with a FlexPod environment to provide global visibility of FlexPod infrastructure components, health, alarms, and metrics for monitoring. It provides a single pane of glass for managing all underlying components of the FlexPod environment. Cisco FlexPod XCS also helps ensure Cisco HCL and NetApp IMT interop compatibility governance to reduce day-to-day operational risks.

For more information

Consult the following references for additional resources on the topics covered in this document:

●      Cisco Intersight platform: https://www.intersight.com

●      Cisco Intersight Integrated Systems Help: https://intersight.com/help/saas/features/integratedsystems/operate

●      Cisco Intersight managed mode: https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/b_Intersight_Managed_Mode_Configuration_Guide.html

●      Cisco Intersight Virtual Appliance and Cisco Intersight Assist: https://www.cisco.com/c/en/us/td/docs/unified_computing/Intersight/b_Cisco_Intersight_Appliance_Getting_Started_Guide.pdf

●      Cisco Intersight managed mode for FlexPod: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/flexpod/cisco-imm-for-flexpod.html

●      FlexPod Datacenter deployment guide with Cisco UCS X-Series Modular System and Cisco UCS 5108 Blade Server Chassis in Cisco Intersight managed mode: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_xseries_vmware_7u2.html

●      FlexPod Datacenter with Cisco UCS 4.2(1) in Cisco UCS managed mode: https://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/flexpod_m6_esxi7u2.html

●      NetApp ONTAP 9: https://www.netapp.com/data-management/oncommand-system-documentation/

●      NetApp Active IQ Unified Manager: https://www.netapp.com/support-and-training/documentation/active-iq-unified-manager/

●      NetApp Active IQ Unified Manager installation: https://docs.netapp.com/us-en/active-iq-unified-manager/install-vapp/concept_requirements_for_installing_unified_manager.html