Cisco ACI with Cisco UCSM Integration

Automating Networking Policies for Cisco UCS Devices with Cisco ACI

Beginning with Cisco Cisco Application Policy Infrastructure Controller (APIC) Release 4.1(1), you can automate networking policies on Cisco Unified Computing System (UCS) devices. To do so, you integrate Cisco UCS Manager (UCSM) into the Cisco Application Centric Infrastructure (ACI) fabric.

Cisco APIC takes hypervisor NIC information from the Cisco UCSM and a virtual machine manager (VMM) to automate VLAN programming. The automation applies to all the devices that the Cisco UCSM manages: Cisco UCS Fabric Interconnects and Cisco UCS B-Series Blade Chassis with UCS Blade Switches and Virtual Interface Card (VIC) Interfaces.

After you fulfill the prerequisites, you must perform two tasks in Cisco Application Policy Infrastructure Controller (APIC) to integrate Cisco UCSM into Cisco ACI:

  • Create an integration group, which is the basis for your security domain.

    Integration groups allow you to tie various types of integrations into the Cisco ACI fabric. Integration groups also allow a specific set of users to access the integrations with that group.

    For example, you may have multiple pods in your fabric and have administrators who are assigned to different pods. You can create an integrations group for each pod and add the integrations that reside within specific pods. You can then assign the security domain to the group for the administrators who oversee the pod.

  • Create an integration of the type UCSM, which allows the Cisco APIC to manage the networking portion of the Cisco UCSM.

You can perform these tasks in the Cisco APIC GUI under the Integrations tab, by using REST API, or the NX-OS style CLI.

You may also need to associate a switch manager with the virtual machine manager:

  • If you use Cisco AVS or Microsoft SCVMM, you must associate a switch manager with the virtual machine manager.

  • If you use Cisco ACI Virtual Edge or VMware vSphere Distributed Switch (VDS), you must associate a switch manager with the virtual machine if one of the following is true:

    • LLPD or CDP is not enabled in the VMM domain vSwitch policy.

    • The ESXi management port (vmknic) is bound to a portgroup managed by Cisco ACI.

Cisco APIC is used only to manage the networking component of Cisco UCS devices. The Cisco UCS data management engine (DME) performs its usual functions. These include managing the databases of all physical elements, the logical configuration data for profile, policies, pools, vNIC and vHBA templates, and networking-related configuration details. DME also monitors the health and state of components.


Note
A VMware distributed virtual switch (DVS) domain with EDM UCSM integration may fail. The domain fails if you configure microsegmentation or enable intra-EPG isolation on the endpoint group (EPG) attached to the domain and you use UCSM Mini 6324, which does not support private VLANs.

The section assumes that you are familiar with Cisco UCS and Cisco UCSM. For more information, see the Cisco UCS documentation and Cisco UCSM documentation on Cisco.com.

Cisco UCSM Integration Prerequisites

Integrating Cisco Unified Computing System Manager (UCSM) with Cisco Application Centric Infrastructure (ACI) fabric has the following prerequisites:

  • Cisco Application Policy Infrastructure Controller (APIC) Release 4.1(1) or later.

  • Cisco UCS and Cisco UCSM properly installed and configured in your data center.

  • Cisco UCSM 3.2 or later.

  • UCSM vNIC templates that are configured as Updating Template type.

  • Creation of a VMware VMM domain or a Microsoft System Center Virtual Machine Manager (SCVMM) domain.

  • Installation of the Cisco External Switch app, which can be found in the Cisco ACI App Center on Cisco.com.

For information about these tasks, see the Cisco APIC documentation and the Cisco UCSM documentation on Cisco.com.

Integrating Cisco UCSM into the Cisco ACI Fabric Using the Cisco APIC GUI

This section contains instructions for integrating Cisco Unified Computing System Manager (UCSM) into the Cisco Application Centric Infrastructure (ACI) fabric using the Cisco Cisco Application Policy Infrastructure Controller (APIC) GUI.

Creating an Integration Group Using the Cisco APIC GUI

Integrating the Cisco Unified Computing System Manager (UCSM) into the Cisco Application Policy Infrastructure Controller (ACI) fabric requires an integration group. The integration group provides a consistent security domain for various integrations in the fabric.

When you create the integration group, you can optionally create a security domain or choose an existing one. A security domain enables you to restrict access to Cisco UCSM devices associated with the group.

You can create the group and configure a security domain in the Cisco APIC GUI.

Before you begin

You must have fulfilled the prerequisites that are listed in the section Cisco UCSM Integration Prerequisites in this guide.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Go to Integrations > Create Group.

Step 3

In the Create Integration Group dialog box, complete the following steps:

  1. In the Name field, enter the name of the integration group.

    Note 
    Step 3b through Step 3d are optional.

  2. In the Security Domains area, click the + (plus) icon.

  3. In the Create Security Domain dialog box, in the Name field.

    Alternatively, you can choose an existing security domain. In that case, skip step 3d.

  4. In the Create Security Domain dialog box, in the Description field, type a description of the security domain.

  5. Click Update and then click Submit.

    The group that you created appears in the Integrations central pane.

What to do next

Create an integration for the integration group. See the section Creating an Integration for the Integration Group Using the Cisco APIC GUI in this guide.

Also, if you created a security domain, assign users and access rights. See the Cisco APIC Security Configuration Guide on Cisco.com.

Creating an Integration for the Integration Group Using the Cisco APIC GUI

After you create an integration group, you must create an integration for it. An integration takes information from the Cisco UCSM and the target virtual machine manager (VMM) domain to program the VLANs on all the Cisco UCSM interfaces. The integration correlates the VMM physical NIC MAC address against the Cisco UCSM MAC address. The integration then programs UCSM NICs configured through UCSM vNIC templates.

Create an integration for each Cisco UCSM fabric. If you have multiple Cisco UCSM fabrics, create one integration for each additional fabric.

Before you begin

You must have completed the following tasks:

Procedure

Step 1

Log in to the Cisco Application Policy Infrastructure Controller (APIC):

Step 2

Go to Integrations > integration group.

Step 3

Double-click the integration group.
Step 4

In the left navigation pane, expand the integration group folder.
Step 5

Right-click the UCSM folder and then choose Create Integration Manager.

Step 6

In the Create Integration dialog box, complete the following steps:

  1. In the Name field, enter a name for the integration.

  2. In the Device IP/FQDN field, enter the Cisco UCSM virtual IP address or fully qualified domain name (FQDN).

    Cisco APIC supports the addition of port number to the IP address if you must specify a port number for a firewall or other authentication device. If you do not specify a port, Cisco APIC configures an HTTP connection.

    The following are examples of device IP addresses or FQDNs:

    • UCSM1.datacenter.intranet

      Note: When configured with an FQDN, the app that was installed as a prerequisite must have picked up the DNS changes. If you have added or removed DNS servers from the Cisco APIC configuration, disable and then re-enable the external SwitchApp so the changes can take effect on the app.

    • UCSM1.datacenter.intranet:8080

    • 172.16.10.2

    • 172.16.10.2:8080

    If you have multiple Cisco UCSM fabrics, each fabric requires its own integration.

  3. In the Username field, type the username that has Network Administrator read and write privileges and Server Profile Administrator read privileges on the Cisco UCSM.

  4. In the Password field, type the user password that has read, write, and computer permissions on the Cisco UCSM.

  5. In the Confirm Password field, retype the password.

  6. In the Deployment policy field, choose Leaf Enforced or accept the default Pre-Provision.

    If you choose the default Pre-Provision policy, Cisco APIC detects which VMM domain that you use. Cisco APIC then pushes all VLANs associated with that domain to the target Cisco UCSM.

    If you choose the Leaf Enforced policy, Cisco APIC detects only the VLANS that are deployed to the top-of-rack leaf nodes. Cisco APIC then filters out any undeployed VLANs, resulting in fewer VLANs pushed to the Cisco UCSM.

    If you choose to deploy a Cisco Application Centric Infrastructure (ACI)-managed EPG to an ESXi management NIC (vmknic), you must do one of the following:

    • Configure the EPG-VMM domain association with a Resolution Immediacy as Pre-Provision.

    • Configure the UCSM Integration Manager Deployment Policy as Pre-Provision.

  7. In the Preserve NIC Profile Config field, choose Overwrite, or accept the default Preserve.

    If you choose the default Preserve option, Cisco APIC does not remove manually configured VLANs present on the virtual NIC (vNIC) templates. If you choose the Overwrite option, manually configured VLANs are removed. You can remove previously configured VLANs later if you wish.

    If you choose Preserve, you can switch to Overwrite once you have completed integration to guarantee consistent configuration between Cisco APIC and Cisco UCSM.

  8. Click Submit.

    Cisco APIC creates the integration, which you can view in the central work pane under the UCSM folder. The System Info section shows the name of the Cisco UCSM target, its capabilities, and firmware version—Information that Cisco APIC took from the Cisco UCSM. The System Info also shows the IDs and management IP addresses of the Cisco Fabric Interconnects.

    You can also see the topology (Fabric Interconnects to top-of-rack switches) under the Topology tab in the work pane for the integration. In the System Info section of the work pane, you can see the path information.

What to do next

Perform the following tasks:

  • (Optional) To change the connection policy, click the Policy tab, change the content of the fields as required, and then click Submit.

  • You may need to specify configuration of specific uplink port channels. If so, follow instructions in the section Managing Uplink Port Channels Using the Cisco APIC GUI.

    This task is not necessary if traffic from the Cisco UCSM fabric only flows up to the Cisco ACI leafs.

  • If you use SCVMM, you must associate a switch manager to the virtual controller, following the instructions in the section Associating a Switch Manager with the Virtual Controller Using the Cisco APIC GUI.

  • If you use Cisco ACI Virtual Edge or VMware vSphere Distributed Switch (VDS), you must associate a switch manager with the virtual machine manager if LLPD or CDP is not enabled in the VMM domain vSwitch policy.

Managing Uplink Port Channels Using the Cisco APIC GUI

By default, any global VLAN created on a Cisco Unified Computing System Manager (UCSM) exists on both fabric interconnects in the Cisco UCSM Fabric. When Cisco Application Policy Infrastructure Controller (APIC) creates a VLAN, that VLAN is available across all uplinks.

However, your deployment may require that you specify a specific uplink port channel. For example, Layer 2 disjoint networks require that you make that specification.

To specify the uplink port channel for Cisco UCSM and the UCSM Fabric Interconnects, complete the steps in this procedure.

Before you begin

You must have created an integration group for Cisco UCSM and an integration for the integration group. If you have not already done so, follow the instructions in the sections Creating an Integration Group Using the Cisco APIC GUI and Creating an Integration for the Integration Group Using the Cisco APIC GUI in this guide.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Go to Integrations > integration group > UCSM > integration.

Step 3

In the Integration work pane, click the Uplink Profiles tab.

The work pane displays the uplink profiles, which are the port channel interfaces named on the Cisco UCSM.

Step 4

Click the desired uplink profile, and under the Managed column, click True.

Step 5

Click the desired uplink profile, check the check box under the Managed column, and then click Update.

What to do next

Note the following:

  • If you use Microsoft SCVMM, you must associate a switch manager with the virtual machine manager.

  • If you use Cisco ACI Virtual Edge or VMware vSphere Distributed Switch (VDS), you must associate a switch manager with the virtual machine if LLPD or CDP is not enabled in the VMM domain vSwitch policy.

If necessary, follow the instructions in the section Associating a Switch Manager with the Virtual Controller Using the Cisco APIC GUI.

Associating a Switch Manager with the Virtual Controller Using the Cisco APIC GUI

If you use Cisco Application Centric Infrastructure (ACI) Virtual Edge with a VMware domain, you can choose a switch manager to associate with the virtual controller. You can also choose a switch manager if you use a Microsoft System Center Virtual Machine Manager (SCVMM) domain.

Associating a Switch Manager to a virtual machine manager (VMM) controller allows the Cisco Unified Computing System Manager (UCSM) Integration to determine the NIC profiles for mapping VMM domains that do not rely on Link Layer Discover Protocol (LLDP) or Cisco Discovery Protocol (CDP) for their endpoint group (EPG) deployments.

For Microsoft System Center Virtual Machine Manager (SCVMM), creating this association is mandatory. In the case of Cisco ACI Virtual Edge and VMware DVS, create the association if LLDP/CDP is not used in your VMM Domain

Before you begin

You must have completed the following tasks:

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Complete one of the following sets of steps, depending on what kind of virtual domain you use: Go to Virtual Networking > Inventory.

  • If you use Microsoft SCVMM, go to Virtual Networking > Inventory > VMM Domains > Microsoft > domain > Controllers > Controller.
  • If you use Cisco ACI Virtual Edge, go to Virtual Networking > Inventory > > VMM Domains > VMware > domain > Controllers > controller.
Step 3

In the Controller Instance central work pane, choose the Policy and General tabs.

Step 4

In the Properties area, click the Associated Switch Managers + (plus) icon.

Step 5

Choose an option from the Switch Manager drop-down list, click Update, and then click Submit.

Downgrading Cisco APIC with Cisco UCSM Integration

If you want to downgrade Cisco Application Policy Infrastructure Controller (APIC) from Release 4.1(1) to an earlier release, you must take extra steps if you have integrated Cisco UCS Manager (UCSM) into the Cisco Application Centric Infrastructure (ACI) fabric. If you do not, global VLANs may be deleted from Cisco UCSM resulting in traffic loss.

Procedure

Step 1

Back up the Cisco UCSM configuration.

See the chapter "Backing Up and Restoring the Configuration" in the Cisco UCS Manager GUI Configuration Guide on Cisco.com.

Step 2

Remove the Cisco External Switch app from the Cisco APIC Apps tab.

Downgrading the Cisco APIC or removing the integration before removing the External Switch app triggers a cleanup of the Cisco UCSM.

Step 3

After Cisco External Switch app has been removed from Cisco APIC, you can proceed the downgrade.

The configuration that was published from Cisco APIC continues to remain on your UCSM.