Integrate and Troubleshoot Virtual Machine Manager

Introduction

This document provides a high level checklist for how to set up Virtual Machine Manager (VMM) integration, then is followed by some common mistakes/errors and faults to look for. It also covers additional commands used to troubleshoot common VMM domain-related issues.

Overview

VMM integration allows a VMM (vCenter, SCVMM, and so on) to be linked to Application Centric Infrastructure (ACI) so that policies can be made available for virtual machines in the same way as bare metal. ACI supports multiple VMM domains to be configured, which can be a mix of Hypervisor managers. At First Customer Ship only vCenter wil be supported, but expect HyperV and other Hypervisors to be added not long after.

End Point Groups (EPGs) are used in the same way with virtual machines as they are with bare metal servers. The only difference is that with bare metal endpoints you normally statically bind an EPG to a leaf/interface, whereas with virtual machines you bind the VMM domain to the EPG. This allows the Application Policy Infrastructure Controller (APIC) to create a distributed virtual switch (DVS) within vCenter to which hosts can be added. Once the Hypervisor hosts (ESX) are added to the DVS, the EPG becomes available to the virtual machines as a network binding (also known as Port Group).

In this figure, ACI EPG is shown in vCenter as a Virtual Machine Network Port Group.

VMM Integration Configuration

There are a number of steps required when you configure VMM integration. A missed step will result in the configuration not applied to vCenter or VMs being able to pass traffic through the fabric. The high level steps are listed with an explanation as to what each step enables. For full details and procedures, see the configuration guides and/or training NPI.

High Level Procedure

The prerequisite tasks are:

• Create the tenant
• Create the bridge domain (BD)
• Assign appropriate IP subnets to the BD
• Create an Associated Attachable Entity Profile (AEP)
• Create the switch profile
• Create the Interface Policy Group
• Create the Interface Profile

VMM Specific Tasks

1. Create the vCenter domain.

   VM Networking > VM Provider VMware > Create VM Provider

   Here you configure the logical VM Domain which includes the defining vCenter Credentials, the vCenter Host details then binding them together. You also create/assign the VLAN pool which will be used by this VM Domain. The VLAN Pool should include all VLANs that your VMs utilize. The last step is to assign this VMM Domain to the AEP previously created. The AEP should have been previously linked to the Interface Policy Group and Interface Profile respectfully. This allows the VM Domain to be accessible on defined leaf interfaces. Essentially you tell ACI where Hypervisors for this VM Domain connect to the fabric. If you fail to associate the AEP, the leaf will never program itself with the related EPGs. Be sure the vCenter Datacenter name matches exactly.

   This figure shows the VMM Controller Datacenter name in APIC vs. vCenter.

   

2. Bind EPG to the VMM domain.

   Tenants > Tenant X > Application Profiles > Application X > Application EPGs > EPG X > Domains (VMs and Baremetal)

   This task makes the EPG available to the VMM domain, which includes all VMs on the associated DVS hosts. The only option other than choosing the VMM Domain Profile is to set the policy deployment and resolution immediacy. This tells the APIC to either push the EPG and related configuration to the associated AEP leafs immediately, or only when a VM comes online which is associated with that EPG/Port Group (On Demand). On Demand is the default and preferred choice for resource scaling.

   This figure shows how to add a VMM Domain Associate to EPG.

   

   If all the prerequisite tasks were completed, the configuration is complete.

VMM Integration Verification

DVS is created on vCenter. As soon as the VMM domain is created, the DVS should be created in vCenter. In order to verify it was created, from the VI client navigate to Home > Inventory > Networking. The DVS should be present along with the name given to the VMM Provider.

Troubleshoot

If you do not see the DVS created on vCenter, check the faults within the VM Networking > VMM Domain section. The likely culprit is simple Layer 2 connectivity. Ensure the management EPG associated with the vCenter host uses the correct BD. Typically this will be the inband BD.

EPGs programmed on leaf - As long as the DVS is created, and you have assigned VMs to the correct EPG/Port Group and powered up the VMs, you should see both the BD and EPG programmed on the Hypervisor connected leaf switches.

Verify

Connect to the leaf via SSH. You can do this directly or from the APIC. Connecting from the APIC allows you to reference the DNS name rather than determining the leaf IP and use 'tab' to autocomplete the leaf name.

admin@apic2:~> ssh admin@leaf101
Password:

leaf101# show vlan extended

 VLAN  Name                                     Status       Ports
 ---- ----------------------------------------- -------------------------------
 13   --                                        active       Eth1/1, Eth1/3
 21   VMM-Test:VMM-Test-BD                      active       Eth1/25
22   VMM-Test:VMM-Test-App:Test_DB             active       Eth1/25

 VLAN Type  Vlan-mode  Encap
 ---- ----- ---------- -------------------------------
 13   enet  CE         vxlan-16777209, vlan-4093
 21   enet  CE         vxlan-16646014
 22   enet  CE         vlan-305


leaf101#

From here you can see that the BD is correctly programmed on the leaf with internal VLAN 21. For intrafabric transport across this BD, the system uses VXLAN 16646014. The encapsulation VLAN (wire-vlan) is 305. This is the VLAN the host will see on the DVS Port Group. This is one of the VLANs pulled from the attached VLAN pool.

Check Visore for the expected configuration. In this example, the EPG name is 'Test_DB".

Workflow and Troubleshooting Checklist

This figure can be used for a pictorial representation as well as a checklist for VMM integration.