Managing Uplinks for VMM Domains

Managing Uplinks for VMM Domains

Beginning with Cisco Application Policy Infrastructure Controller (APIC) Release 4.2(1), you can specify the number of uplinks for endpoint groups (EPGs) that you associate with virtual domains. You can also rename, add, or delete uplinks. You can also configure failover by defining some uplinks as active and some as standby.

Cisco APIC supports specifying and modifying uplinks for VMware vSphere Distributed Switch (VDS) and Cisco Application Centric Infrastructure (ACI) Virtual Edge Virtual Machine Manager (VMM) domains. Cisco APIC supports failover for EPG uplinks for VDS and Cisco Application Centric Infrastructure (ACI) Virtual Edge when it is in native switching mode.

You can specify from one to 32 uplinks for a VMware VDS or Cisco ACI Virtual Edge. However, you are not required to do so; if you do not, Cisco APIC by default specifies eight uplinks, all active. Specifying more uplinks makes it easier to configure failover. You do not need to rename uplinks, but doing so makes it easier to manage them.

Defining some uplinks as active and others as standby is optional; however, doing so enables failover, ensuring that EPG traffic continues to flow even if some uplinks fail.


Note
You can also manage uplinks by applying different enhanced Link Aggregation Control Protocol (LACP) policies to different distributed virtual switch (DVS) uplink port groups. Enhanced LACP is supported for VMware VDS and for Cisco ACI Virtual Edge. For VMware VDS, see the Enhanced LACP Policy Support section in this guide; for Cisco ACI Virtual Edge, see the section "Enhanced LACP Policy Support" in the Cisco ACI Virtual Edge Configuration Guide .

Prerequisites for Managing Uplinks for VMM Domains

Perform the following tasks before you manage uplinks for endpoint groups (EPGs) for VMware vSphere Distributed Switch (DVS) or Cisco Application Centric Infrastructure (ACI) Virtual Edge Virtual Machine Manager (VMM) domains.

  • Install Cisco Application Policy Infrastructure Controller (APIC) Release 4.2(1) in your fabric.

  • Make the basic configurations that are required for VMware vSphere Distributed Switch (VDS) VMM domain or Cisco ACI Virtual Edge VMM domain.

    This includes creating a tenant, bridge domain, attachable access entity profile (AEP), and at least one endpoint group (EPG). It also includes associating the VMM domain to an EPG,

Workflow for Managing Uplinks for VMM Domains

This section lists the tasks that you must perform to manage uplinks for endpoint groups (EPGs) that you associate with a VMware vSphere Distributed Switch (VDS) or Cisco Application Centric Infrastructure (ACI) Virtual Edge Virtual Machine Manager (VMM) domain.

  1. Fulfill all the prerequisites.

  2. Create a VMM domain for the VMware VDS or Cisco ACI Virtual Edge and specify the number of uplinks as part of the domain creation.

    See Create a VMM Domain for a VMware VDS and Specify the Number of Uplinks or Create a VMM Domain for Cisco ACI Virtual Edge and Specify the Number of Uplinks.

  3. Modify the uplinks: You can rename or delete the uplinks that you specified in the previous task, or you can specify more uplinks.

    You do this by editing the VMM domain. You can also specify the number uplinks at this point if you did not do so when you created a VMM domain.

    See the procedure Edit the VMM Domain and Modify the Uplinks.

  4. Associate an endpoint group (EPG) with the VMM domain and optionally configure active and standby uplinks as part of the association.

    Configuring active and standby uplinks enables failover for uplinks within an EPG associated with the VMM domain.

    See the section Define Uplink Roles to Configure Failover.

Specifying Uplinks for the VMM Domain

You can specify the number of uplinks for a specific Virtual Machine Manager (VMM) domain when you create the Virtual Machine Manager (VMM) domain. You can do so only if you create the domain under the Virtual Networking tab in the Cisco Application Policy Infrastructure Controller (APIC) GUI. You cannot specify uplinks if you use create the domain with the configuration wizard under the Fabric tab.

The procedures for creating a VMM domain under the Virtual Networking differ slightly for VMware vSphere Distributed Switch (VDS) and Cisco Application Centric Infrastructure (ACI) Virtual Edge.

If you have already created a VMM domain, you can still specify the uplinks by editing the VMM domain. The procedure is the same for VMware vSphere Distributed Switch or Cisco ACI Virtual Edge.

Create a VMM Domain for a VMware VDS and Specify the Number of Uplinks

When you create a Virtual Machine Manager (VMM) domain for a VMware vSphere Distributed Switch (VDS), you can specify the number of uplinks that will be configured on the VMware VDS. When you specify uplinks, you determine the ports on the virtual switch that connect to the physical leaf switch.

You specify uplinks in step 4 h of this procedure. If you do not specify uplinks, by default, Cisco APIC specifies eight uplinks, all active.


Note
You cannot configure the uplinks when you create the VMM domain using the configuration wizard under the Fabric tab. However, if you have already created the domain, you can still add uplinks. Skip this procedure and complete the procedure Edit the VMM Domain and Modify the Uplinks.

Before you begin

Fulfill the following prerequisites before you configure a Virtual Machine Manager (VMM) domain for the VMware vSphere Distributed Switch (VDS).

  • All fabric nodes are discovered and configured.

  • In-band (inb) or out-of-band (oob) management has been configured on the Cisco APIC.

  • A VMM is installed, configured, and reachable through the inb or oob management network (for example, VMware vCenter).

  • Ensure that you have enough VLAN IDs in the VLAN pool. If you do not, ports on EPGs might report that no encapsulation is available.

  • Ensure that you have the administrator/root credentials to the VMware vCenter.

  • Create interface and switch profiles.

  • Create an attachable entity profile (AEP).

    During the procedure to create a vCenter domain profile, you are asked to choose or create an AEP. If you want to create an AEP ahead of time, follow the procedure "Create a Global Attachable Access Entity Profile" in the Cisco APIC Basic Configuration Guide.

Procedure

Step 1

Log in to Cisco Application Policy Infrastructure Controller (APIC).

Step 2

Go to Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand VMM Domains, right-click VMware, and then choose Create vCenter Domain.

Step 4

In the Create vCenter Domain dialog box, complete the following steps:

  1. In the Virtual Switch Name field, enter a name.

  2. In the Virtual Switch Area, choose VMware vSphere Distributed Switch.

    Choosing VMware vSphere Distributed Switch creates the VMM domain for the VMware VDS.

  3. From the Associated Attachable Entity Profile drop-down list, create or choose a profile that you created earlier.

    See "Create a Global Attachable Access Entity Profile" in the Cisco APIC Basic Configuration Guide for instructions.

  4. From the VLAN Pool drop-down list, choose Create VLAN Pool and configure the pool using the Create VLAN Pool and Create Ranges dialog boxes.

    Note

     
    If you plan to configure a floating Layer 3 outside network connection (L3Out), the VLAN pool must have a static VLAN range. Also, the VLAN pool must be the same as the VLAN pool of the L3Out domain. For example, both the range for the L3Out domain and the VMM domain must be 200-209.

  5. In the vCenter Credentials area, click the + (plus) icon, and in the Create vCenter Credential dialog box, do the following: Enter the VMware vCenter account profile name in the Name field, the VMware vCenter username in the Username field, enter and confirm the VMware vCenter password, and then click OK.

  6. In the vCenter area, click the + (plus) icon, and in the Add vCenter Controller dialog box, do the following: Enter the VMware vCenter controller name, the VMware vCenter host name or IP address, the DVS version, data center name (which must match the data center name configured in VMware vCenter), select the credentials created in the previous step, and then click OK.

  7. In the Create vCenter Domain dialog box, click OK.

    In the VMware work pane, you should see the newly created VMM domain, which will be pushed to VMware vCenter.

    Note

     
    Perform the following step if you plan to specify the number of uplinks. This step is optional.

  8. From the Number of Uplinks drop-down list, choose the number of uplinks for the virtual switch uplink port group.

    You can associate from one to 32 uplinks to the virtual switch uplink port group. This step is optional; if you do not choose a value, eight uplinks are associated with the port group by default.

    You can name the uplinks after you finish creating the VMM domain. You can also configure failover for the uplinks when you create or edit the VMM domain association for an EPG.

What to do next

You can do the following:

Create a VMM Domain for Cisco ACI Virtual Edge and Specify the Number of Uplinks

When you create a Virtual Machine Manager (VMM) domain for a Cisco Application Centric Infrastructure (ACI) Virtual Edge, you can specify the number of uplinks for endpoint groups (EPGs) that you associate with the domain. When you specify uplinks, you determine the ports on the virtual switch that connect to the physical leaf switch with the EPGs.

You specify the uplinks in step 4 h of this procedure. If you do not specify uplinks, by default, Cisco APIC specifies eight uplinks, all active, to an EPG.


Note
You cannot configure the uplinks when you create the Cisco ACI Virtual Edge VMM domain using the configuration wizard under the Fabric tab. However, if you have already created the Cisco ACI Virtual Edge, you can still add uplinks. See the procedure Edit the VMM Domain and Modify the Uplinks.

Before you begin

  • Ensure that the multicast IP address pool has enough multicast IP addresses to accommodate the number of EPGs that will be pushed to the VMware vCenter domain. You can add more IP addresses to a multicast address pool that is already associated with a VMware vCenter domain at any time.

  • Ensure that you have enough VLAN IDs in the VLAN pool. If you do not, ports on EPGs might report that no encapsulation is available.

  • Ensure that VMware vCenter is installed, configured, and reachable through the in-band/out-of-band management network.

  • Ensure that you have the administrator/root credentials to the VMware vCenter.

  • Create interface and switch profiles. See the section "Create Port Channel Switch and Interface Profiles" in Appendix B of the Cisco ACI Virtual Edge Installation Guide for instructions.

  • Create an attachable entity profile (AEP).

    During the procedure to create a vCenter domain profile, you are asked to choose or create an AEP. If you want to create an AEP ahead of time, follow the procedure "Configuring an Attachable Entity Profile Using the GUI" in the Cisco ACI Virtual Edge Configuration Guide .


    Note
    Enable the infrastructure VLAN within the AEP assigned to the Cisco ACI Virtual Edge VMM domain. Do this regardless of whether you create the AEP before or during VMware vCenter domain profile creation. In the Create Attachable Access Entity Profile dialog box, check the Enable Infrastructure VLAN check box.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Go to Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand VMM Domains, right-click VMware, and then choose Create vCenter Domain.

Step 4

In the Create vCenter Domain dialog box, complete the following steps:

  1. In the Virtual Switch Name field, enter a name.

  2. In the Virtual Switch Area, choose Cisco AVE.

    Choosing Cisco AVE creates the VMM domain for Cisco ACI Virtual Edge.

    Note

     

    Perform the following two substeps if you want to use VMware vSphere Proactive HA. Cisco APIC tells VMware vCenter to quarantine a host with a non-working Cisco ACI Virtual Edge and move the VMs to a host with a working Cisco ACI Virtual Edge. The feature is not available for Cisco ACI Virtual Edge when it is part of Cisco ACI vPod.

    Enable Proactive HA in VMware vCenter. See the appendix "Improving Cisco ACI Virtual Edge Availability with VMware vSphere Proactive HA" in the Cisco ACI Virtual Edge Installation Guide .

  3. With the AVE Time Out Time (seconds) selector, choose the time period to trigger VMware vCenter to quarantine a host with a nonworking Cisco ACI Virtual Edge and move VMs from the host.

    You can choose any value between 10 and 300 seconds, inclusive. The default is 30 seconds.

  4. Check the Host Availability Assurance check box.

    Checking the check box creates a VMware Proactive HA object in VMware vCenter. The object enables VMware vCenter to quarantine a host with a nonworking Cisco ACI Virtual Edge and move VMs from the host.

    Note

     
    Activation of VMware Proactive HA in vCenter is required before a host with non-working Cisco ACI Virtual Edge can be quarantined.

  5. In the Switching Preference area, choose No Local Switching or Local Switching.

    For information about switching preferences, see the section "What Cisco ACI Virtual Edge Is" in the Overview chapter of the Cisco ACI Virtual Edge Installation Guide .

    Note

     
    If you choose No Local Switching, you can use only VXLAN encapsulation.

  6. If you chose Local Switching in Step 4f, in the Default Encap Mode area, choose a mode.

    You can choose VLAN mode or VXLAN mode. You can use both encapsulation methods within the same VMM domain. See the section "Mixed-Mode Encapsulation Configuration" in the Cisco ACI Virtual Edge Configuration Guide .

  7. From the Associated Attachable Entity Profile drop-down list, create or choose a profile that you created earlier.

    See "Configuring an Attachable Entity Profile Using the GUI" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

  8. From the VLAN Pool drop-down list, choose or create a VLAN pool.

    If Cisco ACI Virtual Edge will be deployed in mixed-mode or VLAN mode, create two VLAN pools: one for primary encapsulation and one for private VLAN implementation. The role for the private VLAN pool must be internal. If Cisco ACI Virtual Edge will be deployed in VXLAN mode, only a private VLAN pool is necessary.

  9. In the AVE Fabric-Wide Multicast Address field, enter an address.

  10. From the Pool of Multicast Addresses (one per-EPG) drop-down list, choose or create a pool.

  11. In the vCenter Credentials area, click the + (plus) icon, and in the Create vCenter credential dialog box, do the following: Enter the VMware vCenter account profile name in the Name field, the VMware vCenter username in the Username field, enter and confirm the VMware vCenter password, and then click OK.

  12. In the vCenter area, click the + (plus) icon, and in the Create vCenter Controller dialog box, do the following: Enter the VMWare vCenter controller name, the VMWare vCenter host name or IP address, the DVS version, data center name (which must match the data center name configured in VMware vCenter), select the credentials created in the previous step, and then click OK.

    You can choose a DVS version 5.5 or later.

  13. In the Create vCenter Domain dialog box, click Submit.

    In the VMware work pane, you should see the newly created VMM domain, which will be pushed to VMware vCenter.

    Disregard the options for choosing a port channel mode, vSwitch policy, interface controls, and firewall mode. You can configure Distributed Firewall later; see the instructions in the Cisco ACI Virtual Edge Configuration Guide .

  14. From the Number of Uplinks drop-down list, choose the number of uplinks for the virtual switch uplink port group.

    You can associate from one to 32 uplinks to the virtual switch uplink port group. This step is optional; if you do not choose a value, eight uplinks are associated with the port group by default.

    You can name the uplinks after you finish creating the VMM domain. You can also configure failover for the uplinks when you create or edit the VMM domain association for an EPG.

What to do next

Edit the VMM Domain and Modify the Uplinks

You can modify the uplinks that you previously specified for the virtual switch uplink port group by editing the Virtual Machine Manager (VMM) domain. Renaming, adding , or deleting uplinks is supported for VMware vSphere Distributed Switch (VDS) or Cisco Application Centric Infrastructure Virtual Edge.

You do not need to rename uplinks; if you do not, they use the default names assigned by Cisco Application Policy Infrastructure Controller (APIC). The default names are uplink1, uplink2, and so on. Renaming uplinks does not change the unique IDs of the uplinks but can help you organize them by function.


Note
You can also use this procedure to specify uplinks if you did not do so when you created the VMM domain.

Before you begin

You must have created a VMM domain for a VMware VDS or Cisco ACI Virtual Edge.

Procedure

Step 1

Log in to Cisco APIC.

Step 2

Go to Virtual Networking > Inventory.

Step 3

In the Inventory navigation pane, expand the VMM Domains and VMware folders, and then choose the VMM domain.

Step 4

In the central Domain work pane, do the following:

Option Description
If you want to... Then...
Specify uplinks

  1. Make sure that the Create Uplinks check box is checked.

  2. From the Number of Uplinks drop-down list, choose the number of uplinks to specify.

    You can specify up to 32 uplinks. If you do not specify uplinks, Cisco APIC specifies eight by default, all active.

Rename specified uplinks

  1. In the Name of Uplinks area, click the + (plus) icon.

  2. From the Uplink ID field, choose an uplink to name.

  3. In the Uplink Name field, enter a name for the uplink.

  4. Repeat the process to name other uplinks.

  5. Click Update.

Add uplinks

From the Number of Uplinks drop-down list, choose the number of uplinks to specify

You can create from one to 32 uplinks.
Delete uplinks

  1. In the Name of Uplinks area, click the + (plus) icon.

  2. In the uplink table, choose an uplink, and then click the trash can icon.

  3. Repeat the process to name other uplinks that you want to delete.

Note

 
You cannot delete uplinks after you have defined some as active and some as standby for an EPG associated with a VMM domain. Defining uplinks as active or standby enables failover for the uplinks in the EPG.

Step 5

Click Submit.

Define Uplink Roles to Configure Failover

You can define some uplinks in an endpoint group (EPG) as active and some links as standby. Doing so enables failover for the uplinks within the EPG, which you associate with a Virtual Machine Manager (VMM) domain.

If an active link fails, another active link takes over. If there are no active links available, standby links take over.

You define links as active or standby when you create a VMM domain association for the EPG. If you have already associated the EPG with the VMM domain, you can configure failover for the EPG uplinks by editing the VMM domain association. These procedures are the same for VMware vSphere Distributed Switch and Cisco Application Centric Infrastructure (ACI) Virtual Edge.


Note
In addition to defining uplink roles, you can choose a load-balancing mode when you configure a port channel policy. Beginning in Cisco Application Policy Infrastructure Controller (APIC) Release 4.2(1), you can choose explicit failover as a mode. Choosing explicit failover, a non-load balancing mode, ensures that uplinks fail over in the order that you define when you create the EPG-VMM domain association.

Associate an EPG with a VMM Domain and Define Uplink Roles

You can enable failover for the uplinks in the endpoint group (EPG) that you associate with a Virtual Machine Manager (VMM) domain. You do so while associating the EPG with the domain by specifying which of the uplinks are active uplinks and which are standby uplinks.

Before you begin

You must have completed the following tasks:

  • Created a VMM domain for a VMware vSphere Distributed Switch (VDS) or a Cisco Application Centric Infrastructure (ACI) Virtual Edge

  • Created a tenant, application profile, and at least one EPG.

Procedure

Step 1

Log in to Cisco Application Policy Infrastructure Controller (APIC).

Step 2

Go to Tenants > tenant .

Step 3

In the tenant navigation pane, expand the following: tenant > Application Profiles > application_profile .

Step 4

Right-click the Domains (VMs and Bare-Metals) folder and choose Add VMM Domain Association (VMs and Bare-Metals).

Step 5

In the Add VMM Domain Association dialog box, perform the following steps:

  1. From the VMM Domain Profile drop-down list, choose the domain.

  2. Configure the association as is appropriate for your setup.

  3. In the Active Uplinks Order field, enter the IDs of the uplinks that you want to be active, using commas but no spaces to separate the uplinks.

    The order decides the order in which active uplinks take over for a failed uplink.

    Note

     
    If you configure uplink failover, you cannot rename, add, or delete the uplinks. However, you can edit the failover. For example, you can change the uplinks that are active or standby.

  4. In the Standby Uplinks field, enter the IDs of the uplinks that you want to be standby, using commas but no spaces separate the uplinks.

    Note

     
    Uplinks that you specify but do not define as active or standby are classed as unused. To make an active or standby link unused, remove it from the active or standby list. However, you cannot make all uplinks unused. If you do not specify any uplinks, all available uplinks are classed as active.

  5. Click Submit.

Edit the EPG-Domain Association and Define Uplink Roles

If the endpoint group (EPG) is already associated with a Virtual Machine Manager (VMM) domain, you can still define some uplinks as active and some as standby by editing the VMM domain association. Defining uplink roles enables failover for the uplinks in the EPG.

Before you begin

You must have associated an EPG to a VMware vSphere Distributed Switch (VDS) or a Cisco Application Centric Infrastructure (ACI) VMM domain.

Procedure

Step 1

Log in to Cisco Application Policy Infrastructure Controller (APIC).

Step 2

Go to Tenants > tenant .

Step 3

In the tenant navigation pane, expand the following: tenant > Application Profiles > application_profile .

Step 4

Choose the Domains (VMs and Bare-Metals) folder.

Step 5

In the Domains (VMs and Bare-Metals) central work pane, right-click the domain and then choose Edit VMM Domain Association.

Step 6

In the Edit VMM Domain Association dialog box, complete the following steps:

  1. In the Active Uplinks Order field, enter the IDs of the uplinks that you want to be active, using commas but no spaces to separate the uplinks.

    The order decides the order in which active uplinks take over for a failed uplink.

    Note

     
    If you configure uplink failover, you cannot rename, add, or delete the uplinks. However, you can edit the failover. For example, you can change the uplinks that are active or on standby.

  2. In the Standby Uplinks field, enter the IDs of the uplinks that you want to be standby, using commas but no spaces to separate the uplinks.

    Note

     
    Uplinks that you specify but do not define as active or standby are classed as unused. To make an active or standby link unused, remove it from the active or standby list. However, you cannot make all uplinks unused.

  3. Make any other changes as required for your setup.

  4. Click OK.