Cisco ACI Virtual Edge Installation

This chapter describes installation for Cisco ACI Virtual Edge, including prerequisites and installation methods.

About Cisco ACI Virtual Edge Installation

Cisco ACI Virtual Edge installation consists of a series of tasks on the Cisco APIC, and VMware vCenter. You can then use one of three methods to deploy Cisco ACI Virtual Edge on ESXi hosts:

  • Cisco ACI vCenter plug-in

  • VMware PowerCLI (for Windows platforms)

  • Python script


Note

Do not use the vSphere (thick) Client to install Cisco ACI Virtual Edge or modify its vApp properties. Use only the Cisco ACI vCenter plug-in, the VMware Power CLI, or a Python script to install Cisco ACI Virtual Edge. Use only the vSphere Web Client to modify Cisco ACI Virtual Edge vApp properties.

Note

When you deploy the Cisco ACI Virtual Edge VM on the ESXi hosts, OpFlex automatically comes online. Do not attach VMkernel ports to the Infra port group, as was done for OpFlex for Cisco AVS.

The following sections provide information about prerequisites and installation methods. For information about migrating from Cisco AVS to Cisco ACI Virtual Edge, see the chapter Migration from Cisco AVS to Cisco ACI Virtual Edge in this guide. For information about migrating from VMware VDS to Cisco ACI Virtual Edge, see the chapter Migration from VMware VDS to Cisco ACI Virtual Edge in this guide.


Note

Although you can install multiple Cisco ACI Virtual Edge VMs on the same host (one for each Cisco ACI Virtual Edge VMM domain), we recommend that you install only one Cisco ACI Virtual Edge VM per host.

Best Practices for Cisco ACI Virtual Edge Deployments

Follow these best practices to minimize traffic loss and provide more availability due to hardware failure.

  • For ACI Virtual Edge deployments, ensure to configure ProActive HA in the cluster on vCenter, and on the VMM domain on APIC.

  • For ProActive HA deployments for ACI Virtual Edge, ensure the DRS Setting on the vCenter to disable the For Availability, distribute a more even number of virtual machines across hosts option.

See the Improving Cisco ACI Virtual Edge Availability with VMware vSphere Proactive HA chapter, for more details about Enabling vSphere Proactive HA.

Default Port-Groups

When you create a Cisco Application Centric Infrastructure (ACI) Virtual Edge, VMware vCenter automatically creates several port-groups:

  • ave-external-vxlan-1 and ave-external-vxlan-2: The Cisco ACI Virtual Edge virtual machine (VM) uses these port-groups to send and receive VXLAN traffic to and from outside the host. VXLAN traffic is distributed between these two ports based on the incoming VM interface.

  • ave-internal-1 and ave-internal-2: The Cisco ACI Virtual Edge VM uses these port-groups to send and receive PVLAN traffic to and from VMs internal within the distributed virtual switch (DVS). The internal VLAN blocks are distributed evenly between these two port-grouups to load-balance the internal traffic.

  • ave-external-vlan: The Cisco ACI Virtual Edge VM uses this port-group to send and receive VLAN traffic to and from outside the host. It enables the VLANs used for the VLAN mode endpoint groups (EPGs) associated with the VMM domain. The VLANs might include the VLAN assigned to "ave-ctrl" EPG, if it is in VLAN mode.

  • infra: VMs use this special port-group to receive ERSPAN traffic originated from another Cisco ACI Virtual Edge. The port-group is in native mode, and incoming Encapsulated Remote Switched Port Analyzer (ERSPAN) traffic bypasses Cisco ACI Virtual Edge and is forwarded by the DVS.

Cisco ACI Virtual Edge Installation Workflow

This section provides a high-level description of the tasks required to install the Cisco ACI Virtual Edge.

  1. Fulfill all the prerequisites, which include tasks in the Cisco Application Policy Infrastructure Controller (APIC), and vCenter. See the section Prerequisites for Installing Cisco ACI Virtual Edge.

  2. Download the Cisco ACI Virtual Edge Open Virtualization Format (OVF) file from Cisco.com and then upload it to the vCenter content library. You can use the vCenter plug-in, the vCenter power CLI, or a Python script. See one of the following sections for instructions:

  3. Deploy Cisco ACI Virtual Edge on the ESXi hosts. You can use one of three methods. See the following sections for instructions:

  4. Make sure that the that the interface that is used to communicate with Cisco ACI Virtual Edge (kni0) has a virtual tunnel endpoint (VTEP) IP address and verify that OpFlex is up.

    See the section Verify the Cisco ACI Virtual Edge Deployment in this guide.


Note

To ensure a higher level of availability, we recommend that you deploy Cisco ACI Virtual Edge on a local data store.

Prerequisites for Installing Cisco ACI Virtual Edge

Perform the following tasks before you install Cisco Application Centric Infrastructure Virtual Edge:

Storage and Memory

You need at least 30 GB of storage and 4 GB of memory.

Cisco ACI Fabric and Cisco APIC

  • Make sure that Cisco Application Policy Infrastructure Controller (APIC) is set up correctly. See the Cisco APIC Getting Started Guide and Cisco APIC Basic Configuration Guide, on Cisco.com for instructions on how to configure Cisco APIC for the first time.

  • Make sure that all switches are registered and that the Cisco ACI fabric is up-to-date. See Cisco Application Centric Infrastructure Fundamentals and the Cisco APIC Getting Started Guide on Cisco.com for instructions.

  • Make sure that the Cisco ACI fabric is registered inside the vCenter plug-in. See "Connecting vCenter Plug-in to your ACI Fabric" in the chapter "Cisco ACI vCenter Plug-in" in the Cisco ACI Virtualization Guide.

VMM Domain

Create a new vCenter VMM domain and interface and switch profiles for Cisco ACI Virtual Edge.

We recommend that you use the unified configuration wizard to perform these tasks. See the procedure Create vCenter Domain, Interface, and Switch Profiles Using the GUI in this guide. However, you may need to configure separate, more detailed policies. If so, see the appendix Alternate Procedures for Creating vCenter Domain, Interface, and Switch Profiles in this guide.

Hosts

  • Add one or more ESXi hosts and their PNICs to the new Cisco ACI Virtual Edge distributed virtual switch (DVS) in using vSphere Web Client on VMware vCenter.

  • If the host belongs to a Distributed Resource Scheduler (DRS) cluster that already has VMs running on Cisco ACI Virtual Edge, put the host in maintenance mode before you add the Cisco ACI Virtual Edge DVS to it. Starting the installation with the host in maintenance mode prevents the DRS from migrating VMs to the other hosts before the Cisco ACI Virtual Edge VM is fully ready.

  • If the host belongs to a DRS cluster, make sure that the Enhanced VMotion Compatibility (EVC) mode for the DRS cluster is set to Nehalem or higher.

  • When using VMware vSphere Hypervisor (ESXi) 6.5 U1, update the Intel X710 port adapter driver to 1.8.6 or later with firmware 6.01 or later before adding hosts to the Cisco ACI Virtual Edge in VXLAN mode with Cisco Discovery Protocol (CPD) enabled. If you do not update the port adapter driver, you may see the VMware purple diagnostic screen.

VXLAN Encapsulation

When connecting the Cisco ACI Virtual Edge using VXLAN encapsulation, set the maximum transmission unit (MTU) value equal to or greater than 1600 on all intermediate devices on the path between the Cisco ACI fabric and the Cisco ACI Virtual Edge. These include FI switches and UCS-B. However, to optimize performance, set the MTU to the maximum supported size that all intermediate devices on the path between the Cisco ACI fabric and the Cisco ACI Virtual Edge support.

VMware vCenter

  • In order to use the Cisco ACI Virtual Edge management tools, we recommend that you use vCenter 6.0 Update 3 or later. These tools include the ACI vCenter plug-in, the VMware PowerCLI, and Python scripts.

  • If you plan to install Cisco ACI Virtual Edge using the VMware PowerCLI, synchronize the clocks for the vCenter Server, any Active Directory domain controllers, and the host making single-sign-on connection requests. If the clocks are not synchronized, you may encounter problems when deploying Cisco ACI Virtual Edge using the VMware PowerCLI tool.

    For details, see the knowledge base article "Calling the SSOConnection SDK reports the exception: Client received SOAP Fault from server: The time now <timestamp> does not fall in the request lifetime interval extended with clock tolerance of 600000 ms (2125193)" on the VMware website.

Remote Leaf Deployment

If you plan to install Cisco ACI Virtual Edge in a remote leaf deployment, enable DSCP class-cos translation policy for L3 traffic as recommended in the section "Recommended QOS configuration for Remote leaf" of the Cisco ACI Remote Leaf Architecture White Paper on Cisco.com.

Cisco APIC Settings Configuration

The following sections describe how to configure the Cisco ACI Virtual Edge and the VMware ESXi hypervisor with the Cisco APIC:

  1. vCenter Domain, Interface, and Switch Profile Creation

  2. Interface and Switch Profile Guidelines and Prerequisites

  3. vCenter Domain Profile Guidelines and Prerequisites

  4. Create vCenter Domain, Interface, and Switch Profiles Using the GUI

vCenter Domain, Interface, and Switch Profile Creation

Before you can install the Cisco ACI Virtual Edge, you must create vCenter domain, interface, and switch profiles. We recommend that you perform these tasks in the united configuration wizard in the Cisco APIC. See the procedure Create vCenter Domain, Interface, and Switch Profiles Using the GUI in this guide.

Understand and follow the guidelines in this section before proceeding with the tasks.

Alternate Procedures

If you want to configure a FEX profile or detailed interface, switch, or vCenter domain profiles, you can find instructions in Alternate Procedures for Creating vCenter Domain, Interface, and Switch Profiles in this guide.

Firewall Considerations

If you use the recommended united configuration wizard, the Cisco APIC automatically creates a firewall policy, which can be modified later. If you instead use the alternate procedures to create interface, switch, or vCenter domain profiles, you will need to create a firewall policy manually. Follow the instructions in the Distributed Firewall section of the Cisco ACI Virtual Edge Configuration Guide.

Interface and Switch Profile Guidelines and Prerequisites

Follow these guidelines and fulfill the prerequisites when creating interface and switch profiles for your Cisco ACI Virtual Edge.

Guidelines for Creating Interface and Switch Profiles

The Cisco ACI Virtual Edge supports port channel (PC), virtual port channel (VPC), MAC Pinning, and FEX interface policies.

  • If there is a Layer 2 network between the leaf switch and the Cisco ACI Virtual Edge vSphere host, configure the interface policy on the interfaces that are connected to the Layer 2 network.

  • The number of links and leafs that you use determine whether you configure a PC or a VPC policy for the Cisco ACI Virtual Edge:

    • If you are using multiple links between one leaf and an ESXi host, you must configure a PC policy.

    • If you are using multiple links between multiple leafs and an ESXi host, you must configure a VPC policy.

  • Follow these guidelines for choosing a LACP policy:

    • Choose LACP (Active or Passive) if the uplinks from the Cisco ACI Virtual Edge (vSphere host) are directly connected to the leaf switches and you want to use or turn on the LACP channeling protocol.

    • Choose Static Channel - Mode On if the uplinks form the Cisco ACI Virtual Edge are directly connected to the leaf switches but you do not want to use the LACP channeling protocol.

    • Choose MAC Pinning if the uplinks from the Cisco ACI Virtual Edge will not be channeled together and will operate as separate links.


      Note

      Do not use MAC pinning with a direct connection to a VPC leaf pair. Instead, use Link Aggregation Control Protocol (LACP) or enhanced LACP to provide redundancy and reliability. Using MAC pinning with a direct connection leads to traffic loss when peer leaf switches are rebooted. Use MAC Pinning only where virtual port channel (VPC) cannot be supported, such as for Cisco UCS Fabric Interconnects with southbound interfaces.
  • Follow these guidelines for choosing a vSwitch port group for the management interface:

    Ensure that the vSwitch port group that you choose for the Cisco ACI Virtual Edge management interface can provide at least IPv4 addresses through DHCP or the vCenter IP pool. You can configure an additional IPv6 address for the vSwitch port group for the management interface; however, you cannot configure it only with an IPv6 address.


    Note

    The Cisco ACI vCenter plug-in does not support configuration of a static IP address. However, you can configure a static IP address by using the VMware PowerCLI or Python script. See the sections Cisco ACI Virtual Edge Installation Using the VMware PowerCLI and Cisco ACI Virtual Edge Installation Using Python in this guide. Alternatively, you can configure a static IP address in VMware vCenter. See the section Configuring a Static IP Address in VMware vCenter or the section Configure a Static IP Address Using the HTML5 Version of the VMware vSphere Client in this guide.
Prerequisites for Creating Interface and Switch Profiles

Verify that the leaf switch interfaces are physically connected to the ESXi hypervisor. Or, if you are using a Layer 2 device, verify that the leaf is physically connected to the Layer 2 device.

vCenter Domain Profile Guidelines and Prerequisites

You must create a new vCenter domain profile before you can install Cisco ACI Virtual Edge. You cannot convert an existing vCenter domain profile.

Guidelines for Creating a VMware vCenter Domain Profile

You can create multiple data centers and DVS entries under a single domain. However, you can have only one Cisco ACI Virtual Edge assigned to each data center.

You can use IPv6 when creating a VMM domain if the vCenter and ESXi host management are IPv6-enabled.

Prerequisites for Creating a VMware vCenter Domain Profile

Ensure that the multicast IP address pool has enough multicast IP addresses. You must accommodate the number of EPGs to be published to the VMware vCenter domain. You can add more IP addresses to a multicast address pool that is already associated with a VMware vCenter domain at any time.

Ensure that you have enough VLAN IDs. If you do not, ports on endpoint groups (EPGs) might report that no encapsulation is available.

vCenter must be installed, configured, and reachable through the in-band/out-of-band management network.

You must have the administrator/root credentials to the vCenter.

Create vCenter Domain, Interface, and Switch Profiles Using the GUI


Note

If you want to choose a delimiter for the VMware portgroup name when you create a vCenter domain, you cannot do so in this procedure. You also cannot use this procedure if you want to take advantage of the VMware vSphere Proactive HA feature. This procedure uses a configuration wizard that enables you to configure a vCenter domain, interface, and switch profiles.

Instead, you must create the vCenter domain separately. The delimiter option appears in the Create vCenter Domain dialog box. The Create vCenter Domain dialog box also includes an option to create a VMware Proactive HA object in VMware vCenter. It also includes an option to set the time periods before Proactive HA is triggered. See the procedure Create a VMM Domain Profile for Cisco ACI Virtual Edge in this guide.


Before you begin

Before you create a vCenter domain profile, you must establish connectivity to an external network using in-band management network on the Cisco APIC.

Procedure

Step 1

Log in to the Cisco APIC.

Step 2

On the menu bar, click Fabric > Access Policies.

Step 3

In the Policies Navigation pane, lick Quick Start, and then in the central pane, click Configure Interfaces, PC, and VPC.

Step 4

In the Configure Interfaces, PC, and VPC dialog box, expand Configured Switch Interfaces, click the green + icon, and then perform the following steps:

  1. In the Select Switches to Configure Interfaces area, make sure that the Quick radio button is selected.

  2. From the Switches drop-down list, choose the appropriate leaf ID.

    In the Switch Profile Name field, the switch profile name automatically appears.

  3. Click the green + icon again.

    The Configure Interfaces, PC, and VPC dialog box displays a wizard that enables you to configure vCenter domain, interface, and switch profiles.

Step 5

In the wizard, perform the following actions:

  1. In the Interface Type area, choose the appropriate radio button.

    PC and VPC are the only valid options for Cisco ACI Virtual Edge deployment. See the section Interface and Switch Profile Guidelines and Prerequisites in this guide.

  2. In the Interfaces field, enter the interface or interface range for your vSphere hosts.

    Once you enter the interface or interface range, the wizard enters a name in the Interface Selector Name field.

  3. In the Interface Policy Group area, choose the Create One radio button.

    Note 
    This procedure assumes that you are creating interface and switch policies and creating a vCenter domain rather than using existing ones. If you choose the Choose One radio button, you will not be able to create policies in the wizard.
  4. From the CDP Policy or the LLDP Policy drop-down list, create a policy.

    Note 
    • If you use a Cisco Unified Computing System (UCS) server, create two policies. Create one policy to enable a Cisco Discovery Protocol (CDP) policy and a second policy to disable Link Layer Discovery Protocol (LLDP).

    • CDP and LLDP policies are disabled by default. You can enable them in the configuration wizard. Enable CDP or LLDP policies in the Interface Policy Group area to enable them on Cisco ACI Virtual Edge and other switches in the fabric. If you want to enable CDP or LLDP only on Cisco ACI Virtual Edge, enable them in the vSwitch Policy area of the configuration wizard.

  5. From the Link Level Policy drop-down list, choose a link level policy or create one.

    The link level policy specifies the speed of the physical interface. If you do not choose a link level policy, the speed defaults to 10 Gbps.
  6. In the Port Channel Policy drop-down list, choose Create Port Channel Policy.

  7. In the Create Port Channel Policy dialog box, enter a name for the policy, choose a mode, and then click Submit.

    Choose the same policy mode that is on the ESXi server. For example, if the server does not support LACP, you can choose Static Channel - Mode On or MAC Pinning. Other fields in the dialog box are optional.

  8. In the Attached Device Type area, choose AVE VLAN Hosts or AVE VXLAN Hosts.

    Note 
    If the hypervisors are directly connected to leaf switches, you can use either VLAN or VXLAN. (Cisco UCS blade servers, where Fabric Interconnects are connected to the fabric, are considered to be directly connected.) However, if the hypervisors are not directly connected to leaf switches, you must use VXLAN. For more information, see the Cisco ACI Virtual Edge section.
  9. In the Domain area, make sure that the Create One radio button is chosen.

    Use the Create One option to create a new VMM domain for an interface or switch profile, as you do in this procedure. Use the Choose One button to create an interface or switch profile for a new host that you want to make part of an existing VMM domain.

  10. In the Domain Name field, enter the domain name.

    Note 
    When you create the VMM domain, you choose VLAN or VXLAN encapsulation, depending on the attached device type that you chose in Step 5 h. However, you can configure a single VMM domain to use VLAN and VXLAN encapsulation. After you finish installing the Cisco ACI Virtual Edge, you can enable mixed encapsulation mode. See the section "Mixed-Mode Encapsulation Configuration" in the Cisco ACI Virtual Edge Configuration Guide.
  11. Complete one of the following series of steps:

    Mandatory: If you use Cisco ACI Virtual Edge and you deploy it in mixed-mode or VLAN mode, create a single VLAN pool with two VLAN encapsulation blocks. One will be used for primary encapsulation, and one will be used for private VLAN implementation.

    If in Step 5 h you chose...

    Then...

    AVE VLAN Hosts

    1. In the VLAN area, make sure that the Create One radio button is chosen.

    2. In the VLAN Range field, enter the VLAN range as appropriate.

      Note 
      Do not define a range that includes the reserved VLAN ID for the infrastructure network because that VLAN is for internal use.

      The VLAN range is for external or on-the-wire encapsulations. It is used for allocating VLANs for each EPG assigned to the domain. The VLANs are used when packets are sent to or from leafs.

    3. In the Internal VLAN Range field, enter a range.

      The internal VLAN range is used for private VLAN allocations in the internal vSwitch by the Cisco ACI Virtual Edge. The VLANs are not seen outside the ESX host or on the wire.

      Note 
      If you use Cisco ACI Virtual Edge and you deploy it in mixed-mode or VLAN mode, create a single VLAN pool with two VLAN encapsulation blocks. One will be used for primary encapsulation, and one will be used for private VLAN implementation.

    AVE VXLAN Hosts

    1. In the VLAN area, make sure that the Create One radio button is chosen.

    2. In the Internal VLAN Range field, enter a range.

    3. In the Fabric Multicast Address field, enter a multicast address, such as 225.1.1.1.

    4. In the Pool of Multicast Address Ranges field, create a new multicast pool or choose an existing one.

      Note 
      The multicast address that is configured in Step 3 must not overlap with the ranges that are configured in Step 4.
       
    5. In the Local Switching area, choose True or False.

      With local switching, traffic within an EPG does not go to the leaf. So if you choose local switching, you may not see some traffic counters. If you want to see all intra-EPG traffic, choose.False See the section What Cisco ACI Virtual Edge Is for additional information about Local Switching and No Local switching modes.

  12. (Optional) From the Security Domains drop-down list, choose or create a security domain.

  13. In the vCenter Login Name field, enter the vCenter Administrator/root username.

  14. In the Password field, enter the vCenter Administrator/root password.

  15. In the Confirm Password field, reenter the password.

Step 6

Click the + icon to expand vCenter, and in the Create vCenter Controller dialog box, perform the following actions:

Note 
You can create multiple vCenter controllers in the same domain. If you want to create more vCenter controllers, repeat the substeps for step 6 for each new vCenter controller.
  1. In the Name field, enter a name to refer to the vCenter domain.

    The name does not need to be the same as the vCenter domain name; you can use the vCenter hostname.

  2. In the Host Name (or IP Address) field, enter the host name or IP address.

    If you use the hostname, you must already have configured a DNS policy on Cisco APIC. If you do not have a DNS policy configured, enter the IP address of the vCenter server.

  3. From the DVS Version drop-down list, choose a DVS version.

    The DVS version that you choose represents the minimum ESXi version of the host that you can add to the virtual switch. So if you choose DVS version 6.0, you can add or manage hosts of ESXI version 6.0 and later.

    Note 
    Cisco ACI Virtual Edge supports DVS and ESXi versions 6.0 and later.
  4. In the Datacenter field, enter the data center name.

    The name that you enter for Datacenter must match exactly the name in vCenter. The name is case-sensitive.
  5. Click OK.

    Note 
    For the following three steps, if you do not specify port channel, vSwitch, or interface control policies, the same interface policy that you configured earlier in this procedure will take effect for the vSwitch.
Step 7

In the Configure Interface, PC, And VPC dialog box, from the Port Channel Mode drop-down list, choose a mode.

Note 
  • Choose MAC Pinning if you have a Unified Computing System (UCS) Fabric Interconnect (FI) between the top-of-rack switch and the Cisco ACI Virtual Edge.

  • Do not use MAC pinning with a direct connection to a VPC leaf pair. Instead, use Link Aggregation Control Protocol (LACP) or enhanced LACP to provide redundancy and reliability. Using MAC pinning with a direct connection leads to traffic loss when peer leaf switches are rebooted. Use MAC Pinning only where virtual port channel (VPC) cannot be supported, such as for Cisco UCS Fabric Interconnects with southbound interfaces.

Step 8

In the vSwitch Policy area, choose a policy.

Step 9

In the Interface Controls area, choose BPDU Guard, BPDU Filter, or both.

See the section "BPDU Features" in the Cisco ACI Virtual Edge Configuration Guide for information about BPDU Guard and BPDU Filter.

Step 10

From the Firewall drop-down list, choose Learning, Enabled or Disabled mode.

Learning mode, the default, should be used only when upgrading to Cisco ACI Virtual Edge from a version of Cisco AVS that does not support Distributed Firewall. Otherwise, Distributed Firewall should be in Enabled mode. You can change the Distributed Firewall mode later. See the chapter "Distributed Firewall" in the Cisco ACI Virtual Edge Configuration Guide.
Step 11

Disregard the NetFlow Exporter Policy option.

Step 12

Click Save, click Save again, and then click Submit.

Step 13

Verify the new domain and profiles, by performing the following actions:

  1. On the menu bar, choose Virtual Networking > Inventory.

  2. In the navigation pane, expand VMM Domains > VMware > Domain_name > Controllers, and then choose the vCenter.

In the work pane, under Properties, view the virtual machine manager (VMM) domain name to verify that the controller is online. In the work pane, the vCenter properties are displayed including the operational status. The displayed information confirms that connection from the Cisco APIC to the vCenter server is established, and the inventory is available.

Add ESXi Hosts and PNICs Using the VMware vSphere Client HTML5 GUI

Before you can install Cisco Application Centric Infrastructure (ACI) Virtual Edge, you must add one or more ESXi hosts and their respective physical NICs (PNICs) to the DVS where you deploy Cisco ACI Virtual Edge.


Note

When you add hosts to a cluster on which Proactive HA is already configured, and then add the host or attach the host to a Cisco ACI Virtual Edge VMM domain, those hosts may not work properly in some circumstances. The hosts may not work properly in Proactive HA or when Cisco ACI Virtual Edge or OpFlex goes down. The hosts also may not go into quarantine mode although the health status of the host is correctly set to yellow in Cisco Cisco Application Policy Infrastructure Controller (APIC).

To fix the problem, disable Proactive HA on the cluster and then re-enable it.


Before you begin

Procedure


Step 1

Log in to the VMware vSphere Client.

Step 2

From the Home page, go to Networking and then navigate to the Cisco ACI Virtual Edge DVS to which you want to add the hosts and PNICs.

Step 3

In the left navigation pane, right-click the host and choose Add and Manage Hosts from the drop-down list.

Alternatively, you can choose Add and Manage Hosts from the ACTIONS drop-down list at the top of the work pane.

Step 4

In the DVS Add and Manage Hosts dialog box, complete the following steps:

  1. In the Select Task pane, click the Add Hosts radio button, and then click NEXT.

  2. In the Select Hosts pane, click the green plus sign (+) next to Add hosts.

  3. In the Select New Hosts dialog box, check the check box next to the host that you want to add and click OK.

    You can choose multiple hosts.

  4. In the Manage Physical Adapters pane, choose a PNIC for the host that you want to add and then click Assign uplink.

  5. In the Select an uplink dialog box, choose an uplink for the adapter, and then click OK.

    Repeat step 4d and 4e for each additional PNIC that you want to assign to an uplink from that host.

    The Manage Physical Adapters pane shows that the PNIC has been assigned to the host.
  6. Click NEXT.

  7. In the Manage VMkernel adapter pane, view the configuration and then click NEXT.

  8. In the Migrate VM networking pane, click NEXT.

  9. In the Ready to complete pane, click FINISH.


Add ESXi Hosts and PNICs Using the Flash Version of the Cisco ACI vCenter Plug-in

Before you can install Cisco Application Centric Infrastructure (ACI) Virtual Edge, you must add one or more ESXi hosts and their respective PNICs to the new Cisco ACI Virtual Edge DVS.


Note

When you add hosts to a cluster on which Proactive HA is already configured, and then add the host or attach the host to a Cisco ACI Virtual Edge VMM domain, those hosts may not work properly in some circumstances. The hosts may not work properly in Proactive HA or when Cisco ACI Virtual Edge or OpFlex goes down. The hosts also may not go into quarantine mode although the health status of the host is correctly set to yellow in Cisco Cisco Application Policy Infrastructure Controller (APIC).

To fix the problem, disable Proactive HA on the cluster and then re-enable it.


Before you begin

Procedure


Step 1

Log in to the VMware vCenter Web Client.

Step 2

Go to Networking.

Step 3

In the left navigation pane, expand the Cisco ACI Virtual Edge folder and the folder for the newly created Cisco ACI Virtual Edge VMM domain.

Step 4

Right click the Cisco ACI Virtual Edge domain and choose Add and Manage Hosts.

Step 5

In the Add and Manage Hosts dialog box, in the Select task pane, click the Add hosts radio button and then click Next.

Step 6

In the Select hosts pane, click New hosts.

Step 7

In the Select new hosts dialog box, choose all the hosts that you want to add to the Cisco ACI Virtual Edge DVS, and then click OK.

Step 8

In the Add and Manage Hosts dialog box, click Next.

Step 9

Check the Manage physical adapters check box and then click Next.

Step 10

In the Manage physical network adapters pane, choose a PNIC, and click Assign uplink.

Step 11

In the Select an Uplink dialog box, choose an uplink for the adapter, and then click OK.

Step 12

Repeat Step 10 and Step 11 for each additional PNIC you want to add.

Step 13

Click Next, click Next again, and then click Finish.

Each host that you chose in Step 6 appears in the Cisco ACI Virtual Edge domain work pane.

What to do next

Upload the OVF file of the Cisco ACI Virtual Edge VM to the vCenter.

Cisco ACI Virtual Edge Installation Using the vCenter

After you fulfill the installation prerequisites, you can use the VMware vCenter to install Cisco Application Centric Infrastructure (ACI) Virtual Edge. You use the Cisco ACI plug-in for VMware vCenter, which automates the process.

There are two versions of the Cisco ACI plug-in for VMware vCenter. The original version—the Cisco ACI vCenter plug-in—is designed to work with Flash. However, Flash has been deprecated in the 6.7 version of VMware vSphere. Starting in version 6.7, Cisco ACI HTML5 vCenter plug-in, designed to work with HTML5, became available.

Procedures in this section note whether they can be performed with the Flash or HTML5 versions of the Cisco ACI plug-in for VMware vCenter.

You first upload the Cisco ACI Virtual Edge VM Open Virtualization Format (OVF) file to the vCenter content library. You can then deploy Cisco ACI Virtual Edge on the ESXi hosts.


Note

  • If you use a local data store for content library storage, re-create the content library after you remove a host and then reattach it to vCenter. That is because the data store ID changes after the host is reattached, breaking the association between the content library and the data store.

  • After you deploy Cisco ACI Virtual Edge, do not remove it from the vCenter inventory and add it back. Doing so removes all the configurations you made during deployment. Deploy a new Cisco ACI Virtual Edge instead of adding an existing one back to the inventory.


Uploading the Cisco ACI Virtual Edge VM OVF File to the VMware vCenter Content Library

Before you deploy the Cisco Application Centric Infrastructure (ACI) Virtual Edge on the ESXi hosts, you upload the Cisco ACI Virtual Edge virtual machine (VM) OVF file to the VMware vCenter. You can use one of two methods:

Upload the OVF File Using the HTML5 Version of the Cisco ACI vCenter Plug-in

You upload the Cisco Application Centric Infrastructure (ACI) Virtual Edge VM OVF file to the VMware vCenter using the HTML5 version of the Cisco ACI vCenter plug-in.

Before you begin

You must have done the following:

  • Created a VMM domain for the Cisco ACI Virtual Edge on Cisco Application Policy Infrastructure Controller (APIC).

  • Downloaded the folder with the OVF file to your computer.

  • Made sure that the OVF file is compatible with the version of Cisco APIC.

  • If you plan to use the Cisco ACI vCenter plug-in, ensure that the fabric has been successfully registered with the plug-in.

    See the chapter "Cisco ACI vCenter Plug-in" in the Cisco ACI Virtualization Guide for instructions for installing and using the plug-in.

Procedure

Step 1

Log in to the VMware vSphere Client.

Step 2

From the Menu drop-down list, choose Content Libraries.

You can use an existing content library or create one to receive the upload of the Cisco ACI Virtual Edge VM OVF. See VMware documentation for instructions for creating a content library.

Step 3

In the left navigation pane, right-click the library and choose Import Item from the drop-down list.

Step 4

In the Import Library Item dialog box, in the Source file area, complete one of the following steps:

  • To upload the OVF file using a URL, click the URL radio button and enter the file URL.
  • To upload the OVF file from a local file, click the Local file radio button, click UPLOAD FILE, and in the pop-up window, choose the file, and then click Open.
Step 5

In the Destination area, enter a name for the file in the Item name field.

Step 6

Click IMPORT.

Once the OVF file is uploaded to the content library, it appears in the content library work pane under the Templates tab.


Upload the OVF File Using the Flash Version of the Cisco ACI vCenter Plug-in

You upload the Cisco Application Centric Infrastructure (ACI) Virtual Edge VM OVF file to the vCenter using the Flash version of the Cisco ACI vCenter plug-in.

Before you begin

You must have done the following:

  • Created a VMM domain for the Cisco ACI Virtual Edge on Cisco Application Policy Infrastructure Controller (APIC).

  • Downloaded the folder with the OVF file to your computer.

  • Made sure that the OVF file is compatible with the version of Cisco APIC.

  • If you plan to use the Cisco ACI vCenter plug-in, ensure that the fabric has been successfully registered with the plug-in.

    See the chapter "Cisco ACI vCenter Plug-in" in the Cisco ACI Virtualization Guide for instructions for installing and using the plug-in.

Procedure

Step 1

Log in to the vSphere Web Client.

Step 2

Choose Content Libraries.

You can use an existing content library or create one to receive the upload of the Cisco ACI Virtual Edge VM OVF. See VMware documentation for instructions for creating a content library.

Step 3

Choose the library and then click Import item.

Step 4

In the Import library item dialog box, click the Browse button.

Step 5

In the pop-up dialog box, choose the OVF file and click Open.

Another pop-up dialog box appears, which prompts you to choose the virtual machine disk (VMDK) file and XML file in the OVF folder.

Step 6

Choose the VMDK file and XML files and then click OK.

Once the OVF file is uploaded to the content library, it appears in the work pane under the Templates tab.


What to do next

Deploy Cisco ACI Virtual Edge on the ESXi hosts.

Deploy Cisco ACI Virtual Edge Using the HTML5 Version of the Cisco ACI vCenter Plug-in

After you upload the Cisco Application Centric Infrastructure (ACI) Virtual Edge VM OVF file to VMware vCenter, you deploy Cisco ACI Virtual Edge on the ESXi hosts.

Before you begin

You must have done the following:

  • Created a VMM domain for the Cisco ACI Virtual Edge on Cisco APIC.

  • Added one or more ESXi hosts and PNICs to the new Cisco ACI Virtual Edge DVS in VMware vCenter.

  • Uploaded the Cisco ACI Virtual Edge VM OVF file to VMware vCenter.

Procedure


Step 1

Log in to the VMware vSphere Client.

Step 2

On the Home page, in the left navigation pane, click Cisco ACI Fabric.

Step 3

In the Cisco ACI Fabric navigation pane, choose the fabric from the Fabric drop-down list.

Step 4

In the left navigation pane, click AVE to display the list of domains associated with the fabric, and then double-click the domain where you want to deploy the Cisco ACI Virtual Edge VM.

Step 5

In the AVE work pane, click the AVE tab.

Step 6

(Optional) In the upper right of the work pane, click Max concurrent tasks pencil icon to choose the number of deployments to run at the same time.

If you want to deploy the Cisco ACI Virtual Edge VM on multiple hosts, if you specify the number of concurrent tasks, the VM will be deployed on the number of hosts that you specify. For example, if you choose to deploy the VM on five hosts and choose three tasks to run concurrently, deployment will proceed on three of the hosts at the same time while deployment for the other two hosts is queued.

Step 7

In the Datacenter table, check the check box for each host on which you want to deploy the Cisco ACI Virtual Edge VM.

Step 8

Click DEPLOY AVE.

Step 9

In the pop-up window, click CONTINUE.

The New AVE Wizard appears.
Step 10

In the Version pane, click the radio button for the Cisco ACI Virtual Edge version that you want to use and then click NEXT.

Step 11

In the Networking pane, click the radio button for the management port group that you want to use with the Cisco ACI Virtual Edge VM and then click NEXT.

Step 12

In the Storage pane, complete one of the following actions:

  • Leave the check box checked for Let vCenter select the Datastore Automatically and then click NEXT.

    Uncheck the check box for Let vCenter select the Datastore Automatically, from the host drop-down list, choose a datastore, and then click NEXT.

Step 13

In the Settings pane, in the Admin Password and the Confirm Admin Password fields, enter your password for the VMware vCenter and then click NEXT.

Step 14

In the Summary pane, view information about the newly deployed VM and then click FINISH.


What to do next

Verify that the deployment is underway. In the AVE work pane, a New AVE pop-up window appears on the host where you deployed the VM. The host displays the percentage of how much of the deployment has completed. You also can click the clipboard icon at the upper right of the work pane and in the ACI Tasks & Settings, view information about the New AVE task.

Deploy Cisco ACI Virtual Edge Using the Flash Version of the Cisco ACI vCenter Plug-In

After you upload the Cisco ACI Virtual Edge VM OVF file to VMware vCenter, you deploy Cisco ACI Virtual Edge on the ESXi hosts. You can deploy Cisco ACI Virtual Edge as a component of a Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) in a remote site. Or you can deploy Cisco ACI Virtual Edge without making it part of a Cisco ACI vPod. See Cisco ACI vPod product documentation for more information.

Before you begin

You must have done the following:

  • Created a VMM domain for the Cisco ACI Virtual Edge on Cisco APIC.

  • Added one or more ESXi hosts and PNICs to the new Cisco ACI Virtual Edge DVS in VMware vCenter.

  • Uploaded the Cisco ACI Virtual Edge VM OVF file to VMware vCenter.


    Note

    If you use VMware vCenter 6.0 Web Client, the pop-up window for browsing to the OVF file may not appear. In that case, upload the OVF file, Virtual Machine Disk (VMDK) file, and XML file to the HTTP server. Then use the OVF file URL from the server to download the OVF file to the content library.

Procedure


Step 1

Log in to the vSphere Web Client.

Step 2

In the Home work pane, click the Cisco ACI Fabric icon.

Step 3

In the Cisco ACI Fabric navigation pane, click ACI Virtual Edge.

Step 4

In the ACI Virtual Edge work pane, if there are multiple virtual domains, choose the domain from the Select an ACI Virtual Edge Domain drop-down list; if there is only one virtual domain, skip to the next step.

Step 5

Choose the host or hosts on which you want to deploy Cisco ACI Virtual Edge.

Step 6

From the ACI Virtual Edge version drop-down list, choose the version to be deployed.

Step 7

From the Management PortGroup drop-down list, choose the management port group.

Step 8

From the Datastore drop-down list, choose Custom, click Edit.

Step 9

In the Custom AVE Datastore selection dialog box, choose a local or a remote data store for each Cisco ACI Virtual Edge.

Note 
To ensure a higher level of availability, we recommend that you choose a local data store if you have one.
Note 
You may not see all types of local storage in VMware vCenter. However, if you uncheck the Use local datastore only check box, VMware vCenter shows all local data stores. For details, see the document "When installing ESX/ESXi 4.x or 5.x to a physical server, the local SAS drive appears as a remote storage (1027819)" on the VMware website for details.
Step 10

In the VM Admin Password fields, enter a new password for the Cisco ACI Virtual Edge VMs.

Step 11

If you want to deploy the Cisco ACI Virtual Edge as part of a Cisco ACI vPod, complete the following steps:

  1. Check the vPod Mode check box.

  2. From the vPod drop-down list, choose the Cisco ACI vPod that you want to associate the Cisco ACI Virtual Edge with.

Step 12

Click Install/Upgrade ACI Virtual Edge.

Step 13

In the Install dialog box, click Yes.

In the work pane, the installed hosts display OpFlex status, the Cisco ACI Virtual Edge VM, and management IP. It could take a little while for OpFlex to come up.


What to do next

  • Attach the correct EPGs to the VMM domain on the Cisco APIC controller or through VMware vCenter using the Cisco ACI vCenter plug-in.

  • Put the VMs into the correct port groups in vCenter.

Cisco ACI Virtual Edge Installation Using the VMware PowerCLI

After you fulfill the preinstallation prerequisites, you can use the VMware PowerCLI to install Cisco ACI Virtual Edge.

You first set up the VMware Power CLI environment. You then download the .zip file containing the VMware PowerCLI file, import the Cisco ACI Virtual Edge module, then deploy the new Cisco ACI Virtual Edge VM from the vCenter content library.

Setting Up the PowerCLI Environment

Before you can use the PowerCLI to deploy the Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) or Cisco Application Centric Infrastructure (ACI) Virtual Edge virtual machines (VMs), you import the CiscoAVE PowerCLI module and establish a connection to the VMware vCenter.

Before you begin

Make sure that you have PowerCLI 6.0 Release 3 or later.

Procedure


Step 1

Download the CiscoAVE .zip file containing the high-level configuration files for Cisco ACI vPod or Cisco ACI Virtual Edge.

The zip file contains the following:

  • CiscoAVE.psm1: The CiscoAVE VMware Power CLI module file

  • lib/: The module library

Step 2

Import the CiscoAVE PowerCLI module using the Import-Module command.

Example:

PowerCLI C:\> Import-Module CiscoAVE.psm1
Step 3

Connect to the VMware vCenter using the standard PowerCLI commands: Connect-VIServer and Connect-CisServer.

The Connect-CisServer command is required for features such as tagging and managing the VMware vCenter content library.

Example:

PowerCLI C:\> Connect-VIServer -Server 172.23.143.235 -User admin -Password lab
Name 			Port 		User
---- 			---- 		----
172.23.143.235 	443 		admin

Example:

PowerCLI C:\> Connect-CisServer -Server 172.23.143.235 -User admin -Password lab
Name              User             Port
----              ----             ----
172.23.143.235    admin@localos    443

Managing the VMware vCenter Content Library Using the VMware PowerCLI

Upload the Open Virtualization Format (OVF) file to the VMware vCenter content library so the scripts in the file to deploy the virtual machines (VMs).

You can use an existing content library or create one. You create a new content library in the VMware vSphere Web Client UI or with the PowerCLI commands in this section.

Procedure


Step 1

Create a new VMware vCenter content library using the New-LocalContentLibrary command.

The following text shows the command syntax:
New-LocalContentLibrary [-Name] Object [-Datastore] Object [-Datacenter] Object [CommonParameters]

Example:

PowerCLI C:\> New-LocalContentLibrary -Name ave-lib -Datastore 129-local -Datacenter mininet
Connecting to vCenter.................................................[ok]
Creating content library 'ave-lib'....................................[ok]
Step 2

Upload an OVF file to the VMware vCenter content library using the New-ContentLibraryItem command.

The OVF (or .ova) file must be available on the local machine where you run the command.

The following text shows the command syntax:
New-ContentLibraryItem [-Name] Object [-ContentLibrary] Object [-Ovf] Object [CommonParameters]

Example:

PowerCLI C:\> New-ContentLibraryItem -Name vpod-ova -ContentLibrary ave-lib -Ovf L:\ova\aci-vpod.14.0.0.84.ova
Connecting to vCenter.................................................[ok]
Extracting OVA........................................................[ok]
Validating............................................................[ok]
Uploading aci-vpod.14.0.0.84-disk1.vmdk...............................[ok]
Uploading aci-vpod.14.0.0.84.ovf......................................[ok]
Finishing up..........................................................[ok]
Step 3

Remove an item from the VMware vCenter content library using the Remove-LocalContentLibraryItem command:

The following text shows the command syntax:
Remove-LocalContentLibraryItem [-Name] Object [-ContentLibrary] Object [CommonParameters]

Example:

PowerCLI C:\> Remove-LocalContentLibraryItem -Name vpod-14.0.0.84 -ContentLibrary vpod-ova
Connecting to vCenter.................................................[ok]
Deleting content library item 'vpod-14.0.0.84'........................[ok]

Deploying Cisco ACI Virtual Edge Using the VMware PowerCLI

If you have a Windows platform, you can use the VMware PowerCLI to install Cisco Application Centric Infrastructure (ACI) Virtual Edge. You can deploy Cisco ACI Virtual Edge as a component of a Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) in a remote site. Or you can deploy it without making it part of a Cisco ACI vPod. See Cisco ACI vPod documentation for more information.


Note

You can use 'Get-Help' on any command to get help for any of the parameters. For example: Get-Help New-LocalContentLibrary

Procedure


Step 1

Take one of the following actions, depending on how you want to use Cisco ACI Virtual Edge:

Option Description
If you want to deploy Cisco ACI Virtual Edge... Then...
As part of a Cisco ACI vPod Go to Step 2.
Not as part of a Cisco ACI vPod Go to Step 3.
Step 2

Deploy Cisco ACI Virtual Edge as part of a Cisco ACI vPod using the New-VPodAveVM command.

The following text shows the command syntax:
New-VPodAveVM [-HostName] Object [-DomainName] Object [-MgmtPortgroupName] Object [-AdminPassword] SecureString [-InfraVlan]
Object [-OvfItem] Object [-ApicVersion] Object [-VpodId] Object [[-Vtor1Ip] String] [[-Vtor2Ip] String] [[-VtepIp]
String] [[-VtepNetmask] String] [[-VtepGateway] String] [[-Library] String] [[-DatastoreName] String] [[-Ip] String]
[[-Netmask] String] [[-Gateway] String] [[-Nameserver] String] [[-VmHostname] String] [CommonParameters]

Example:

PowerCLI C:\> $pass = Read-Host -AsSecureString
********
PowerCLI C:\> New-VPodAveVM -HostName 198.51.100.15 -DomainName mininet -MgmtPortgroupName "VM Network" 
-AdminPassword $pass -InfraVlan 4 -OvfItem cisco-ave-build312 -ApicVersion "4.0(1.0)" -VpodId 2

Connecting to vCenter.................................................[ok]
Validating configuration..............................................[ok]
Deploying OVF (this might take several minutes).......................[ok]
Applying Cluster configuration........................................[ok]
Applying Cluster configuration........................................[ok]
Applying VM configuration.............................................[ok]
Applying Host configuration...........................................[ok]
Powering On VM........................................................[ok]
Step 3

Deploy Cisco ACI Virtual Edge not as part of a Cisco ACI vPod using the New-AveVM command.

The following text shows the command syntax:

New-AveVM [-HostName] Object [-DomainName] Object [-MgmtPortgroupName] Object [-AdminPassword] SecureString [-InfraVlan]
<Object> [-OvfItem] Object [-ApicVersion] Object [[-Library] String] [[-DatastoreName] String] [[-Ip] String] [[-Netmask]String] [[-Gateway] String] [[-Nameserver] String] [[-VmHostname] String] [CommonParameter]

Example:

PowerCLI C:\> New-AveVM -HostName 198.51.100.15 -DomainName AVE-FI -MgmtPortgroupName 'VLAN418' -InfraVlan 5 -OvfItem "cisco-ave-2.0.0.466-r3" -Library 466 -Ip 10.197.143.195 
-Netmask 255.255.255.0 -Gateway 198.51.100.160 -DatastoreName datastore-248 -ApicVersion "4.0(1.0)" -Verbose

cmdlet New-AveVM at command pipeline position 1
Supply values for the following parameters:
AdminPassword: ********
Connecting to vCenter.................................................[ok]
Validating configuration..............................................[ok]
Deploying OVF (this might take several minutes).......................[ok]
Applying Cluster configuration........................................[ok]
Applying Cluster configuration........................................[ok]
Applying VM configuration.............................................[ok]
Applying Host configuration...........................................[ok]
Powering On VM........................................................[ok]
PowerCLI C:\>
Step 4

Get a list of deployed Cisco ACI Virtual Edge virtual machines (VMs) using the Get-AveVM command.

The following text shows the command syntax:
Get-AveVM [<CommonParameters>]

Example:

PowerCLI C:\> Get-AveVM | Format-Table

VirtualMachine 				HostName 		DVS		ManagementIp
-------------- 				-------- 		---		------------
cisco-ave_198.51.100.15_mininet 	198.51.100.15 		mininet 	198.51.100.41

Cisco ACI Virtual Edge Installation Using Python

After you fulfill the preinstallation prerequisites, you can use Python to install Cisco ACI Virtual Edge.

You first download the zip file containing the Python files, set up the environment to run Python, and then use Python commands to create a content library on vCenter, upload the Cisco ACI Virtual Edge VM OVF file to the vCenter content library, and then deploy the new VM from the content library.

Setting Up the Python Environment

Set up the Python environment so you can use Python to install Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) or Cisco Application Centric Infrastructure (ACI) Virtual Edge.


Note

We strongly recommend that you use a virtual environment to avoid any Python dependency problems.

Before you begin

You must have done the following:

  • Made sure that you have Python 2.7.9 or a later version.

  • Made sure that you have VMware vCenter 6.0 GA U3 or later.

  • Made sure that you have Git and PIP installed.

Procedure


Step 1

Download the .zip file containing the high-level Python configuration scripts for deploying Cisco ACI vPod and Cisco ACI Virtual Edge.

The .zip file contains the following:

  • get-avevm.py: Gets the list of Cisco ACI Virtual Edge virtual machines (VMs) currently deployed.

  • new-avevm.py: Deploy a new Cisco ACI Virtual Edge VM.

  • remove-avevm.py: Removes a Cisco ACI Virtual Edge VM.

  • content-library.py: Interact with the VMware vCenter content library.

  • get-vpodvm.py: Get a list of Cisco ACI vPod VMs currently deployed.

  • new-vpodvm.py: Deploy a new pair (one virtual spine [vSpine] and one virtual leaf [vLeaf]) of Cisco ACI vPod VMs.

  • remove-vpodvm.py: Remove all Cisco ACI vPod VMs.

  • requirements.txt: Python dependencies list used by the PIP package management system.

Step 2

(Optional but recommended) Set up a Python virtual environment.

  1. Enter the following commands:

    Example:

    $ pip install virtualenv
    $ virtualenv venv
  2. Enter one of the following commands:

    • If you have a Linux or Macintosh system, enter the following command:
      $ . venv/bin/activate
    • If you have a Windows system, enter the following command:
      > ven\Scripts\activate
Step 3

Install the VMware vSphere Automation software development kit (SDK).

  1. Download the VMware vSphere Automation SDK from GitHub; there is currently no up-to-date version in the Python Package Index (PyPi).

    Example:

    (venv) $ git clone https://github.com/vmware/vsphere-automation-sdk-python.git
    (venv) $ cd vsphere-automation-sdk-python
    
    Linux:
    
    (venv) $ pip install --upgrade -r requirements.txt --extra-index-url file://`pwd`/lib
    
    Windows:
    
    > pip install --upgrade --force-reinstall -r requirements.txt --extra-index-url file:///absolute_dir_to_sdk/lib
Step 4

Install all other dependencies.

Example:

(venv) $ cd ../
(venv) $ pip install -r requirements.txt

The requirements.txt file contains all the dependencies that the script relies on. Installing the dependencies in this file is a one-time task.


Managing the VMware vCenter Content Library Using Python

You upload the Open Virtualization Format (OVF) file to the VMware vCenter content library so the scripts in the file can deploy the virtual machines (VMs).

You can use an existing library or create a new one. You create a new content library in the VMware vSphere Web Client UI or with the Python commands in this section.

Procedure


Step 1

Create a new content library using the subcommand Create.

The following text shows the command usage:
usage: content-library.py [-h] --vcenter VCENTER --vc-username VC_USERNAME
[--vc-password VC_PASSWORD] [--silent] Create --name NAME --datacenter DATACENTER
--datastore DATASTORE

Example:

(venv) $ python content-library.py --vcenter 172.23.143.235 --vc-username admin --vcpassword
lab Create --name ave_repo --datacenter mininet --datastore 129-local
Connecting to vCenter.................................................[ok]
Creating content library 'ave_repo'...................................[ok]
Step 2

Copy the ave vmdk file to the datastore of any of the host in the VMware vCenter.

Example:

scp cisco-ave-2.1.1.321-disk1.vmdk root@10.23.238.203:/vmfs/volumes/datastore2/
Step 3

Upload the OVF file to the VMware vCenter content library using the subcommand Upload.

The OVF file must be available on the local machine where you run the Python script. Provide the full datastore path of the copied vmdk file in —vmdk-ds-path.

The following text shows the command usage:
usage: content-library.py [-h] --vcenter VCENTER --vc-username VC_USERNAME
[--vc-password VC_PASSWORD] [--silent] Upload --library LIBRARY --item ITEM --path PATH
[--vmdk-ds-path VMDK_DS_PATH]

Example:

(venv) $ python content-library.py --vcHost 10.23.219.150 --vcUser 'administrator' --vcPwd ‘lab' Upload --library repo --item cisco-ave-2.1.1.321.ovf --path /Users/User/dev/ovf/cisco-ave-2.1.1.321.ovf --vmdk-ds-path ds:///vmfs/volumes/59348426-b1a50255-8787-cc167ee18b76/cisco-ave-2.1.1.321-disk1.vmdk
Connecting to vCenter.................................................[ok]
Extracting OVA........................................................[ok]
Validating............................................................[ok]
Uploading aci-vpod.14.0.0.84-disk1.vmdk...............................[ok]
Uploading aci-vpod.14.0.0.84.ovf......................................[ok]
Finishing up..........................................................[ok]
Step 4

Remove an item from the content library using the subcommand Remove.

The following text shows the command usage:
usage: content-library.py [-h] --vcenter VCENTER --vc-username VC_USERNAME
[--vc-password VC_PASSWORD] [--silent] Remove --library LIBRARY --item ITEM

Example:

(venv) $ python content-library.py --vcenter 172.23.143.235 --vc-username admin --vcpassword
lab Remove --library repo --item vpod-14.0.0.84
Connecting to vCenter.................................................[ok]
Deleting content library item 'vpod-14.0.0.84'........................[ok]

Deploying Cisco ACI Virtual Edge Using Python

You can use a Python script to deploy Cisco Application Centric Infrastructure (ACI) Virtual Edge. You can deploy Cisco ACI Virtual Edge as a component of a Cisco Application Centric Infrastructure (ACI) Virtual Pod (vPod) in a remote site. Or you can deploy it without making it part of a Cisco ACI vPod. See Cisco ACI vPod documentation for more information.


Note

You can enter -h on any script to get help for any of the parameters. Example:

# python new-avevm.py -h

Before you begin

  • Make sure that you have set up the Python environment. See the procedure Setting Up the Python Environment in this guide.

  • If you used a proxy to access the Internet when setting up the Python environment, unset it before running Python scripts:
    unset http_proxy
    unset https_proxy

Procedure


Step 1

Take one of the following actions, depending on how you want to use Cisco ACI Virtual Edge:

Option Description
If you want to deploy Cisco ACI Virtual Edge... Then...
as part of a Cisco ACI vPod Go to Step 2.
not as part of a Cisco ACI vPod Go to Step 3.
Step 2

Deploy Cisco ACI Virtual Edge as part of a Cisco ACI vPod using the vPod subcommand.

The following text shows the command usage:
usage: new-avevm.py [-h] [--silent] --vcenter VCENTER --vc-username
VC_USERNAME [--vc-password VC_PASSWORD] --host-name
HOST_NAME --domain-name DOMAIN_NAME --mgmt-pg MGMT_PG
[--admin-password ADMIN_PASSWORD] --infra-vlan INFRA_VLAN
--ovf-item OVF_ITEM [--library LIBRARY]
[--datastore DATASTORE] [--ip IP] [--netmask NETMASK]
[--gateway GATEWAY] [--nameserver NAMESERVER]
[--vm-hostname VM_HOSTNAME] --apic-version APIC_VERSION
vPod --vpod-id VPOD_ID [--vtor1-ip VTOR1_IP]
[--vtor2-ip VTOR2_IP] [--vtep-ip VTEP_IP]
[--vtep-netmask VTEP_NETMASK]
[--vtep-gateway VTEP_GATEWAY]

Example:

python new-avevm.py --vcenter 172.23.143.235 --vc-username 'administrator@vsphere.local' --vc-password 'vcpassword' --host-name 172.23.143.129 --domain-name 'ave-dom1' --mgmt-pg 'VM Network' --infra-vlan 10 --ovf-item cisco-ave-2.0.0.476 --admin-password 'adminpassword' --apic-version '4.0(0.0)' vPod --vpod-id 2

Connecting to vCenter.................................................[ok]
Validating configuration..............................................[ok]
Deploying OVF (this might take several minutes).......................[ok]
Applying Cluster configuration........................................[ok]
Applying Cluster configuration........................................[ok]
Applying VM configuration.............................................[ok]
Applying Host configuration...........................................[ok]
Powering On VM........................................................[ok]
Note 
If the management port group is on a VMware VDS, you must specify the VDS name in the following format: --mgmt-pg 'vds-name/portgroup-name'
Note 
To use a static management IP address, use the --ip parameter, placed before the vPod subcommand:
[...] --ip 172.31.100.11 --netmask 255.255.255.0 --gateway 172.31.100.1 --nameserver 172.23.140.25 vPod [...]
Step 3

Deploy Cisco ACI Virtual Edge not as part of a Cisco ACI vPod using the Enterprise subcommand.

The following text shows the command usage:
usage: new-avevm.py [-h] [--silent] --vcenter VCENTER --vc-username
VC_USERNAME [--vc-password VC_PASSWORD] --host-name
HOST_NAME --domain-name DOMAIN_NAME --mgmt-pg MGMT_PG
[--admin-password ADMIN_PASSWORD] --infra-vlan INFRA_VLAN
--ovf-item OVF_ITEM [--library LIBRARY]
[--datastore DATASTORE] [--ip IP] [--netmask NETMASK]
[--gateway GATEWAY] [--nameserver NAMESERVER]
[--vm-hostname VM_HOSTNAME] --apic-version APIC_VERSION
{vPod,Enterprise} ...

Example:

(venv) $ python new-avevm.py --vcenter 172.23.143.235 --vc-username admin --vc-password 
lab --host-name 172.23.143.129 --domain-name mininet --mgmt-pg 'VM Network' --infra-vlan 4 
--ovf-item cisco-ave-build312 --apic-version '4.0(0.0)' --admin-password password Enterprise

Connecting to vCenter.................................................[ok]
Validating configuration..............................................[ok]
Deploying OVF (this might take several minutes).......................[ok]
Applying Cluster configuration........................................[ok]
Applying Cluster configuration........................................[ok]
Applying VM configuration.............................................[ok]
Applying Host configuration...........................................[ok]
Powering On VM........................................................[ok]
Step 4

Get a list of deployed Cisco ACI Virtual Edge virtual machines (VMs) using the get-avevm.py script.

The following text shows the script usage:
usage: get-avevm.py [-h] [--silent] --vcenter VCENTER --vc-username
VC_USERNAME [--vc-password VC_PASSWORD]

Example:

(venv) $ python get-avevm.py --vcenter 172.23.143.235 --vc-username admin --vc-password lab

+----------------------------------+----------------+---------+----------------+
| Virtual Machine | Host | Domain | Management IP |
+----------------------------------+----------------+---------+----------------+
| cisco-ave_172.23.143.129_mininet | 172.23.143.129 | mininet | 172.31.143.146 |
| cisco-ave_172.23.143.228_mininet | 172.23.143.228 | mininet | None |
+----------------------------------+----------------+---------+----------------+

Verify the Cisco ACI Virtual Edge Deployment

After you deploy Cisco Application Centric Infrastructure (ACI) Virtual Edge, verify the deployment by ensuring that the interface that is used to communicate with Cisco ACI Virtual Edge (kni0) has a virtual tunnel endpoint (VTEP) IP address. Also verify that OpFlex is up.

Before you begin

You must have deployed Cisco ACI Virtual Edge in VMware vCenter.

Procedure


Step 1

Enter the ipconfig command and examine the output.

Example:

kni0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.9.3  netmask 255.255.252.0  broadcast 192.168.11.255
        inet6 fe80::250:56ff:fea7:fac  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:a7:0f:ac  txqueuelen 1000  (Ethernet)
        RX packets 374443  bytes 52541802 (50.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 161054  bytes 20000611 (19.0 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Step 2

Check if OpFlex is up by entering the appropriate vemcmd command:

  • If you are using Cisco ACI Virtual Edge as part of Cisco ACI Virtual Pod (vPod), enter the command vemcmd show opflex cloud, as shown in the following example:
    AVE-36:~$ vemcmd show opflex cloud
    Status: READY
    Peer 1, host: 192.168.8.16, port: 8009, status: READY
    Peer 2, host: 192.168.8.17, port: 8009, status: READY
    Dvs name: comp/prov-VMware/ctrlr-[vpod]-vc/sw-dvs-1983
    
  • If you are using Cisco ACI Virtual Edge and it is not part of Cisco ACI vPod, enter the command vemcmd show opflex, as shown in the following example:
    cisco-ave:~$ vemcmd show opflex
    Status: 12 (Active)
    Channel0: 12 (Active), Channel1: 12 (Active)
    Dvs name: comp/prov-VMware/ctrlr-[vpod]-vc/sw-dvs-1983
    Remote IP: 10.0.0.30 Port: 8000
    Infra vlan: 4093
    FTEP IP: 10.0.0.32
    Switching Mode: LS
    Encap Type: VXLAN
    NS GIPO: 228.1.1.1
    

What to do next

Read the sections View Cisco ACI Virtual Edge Licenses Using the GUI and Post-Installation Configuration in this guide.

View Cisco ACI Virtual Edge Licenses Using the GUI

Beginning with Cisco APIC Release 3.2(1), you can view Cisco ACI Virtual Edge licenses in the Cisco ACI Fabric as part of the Smart Licensing feature.

You also can use NX-OS style CLI commands to view licensing information. For detailed information, see the knowledgebase article Smart Licensing on Cisco.com.

Before you begin

You must register for Smart Licensing. See the knowledgebase article Smart Licensing on Cisco.com.

Procedure


Step 1

Log in to Cisco APIC.

Step 2

Go to System > Smart Licensing.

The central pane, in the Smart License Usage area, displays a list of licenses, their number, and status. For the Cisco ACI Virtual Edge license, the Count column displays the number of Cisco ACI Virtual Edge instances in the Cisco ACI Fabric. Only Cisco ACI Virtual Edge instances that are turned on and connected through OpFlex are counted.

The Count column displays the number of Cisco ACI Virtual Edge instances present in the VMware vCenter DVS that is managed by Cisco APIC. Even Cisco ACI Virtual Edge instances that are not powered on are counted for licensing.

Note 
Cisco ACI Virtual Edge license count may be incorrect while upgrade or downgrade is being performed.

Configuring a Static IP Address in VMware vCenter

After you deploy Cisco Application Centric Infrastructure (ACI) Virtual Edge, you can configure a static IP address for it in VMware vCenter. You perform the procedure after accessing VMware vCenter using either the Cisco ACI HTML5 vCenter plug-in or the Flash version of the Cisco ACI vCenter plug-in:

Configure a Static IP Address Using the HTML5 Version of the VMware vSphere Client

You can configure a static IP address for the Cisco Application Centric Infrastructure (ACI) Virtual Edge. If you do not use Python or the VMware PowerCLI, you can configure the static IP address in the VMware vCenter. Complete this procedure if you use the HTML5 version of the Cisco ACI HTML5 vCenter plug-in.

Before you begin

You must have installed Cisco ACI Virtual Edge in the VMware vCenter.

Procedure


Step 1

Log in to the VMware vSphere Client.

Step 2

Go to Hosts and Clusters > Datacenter > host and select the Cisco ACI Virtual Edge virtual machine (VM) on the host.

Step 3

In the menu bar, click the square red icon to shut down the the Cisco ACI Virtual Edge VM, and then in the Confirm Power Off dialog box, click YES.

Step 4

With the Cisco ACI Virtual Edge VM chosen in the left navigation pane, click Configure and then click vApp Options.

Step 5

Click EDIT in the upper right of the work pane.

Step 6

In the Edit vApp Options dialog box, complete the following steps:

  1. Ensure that the IP Allocation tab is chosen.

  2. In the Authoring area, leave the IP allocation check boxes checked for DHCP and OVF environment.

  3. In the Deployment area, choose Static - Manual from the IP allocation drop-down list.

  4. Click OK.

Step 7

Enter the IP address, mask, and subnet information for unrecognized OVF by completing the following steps:

  1. In the vApp Options work pane, click the Configure tab.

  2. In the Properties area at the bottom of the work pane, click the radio button for the Management Address, and then click Set Value.

  3. In the Set Value dialog box, in the IP value field, enter the IP management address, and then click OK.

  4. Repeat step 7b and 7c for the Management Netmask and Management Gateway.

Step 8

In the left navigation pane, right-click the Cisco ACI Virtual Edge VM, and choose Power from the drop-down list. then choose Power On.


Configuring a Static IP Address in VMware vCenter

You can configure a static IP address for the Cisco Application Centric Infrastructure (ACI) Virtual Edge. If you do not use Python or the VMware PowerCLI, you can configure the static IP address in the VMware vCenter.

Procedure


Step 1

Log in to the VMware vCenter Web Client.

Step 2

Power off the Cisco ACI Virtual Edge.

Step 3

Navigate to the host and virtual machine (VM) and then choose the Configure tab.

Step 4

In the VM pane, choose Edit and then in the Edit Settings dialog box, choose vApp Options.

Step 5

In the Deployment area, from the IP allocation drop-down list, choose Static - Manual.

Step 6

In the Unrecognized OVF sections area, enter the IP address, mask and gateway information.

Step 7

Click OK.


Post-Installation Configuration

After you install the Cisco ACI Virtual Edge, perform key configuration tasks:

  • Deploy an application profile, which includes creating a tenant, application profile, EPGs, filters, and contracts, and assigning port groups to VMs. Then verify the application profile.

    See the Cisco APIC Basic Configuration Guide for instructions.

  • If you want to use Distributed Firewall, Enable it after installation. See the chapter "Distributed Firewall" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

  • In order for Cisco ACI Virtual Edge to forward multi-destination traffic—especially when traffic goes through a blade switch—configure an IGMP querier under the infra BD subnet. This enables devices to build their Layer 2 multicast tree.

    See the section "Configuring IGMP Querier and Snooping" in the Cisco ACI Virtual Edge Configuration Guide for instructions.

You can find instructions for other configuration tasks—including microsegmentation, SPAN, intra-EPG isolation enforcement, mixed-mode encapsulation, and BPDU features—in the Cisco ACI Virtual Edge Configuration Guide.