Schemas and Application Templates

Shadow Objects

When a contract exists between site-local EPGs in stretched VRF or in Shared Services use-cases where provider and consumer are in different VRFs and communicate through Tenant contracts, the EPGs and bridge domains (BDs) are mirrored on the remote sites. The mirrored objects appear as if they are deployed in each of these sites' controllers, while only actually being deployed in one of the sites. These mirrored objects are called "shadow" objects.


Note

Shadow objects should not be removed using the APIC GUI.

For example, if a tenant and VRF are stretched between Site1 and Site2, provider EPG and its bridge domain are deployed in Site2 only, and consumer EPG and its domain are deployed in Site1 only, then corresponding shadow bridge domains and EPGs will be deployed as shown in the figure below. They appear with the same names as the ones that were deployed directly to each site.

Figure 1. Basic Shadow EPG

The following objects can be shadowed:

  • VRFs

  • Bridge Domains (BDs)

  • L3Outs

  • External EPGs

  • Application Profiles

  • Application EPGs

  • Contracts (Hybrid Cloud deployments)

If your fabrics are running APIC Release 5.0(2) or later, when you select a shadow object in the APIC GUI, you will see a This is a shadow object pushed by MSC to support intersite policies. Do not make any changes or delete this object. warning at the top of main GUI pane. In addition, shadow EPGs that are not part of a VMM domain will not have static ports, while shadow BDs will have No Default SVI Gateway option enabled in the APIC GUI.

Other Use Cases with Shadow Objects

Shadow objects are also created in a number of other use cases, such as Preferred Group, vzAny, and Layer 3 Multicast, and hybrid cloud, as shown in the figures below.

Figure 2. Preferred Group

In case of multicast, the shadow objects are created only for EPGs/BDs that have multicast sources connected and the option explicitly configured at the EPG level.

Figure 3. L3 Multicast

In case of hybrid cloud deployments, even stretched objects will create shadow objects where implicit contracts exist. For example, in the following case where an EPG is stretched between an on-premises and cloud sites, shadow external EPGs are created in each site with implicit shadow contracts between the stretched EPG and the shadow external EPGs.

Figure 4. Hybrid Cloud

Starting with Cisco APIC, Release 5.2(3), shadow objects are indicated by a unique icon in the Cisco APIC GUI. Regular Orchestrator-created objects are shown with a green cloud symbol, whereas the shadow objects will have a gray cloud icon.

Hiding Shadow Objects in APIC GUI

Starting with APIC Release 5.0(2), you can choose to show or hide the shadow objects created by the Nexus Dashboard Orchestrator in the on-premises site's APIC GUI. Shadow objects in Cloud Network Controller are always hidden.

If you want to hide shadow objects from the GUI, keep the following in mind:

  • This option cannot be set globally from the Orchestrator and must be set directly in each site's APIC as described in this section.

  • The option to show shadow objects is turned off by default for all new APIC Release 5.0(2) installations and upgrades, so previously visible objects may become hidden.

  • Hiding shadow objects relies on a flag set by the Nexus Dashboard Orchestrator specifically for this feature, which is enabled from Orchestrator Release 3.0(2) and later:

    • If shadow objects are deployed by an earlier Orchestrator version, they will not have the required tag and will always be visible in the APIC GUI.

    • If shadow objects are deployed by Orchestrator version 3.0(2) or later, they will have the tag and can be hidden or shown using the APIC GUI setting.

    • We recommend upgrading each fabric to APIC Release 5.0(2) before upgrading the Nexus Dashboard Orchestrator.

      When the Nexus Dashboard Orchestrator is upgraded to Release 3.0(2), any objects deployed to sites running APIC Release 5.0(2) or later will be tagged with appropriate tags and can be shown or hidden using the APIC GUI without having to re-deploy them.

      If you upgrade the Orchestrator before the fabric's APIC, the site's objects will not be tagged and you will need to manually re-deploy the configuration after the fabric is upgraded for the flag to be set.

  • If you ever downgrade your fabric to a release prior to Release 5.0(2), the shadow objects will no longer be hidden and you may see a different icon for them in the APIC GUI.

Procedure

Step 1

Log in to the site's APIC.

Step 2

In the top right corner, click the Manage my profile icon and choose Settings.

Step 3

In the Application Settings window, enable or disable the Show Hidden Policies checkbox.

The setting is stored in the user profile and is enable or disabled separately for each user.

Step 4

Repeat the process for any additional APIC sites.

Creating Schemas and Templates

Before you begin

Procedure

Step 1

Log in to your Nexus Dashboard and open the Nexus Dashboard Orchestrator service.

Step 2

Create a new schema.

  1. From the left navigation pane, choose Application Management > Schemas.

  2. On the Schemas page, click Add Schema.

  3. In the schema creation dialog, provide the Name and optional description for the schema and click Add.

    By default, the new schema is empty, so you need to add one or more templates.

Step 3

Create a template.

  1. In the schema page, click Add New Template.

  2. In the Select a Template type window, choose ACI Multi-Cloud and click Add.

    • ACI Multi-Cloud—Templates used for Cisco ACI on-premises and cloud sites, which allow template and object stretching between multiple sites. This template supports two deployment types:

      • Multi-Site - The template can be associated to a single site (site-local policies) or to multiple sites (stretched policies) and the option should be selected for Multi-Site Network (ISN) or VXLAN intersite communication to allow template and object stretching between multiple sites.

      • Autonomous - The template can be associated to one or more sites that are operated independently and are not connected through an Inter-Site Network (no intersite VXLAN communication).

        Because autonomous sites are by definition isolated and do not have any intersite connectivity, there is no shadow object configuration across sites and no cross-programming of pctags or VNIDs in the spine switches for intersite traffic flow.

        The autonomous templates also allow for significantly higher deployment scale.

      The following sections focus primarily on this type of templates.

    • NDFC—Templates designed for Cisco Nexus Dashboard Fabric Controller (formerly Data Center Network Manager) sites.

      This guide described Nexus Dashboard Orchestrator configurations for on-premises Cisco ACI fabrics. For information on working with Cisco NDFC sites, see the Cisco Nexus Dashboard Orchestrator Configuration Guide for NDFC Fabrics instead.

    • Cloud Local—Templates designed for specific Cloud Network Controller use cases, such as Google Cloud site connectivity, and cannot be stretched between multiple sites.

      This guide describes Nexus Dashboard Orchestrator configurations for on-premises Cisco ACI fabrics. For information on working with Cloud Network Controller fabrics, see the Nexus Dashboard Orchestrator use case library instead.

  3. In the right sidebar, provide the Display Name for the template.

  4. (Optional) Provide a Description.

  5. From the Select a Tenant dropdown, select the Tenant for this template.

    Keep in mind, the user account you're using to create a new schema must be associated with the tenant you are trying to add to it, otherwise the tenant will not be available in the dropdown menu. Associating a user account with a tenant is described in Tenants and Tenant Policies Templates.

  6. In the template view page, click Save.

    You must save the template after this initial configuration for additional options (such as site association) to become available.

  7. Repeat this step to create any additional templates.

    For more information on schema and template design, see Schema and Template Design Considerations.

Step 4

Assign the templates to sites.

You deploy fabric configuration by deploying one template at a time to one or more sites. So you need to associate the template with at least one site where you want to deploy the configuration.

  1. In the template view page, click Actions and choose Sites Association.

  2. In the Add Sites to <template> dialog, select one or more sites where you want to deploy the template and click Ok.

What to do next

After you have created a schema and one or more templates, you can proceed with editing the templates as described in the following sections of this document based on your specific use cases. After you finish defining configurations, you can deploy the templates as described in Deploying Templates.

Importing Schema Elements From APIC Sites

You can create new objects and push them out to one or more sites or you can import existing site-local objects and manage them using the Multi-Site Orchestrator. This section describes how to import one or more existing objects, while creating new objects is described later on in this document.

When importing policies from APIC into NDO, the common practice is to import some objects, such as VRFs or contracts, into a stretched template and other objects, such as non-stretched EPGs or BDs, into site-local templates.

Prior to Release 3.1(1), importing an object into a site-local template that referenced another object that is part of a stretched template presented certain challenges, for example:

  • If a referenced object already exists in NDO and a new object is imported with the Include Relations option enabled, NDO would throw an error when trying to deploy the site-local template because of object duplication since the referenced object already existed.

  • However, not importing the referenced object (Include Relations option disabled) would require an administrator to perform manual mapping with the referenced object after the import.

When importing an object into a site-local template that has references with another object that is part of a different template (in the same or a different schema), the references are automatically resolved by NDO. In such cases, the Import Relations option will be grayed-out in the UI for the object that is being imported and a warning tooltip will provide additional info, such as: [Referenced Object] already exists in [Template]. Existing relations are imported by default. While such objects are imported with their relations by default, you can change the references once the import operation is completed, for example by re-mapping a BD to a different VRF. The new behavior applies to all configuration objects that can be imported.

To import one or more objects from sites:

Procedure

Step 1

Open the Schema where you want to import objects.

Step 2

In the left sidebar, select the Template where you want to import objects.

Step 3

In the main pane click the Import button and select the Site from which you want to import.

Step 4

In the Import from <site-name> window that opens, select one or more objects.

Note

 

The names of the objects imported into NDO must be unique across all sites. Importing different objects with duplicate names will cause a schema validation error and the import to fail. If you want to import objects that have the same name, you must first rename them.

Step 5

(Optional) Enable the Import Relations knob to import all related objects.

For example, when importing a BD, enabling the Import Relations knob will import the associated VRF as well.

Note

 

As described previously, the Import Relations knob will be enabled by default and cannot be disabled for objects whose related objects already exist in NDO.

Step 6

Click Import.

Configuring VRFs

This section describes how to configure a VRF.

Before you begin

You must have the schema and template created and a tenant assigned to the template, as described in Creating Schemas and Templates.

Procedure

Step 1

Select the schema and template where you want to create the VRF.

  1. In the main pane, select +Create Object > VRF.

    Alternatively, you can scroll down to the VRFs area, mouse over the tile, and click Add VRF.

  2. In the right pane, provide the Display Name for the VRF.

  3. (Optional) Provide a Description.

Step 2

Configure the On-Premises Properties for the VRF.

  1. Specify Policy Control Enforcement Preference.

    Note that you cannot change the Policy Control Enforcement for newly created VRFs and the setting is locked to the enforced mode.

    However, you can use this to transition any VRF that you import from an APIC site that is configured as unenforced to the enforced mode after importing it. A typical use case is for brown field deployments where existing VRFs must be converted to enforced mode to support stretching them between sites. Once you have transitioned an imported VRF from unenforced to enforced in NDO, you will not be able to make further changes to this field.

    • Enforced—Security rules (contracts) will be enforced.

    • Unenforced—Security rules (contracts) will not be enforced.

  2. (Optional) Enable IP Data-Plane Learning.

    Defines if IP addresses are learned through data-plane packets for the VRF.

    When disabled, IP addresses are not learned from the data-plane packets. Local and remote MAC addresses are still learned, but local IP addresses are not learned from data packets.

    Regardless of whether this parameter is enabled or disabled, local IP addresses can still be learned from ARP, GARP, and ND.

  3. (Optional) Enable L3 Multicast for the VRF.

    For additional information, see Layer 3 Multicast.

  4. (Optional) Enable vzAny for the VRF.

    For additional information, see vzAny Contracts.

  5. (Optional) Enable Preferred Group for the VRF.

    For additional information, see EPG Preferred Groups Overview and Limitations

Configuring Bridge Domains

This section describes how to configure a Bridge Domain (BD).

Before you begin

Procedure

Step 1

Select the schema and template where you want to create the bridge domain.

Step 2

Create a bridge domain.

  1. In the main pane, select +Create Object > Bridge Domain.

    Alternatively, you can scroll down to the Bridge Domains area, mouse over the tile, and click Add Bridge Domain.

  2. In the right pane, provide the Display Name for the bridge domain.

  3. (Optional) Provide a Description.

Step 3

Configure On-Premises Properties.

  1. From the Virtual Routing & Forwarding dropdown, select the VRF for this BD.

  2. (Optional) Enable L2 Stretch.

  3. (Optional) Enable Intersite BUM Traffic Allow.

    This option becomes available if you enabled L2 Stretch.

  4. (Optional) Enable Optimized WAN Bandwidth.

    This option becomes available if you enabled L2 Stretch.

  5. (Optional) Enable Unicast Routing.

    If this setting is enabled and a subnet address is configured, the fabric provides the default gateway function and routes the traffic. Enabling unicast routing also instructs the mapping database to learn the endpoint IP-to-VTEP mapping for this bridge domain. The IP learning is not dependent upon having a subnet configured under the bridge domain.

  6. (Optional) Enable L3 Multicast for the BD.

    For additional information about Layer 3 multicast, see Layer 3 Multicast.

  7. (Optional) Choose L2 Unknown Unicast mode.

    By default, unicast traffic is flooded to all Layer 2 ports. If enabled, unicast traffic flooding is blocked at a specific port, only permitting egress traffic with MAC addresses that are known to exist on the port. The method can be Flood or Hardware Proxy.

    When the BD has L2 Unknown Unicast set to Flood, if an endpoint is deleted the system deletes it from both the local leaf switches as well as the remote leaf switches where the BD is deployed, by selecting Clear Remote MAC Entries. Without this feature, the remote leaf continues to have this endpoint learned until the timer expires.

    Note

     

    Modifying the L2 Unknown Unicast setting causes traffic to bounce (go down and up) on interfaces to devices attached to EPGs associated with this bridge domain.

  8. (Optional) Choose Unknown Multicast Flooding mode.

    This is applicable for IPv4 unknown multicast traffic and is the node forwarding parameter for Layer 3 unknown multicast destinations.

    • Flood (default)—Unknown IPv4 multicast traffic is flooded on all front panel ports attached with the EPGs associated with this bridge domain. Flooding is not restricted to only M-Router ports of the bridge domain.

    • Optimized Flood—Send the data only to M-router ports in the bridge domain.

  9. (Optional) Choose IPv6 Unknown Multicast Flooding mode.

    This is applicable for IPv6 unknown multicast traffic and is the node forwarding parameter for Layer 3 unknown multicast destinations.

    • Flood (default)—Unknown IPv6 multicast traffic is flooded on all front panel ports attached with the EPGs associated with this bridge domain. Flooding is not restricted to only M-Router ports of the bridge domain.

    • Optimized Flood—Send the data only to M-router ports in the bridge domain.

  10. (Optional) Choose Multi-Destination Flooding mode.

    The multiple destination forwarding method for Layer 2 multicast and broadcast traffic.

    • Flood in BD—Sends the data to all ports on the same bridge domain.

    • Drop—Drops Packet. Never sends the data to any other ports.

    • Flood in Encapsulation—Send the data to all the EPG ports with the same VLAN within the bridge domain, except for the protocol packets which are flooded to the entire bridge domain.

      Note

       

      This mode is supported only when the L2 Stretch option is disabled and is not supported for BDs that are stretched across sites.

  11. (Optional) Enable ARP Flooding.

    Enables ARP flooding, so that the Layer 2 broadcast domain maps IP addresses to the MAC addresses. If flooding is disabled, unicast routing will be performed on the target IP address.

    Enables ARP flooding, so that ARP request will be flooded inside the Layer 2 broadcast domain. If the BD is stretched across sites, enabling ARP flooding is only possible in conjunction with enabling Intersite BUM Traffic Allow. When ARP flooding is disabled, the leaf receiving the ARP request from a locally connected endpoint will forward it directly to the remote leaf where the target endpoint of the ARP request is connected (if the IP for the remote endpoint is known in the endpoint table) or to the spines (if the IP for the remote endpoint is not known in the endpoint table).

    If you set the L2 Unknown Unicast mode to Flood, the ARP Flooding cannot be disabled. If the L2 Unknown Unicast mode is set to Hardware Proxy, ARP flooding can be enabled or disabled.

  12. (Optional) Provide Virtual MAC Address.

    The BD virtual MAC address and the subnet virtual IP address must be the same for all ACI fabrics for that bridge domain. Multiple bridge domains can be configured to communicate across connected ACI fabrics. The virtual MAC address and the virtual IP address can be shared across bridge domains.

    Note

     

    Virtual MAC along with virtual IP subnet should used only for migration of individual sites to NDO-orchestrated multi-site fabric. Once the migration is completed, these flags can be disabled.

Step 4

Add one or more Subnets for the BD.

  1. Click +Add Subnet.

    An Add New Subnet window opens.

  2. Enter the subnet's Gateway IP address and a Description for the subnet you want to add.

  3. If necessary, enable Treat as virtual IP address option.

    This option along with the Virtual MAC Address on the BD can be used for migration scenarios from individual Common Pervasive Gateway configuration to NDO-managed Multi-Site deployments.

  4. Select the Scope for the subnet.

    The network visibility of the subnet.

    • Private to VRF—Prevents the subnet from being announced via L3Out toward an external network domain.

    • Advertised Externally—The subnet can be announced via L3Out toward an external network domain.

  5. (Optional) Enable Shared Between VRFs.

    Shared between VRFs—The subnet can be shared with and exported to multiple contexts (VRFs) in the same tenant or across tenants as part of a shared service. An example of a shared service is a routed connection to an EPG present in another context (VRF) in a different tenant. This enables traffic to pass in both directions across contexts (VRFs). An EPG that provides a shared service must have its subnet configured under that EPG (not under a bridge domain), and its scope must be set to advertised externally, and shared between VRFs.

    Shared subnets must be unique across the contexts (VRF) involved in the communication. When a subnet under an EPG provides a Layer 3 external network shared service, such a subnet must be globally unique within the entire ACI fabric.

  6. Leave the No Default SVI Gateway option unchecked.

    Enabling this options means that only the proxy route (subnet route to spine proxy) is programmed on the leaf switches and no SVI is created, which means SVI cannot be used as the gateway.

    We recommend that SVI is created by the BD subnet as the gateway and the No Default SVI Gateway option is enabled on the EPG instead because EPG subnets should only be used for route leaking.

  7. (Optional) Enable Querier option.

    Enables IGMP Snooping on the subnet

  8. (Optional) Enable Primary option to designate the subnet as primary.

    There can be one primary IPv4 subnet and one primary IPv6 subnet.

  9. Click Save.

Step 5

(Optional) Add an IGMP Interface Policy.

Beginning with Release 4.0(1), you can configure a number of Tenant Policy templates and associate them with policy objects. For more information, see Creating Tenant Policy Templates.

Step 6

(Optional) Add an IGMP Snoop Policy.

Beginning with Release 4.0(1), you can configure a number of Tenant Policy templates and associate them with policy objects. For more information, see Creating Tenant Policy Templates.

Step 7

(Optional) Add an MLD Snoop Policy.

Beginning with Release 4.0(1), you can configure a number of Tenant Policy templates and associate them with policy objects. For more information, see Creating Tenant Policy Templates.

Step 8

(Optional) Add a DHCP Policy.

For additional information, see DHCP Relay.

Step 9

Configure the bridge domain's site-local properties as necessary.

In addition to the template-level configurations, you can also define one or more site-local properties for the bridge domain, as described in Configuring Bridge Domain's Site-Local Properties

Configuring Bridge Domain's Site-Local Properties

In addition to the template-level properties you typically configure for the object when you create it in a template, you can also define one or more properties that are specific to each site to which you assign the template.

When you deploy the object to more than 1 site, the same template-level configurations are deployed to all sites, while the site-local configurations are deployed to those specific sites only.

Before you begin

You must have:

  • Created the bridge domain and configured its template-level properties, as described in Configuring Bridge Domains.

  • Assigned the template that contains the bridge domain to one or more sites.

Procedure

Step 1

Open the schema that contains the template with the bridge domain.

Step 2

In the left sidebar, select the template that contains the bridge domain under the specific site that you want to configure.

Step 3

In the main pane, select the bridge domain.

For most fields, you will see the values you have configured at the template level, which you cannot edit here.

Step 4

Click +L3Out to add an L3Out.

This is required to advertise the BD subnet out of the remote L3Out and ensure that inbound traffic to the BD can be maintained even if the local L3Out failed. In this case, you would also need to configure the subnet with the Advertised Externally flag. For more information, see the Intersite L3Out use case.

Step 5

Enable Host Route.

This enables Host Based Routing on the bridge domain. When this knob is enabled, the border leaf switches will also advertise individual end-point (EP) host-routes (/32 or /128 prefixes) along with the subnet. The host-route information is advertised only if the host is connected to the local Pod. If the EP is moved away from the local Pod or once the EP is removed from EP database, the route advertisement is then withdrawn.

Step 6

If necessary, change the SVI MAC Address.

The SVI MAC addresses must be unique per site, when virtual MAC and virtual IP are enabled for Common Pervasive Gateway (CPG) scenario. This field can also be used when CPG is not enabled, which will change the default router MAC of the BD

Step 7

Add one or more Subnets for the BD.

The concept is the same as adding subnets to the BD at the template level, except the subnets will be configured for the bridge domain on this specific site only.

  1. Click +Add Subnet.

    An Add New Subnet window opens.

  2. Enter the subnet's Gateway IP address and a Description for the subnet you want to add.

  3. Select the Scope for the subnet.

    The network visibility of the subnet.

    • Private to VRF—The subnet applies only within its tenant.

    • Advertised Externally—The subnet can be exported to a routed connection.

  4. (Optional) Enable Shared Between VRFs.

    Shared between VRFs—The subnet can be shared with and exported to multiple contexts (VRFs) in the same tenant or across tenants as part of a shared service. An example of a shared service is a routed connection to an EPG present in another context (VRF) in a different tenant. This enables traffic to pass in both directions across contexts (VRFs). An EPG that provides a shared service must have its subnet configured under that EPG (not under a bridge domain), and its scope must be set to advertised externally, and shared between VRFs.

    Shared subnets must be unique across the contexts (VRF) involved in the communication. When a subnet under an EPG provides a Layer 3 external network shared service, such a subnet must be globally unique within the entire ACI fabric.

  5. (Optional) Enable No Default SVI Gateway.

    Enabling this options means that only the proxy route (subnet route to spine proxy) is programmed on the leaf switches and no SVI is created, which means SVI cannot be used as the gateway.

    We recommend that SVI is created by the BD subnet as the gateway and the No Default SVI Gateway option is enabled on the EPG instead because EPG subnets should only be used for route leaking.

  6. (Optional) Enable Querier.

    Enables IGMP Snooping on the subnet

  7. (Optional) Enable Primary option to designate the subnet as primary.

    There can be one primary IPv4 subnet and one primary IPv6 subnet.

  8. Click Save.

Configuring Application Profiles and EPGs

This section describes how to configure an Application Profile and an EPG.

Before you begin

You must have the schema and template created and a tenant assigned to the template, as described in Creating Schemas and Templates.

This section also assume you have a Contract and a Bridge Domain created.

Procedure

Step 1

Select the schema and template where you want to create the application profile.

Step 2

Create an application profile.

  1. In the main pane, select +Create Object > Application Profile.

    Alternatively, you can scroll down to the Application Profile area, mouse over the tile, and click Add Application Profile.

  2. In the right pane, provide the Display Name for the application profile.

    You can create application profiles with the same name in different templates without any conflicts. You cannot however create other objects (such as VRFs, BDs, EPGs) with the same name in different templates if they will be deployed to the same site and tenant.

  3. (Optional) Provide a Description.

Step 3

Create an EPG.

  1. In the main pane, select +Create Object > EPG, then select the application profile where you want to create the EPG.

    Alternatively, you can scroll down to the specific Application Profile area, mouse over the EPGs tile, and click Add EPG.

  2. In the right pane, provide the Display Name for the EPG.

  3. (Optional) Provide a Description.

Step 4

Add a contract for the EPG.

Creating contracts and filters is described in detail in Configuring Contracts and Filters. If you already have a contract created:

  1. Click + Contract.

  2. On the Add Contract dialog, enter the contract name and type.

  3. Click SAVE.

Step 5

From the Bridge Domain dropdown, select the bridge domain for this EPG.

If you are configuring an on-premises EPG, you must associate it with a bridge domain.

Step 6

(Optional) Click + Subnet to add a subnet to your EPG.

You may choose to configure a subnet on the EPG level rather than the bridge domain level, for example for a VRF route-leaking use-case.

  1. On the Add Subnet dialog, enter the Gateway IP address and a description for the subnet you plan to add.

  2. In the Scope field select either Private to VRF or Advertised Externally.

  3. Click the check box for Shared Between VRFs if appropriate.

  4. Click the check box for No Default SVI Gateway if appropriate.

  5. Click OK.

Step 7

(Optional) Enable microsegmentation.

If you are configuring a microsegmentation EPG (uSeg), you must provide one or more uSeg attributes for matching endpoints to the EPG.

  1. Check the uSeg EPG checkbox.

  2. Click +uSeg Attribute.

  3. Provide the Name and Type for the uSeg attribute.

  4. Based on the attribute type you have selected, provide the attribute details.

    For example, if you have selected MAC for the attribute type, provide the MAC address to identify an endpoint in this EPG.

  5. Click SAVE.

Step 8

(Optional) Enable intra-EPG isolation.

By default, endpoints in EPG can freely communicate with each other. If you would like to isolate the endpoints from each other, set the isolation mode to Enforced.

Intra-EPG endpoint isolation policies provide full isolation for virtual or physical endpoints; no communication is allowed between endpoints in an EPG that is operating with isolation enforced. Isolation-enforced EPGs reduce the number of EPG encapsulations required when many clients access a common service but are not allowed to communicate with each other.

Step 9

(Optional) Enable Layer 3 multicast for the EPG.

For additional information about Layer 3 multicast, see Layer 3 Multicast

Step 10

(Optional) Enable preferred group membership for the EPG.

The Preferred Group feature allows you to include multiple EPGs within a single VRF to allow full communication between them with no need for contracts to be created. For additional information about EPG preferred group, see EPG Preferred Groups Overview and Limitations

Step 11

Configure the EPG's site-local properties as necessary.

In addition to the template-level configurations, you can also define one or more site-local properties for the EPG, as described in Configuring EPG's Site-Local Properties

Configuring EPG's Site-Local Properties

In addition to the template-level properties you typically configure for the object when you create it in a template, you can also define one or more properties that are specific to each site to which you assign the template.

When you deploy the object to more than 1 site, the same template-level configurations are deployed to all sites, while the site-local configurations are deployed to those specific sites only.

Before you begin

You must have:

  • Created the application profile and EPG and configured the template-level properties, as described in Configuring Application Profiles and EPGs.

  • Assigned the template that contains the bridge domain to one or more sites.

Procedure

Step 1

Open the schema that contains the template with the EPG.

Step 2

In the left sidebar, select the template that contains the EPG under the specific site that you want to configure.

Step 3

In the main pane, select the EPG.

For most fields, you will see the values you have configured at the template level, which you cannot edit here.

Step 4

Add one or more Subnets for the EPG.

  1. Click +Add Subnet.

    An Add New Subnet window opens.

  2. Enter the subnet's Gateway IP address and a description for the subnet you want to add.

  3. Select the Scope for the subnet.

    The network visibility of the subnet.

    • Private to VRF—Prevents the subnet from being announced via L3Out toward an external network domain.

    • Advertised Externally—The subnet can be announced via L3Out toward an external network domain.

  4. (Optional) Enable Shared Between VRFs.

    Shared between VRFs—The subnet can be shared with and exported to multiple contexts (VRFs) in the same tenant or across tenants as part of a shared service. An example of a shared service is a routed connection to an EPG present in another context (VRF) in a different tenant. This enables traffic to pass in both directions across contexts (VRFs). An EPG that provides a shared service must have its subnet configured under the BD (not under the EPG), and its scope must be set to advertised externally, and shared between VRFs.

    Shared subnets must be unique across the contexts (VRF) involved in the communication. When a subnet under an EPG provides a Layer 3 external network shared service, such a subnet must be globally unique within the entire ACI fabric.

  5. (Optional) Enable No Default SVI Gateway.

    Enabling this options means that only the proxy route (subnet route to spine proxy) is programmed on the leaf switches and no SVI is created, which means SVI cannot be used as the gateway.

    We recommend enabling this option on the EPG subnets, which should only be used for route leaking and leaving this option disabled on the BD subnets so that the SVI can be used as a gateway.

  6. Click Ok to save.

Step 5

Add one or more Static ports.

  1. Click +Static Port.

  2. From the Path Type dropdown, select the type of port.

  3. If configuring a physical interface, select the Pod

  4. Choose whether you want to configure a single port or a range of ports.

    For the interface configuration, you will have an option to do it either by entering a single Leaf and a Path or by entering a range of Leaf eg 120-125 and Path eg1/17-20. You will also have an option to enter a range of Leaf and associate it with one single Path, or enter a range of Path for one single Leaf.

    However, after the configuration it will still be displayed as individual ports in the UI and will require individual changes for any future updates.

  5. Select the Port Encap VLAN.

    When manually configuring the port encap on a domain for an EPG, the VLAN ID must belong to a static VLAN block within a dynamic VLAN pool.

    If EPG is enabled for microsegmentation at the template level, when a Primary MICRO-SEG VLAN is configured, the Port Encap VLAN is configured as an Isolated Secondary VLAN for the Primary VLAN. Traffic is sent from the host to the leaf using the secondary VLAN and return traffic from the leaf to the host is sent using the primary VLAN.

  6. (Optional) Select the Primary MICRO-SEG VLAN.

    The VLAN identifier for microsegmentation.

  7. (Optional) Select the Deployment Immediacy.

    Once policies are downloaded to the leaf nodes, deployment immediacy can specify when the policy is pushed into the hardware policy CAM:

    • Immediate—Specifies that the policy is programmed in the hardware policy CAM as soon as the policy is downloaded in the leaf software.

    • On Demand—Specifies that the policy is programmed in the hardware policy CAM only when the first packet is received through the data path. This process helps to optimize the hardware space.

  8. (Optional) Select the Mode.

    The mode of the static association with the path. EPG tagging refers to configuring a static path under an EPG:

    • Trunk—The default deployment mode. Select this mode if the traffic from the host is tagged with a VLAN ID.

    • Access (802.1P)—Select this mode if the traffic from the host is tagged with a 802.1P tag. When an access port is configured with a single EPG in native 802.1p mode, its packets exit that port untagged. When an access port is configured with multiple EPGs, one in native 802.1p mode, and some with VLAN tags, all packets exiting that access port are tagged VLAN 0 for EPG configured in native 802.1p mode and for all other EPGs packets exit with their respective VLAN tags. Note that only one native 802.1p EPG is allowed per access port.

    • Access (Untagged)—Select this mode if the traffic from the host is untagged (without VLAN ID). When a leaf switch is configured for an EPG to be untagged, for every port this EPG uses, the packets will exit the switch untagged. Note that when an EPG is deployed as untagged, do not deploy that EPG as tagged on other ports of the same switch.

Step 6

Add one or more Static Leaf nodes.

  1. Click +Static Leaf.

  2. From the Leaf dropdown, select the leaf node you want to add.

  3. (Optional) In the VLAN field, provide the VLAN ID for tagged traffic.

Step 7

Add one or more Domains nodes.

  1. Click +Domain.

  2. Select the Domain Association Type.

    This is the type of the domain you are adding:

    • VMM

    • Fibre Channel

    • L2 External

    • L3 External

    • Physical

  3. Select the Domain Profile name.

  4. Select the Deployment Immediacy.

    Deployment immediacy can specify when the policy is pushed:

    • Immediate—Specifies that the policy is programmed in the hardware policy CAM as soon as the policy is downloaded in the leaf software.

    • On Demand—Specifies that the policy is programmed in the hardware policy CAM only when the first packet is received through the data path. This process helps to optimize the hardware space.

  5. Select the Resolution Immediacy.

    Specifies whether policies are resolved immediately or when needed. The options are:

    • Immediate—Specifies that EPG policies are pushed to the leaf switch nodes upon hypervisor attachment to the VMware vSphere Distributed Switch (VDS). LLDP or OpFlex permissions are used to resolve the hypervisor to leaf node attachments.

    • On Demand—Specifies that EPG policies are pushed to the leaf switch nodes only when a hypervisor is attached to VDS and a VM is placed in the port group (EPG).

    • Pre-provision—Specifies that EPG policies are pushed to the leaf switch nodes even before a hypervisor is attached to the VDS. The download pre-provisions the configuration on the switch.

Configuring Contracts and Filters

This section describes how to configure a contract, a filter, and assign the filter to the contract. A filter is similar to an Access Control List (ACL), it is used to filter traffic through contracts associated to EPGs.

Procedure

Step 1

Select the schema and template where you want to create contract and filter.

You can create the contract in the same or different template as the objects (EPGs and external EPGs) to which you will apply it. If the objects that will use the contract are deployed to different sites, we recommend defining the contract in a template associated to multiple sites. However, this is not strictly required and even if the contract and filters are defined only as local objects in Site1, NDO will create those objects in a remote Site2 when a local EPG or external EPG in Site2 needs to consume or provide that contract.

Step 2

Create a filter.

  1. In the main pane, select +Create Object > Filter.

    Alternatively, you can scroll down to the Filters area, mouse over the tile, and click Add Filter.

  2. In the right pane, provide the Display Name for the filter.

  3. (Optional) Provide a Description.

Step 3

Create a filter entry.

  1. In the right pane, click + Add Entry.

    The filter entry is a combination of network traffic classification properties. You can specify one or more options as described in the following step.

  2. Provide the Name for the filter.

  3. Choose the Ether Type.

    For example, ip.

  4. Choose the IP Protocol.

    For example, icmp.

  5. Choose the Destination Port Range From and Destination Port Range To.

    The start and end of the destination ports range. You can define a single port by specifying the same value in both fields or you can define a range of ports from 0 to 65535. You can also choose to specify one of the server types instead of specific port numbers, for example http.

  6. Enable Match only fragments option.

    When enabled, the rule applies to any IP fragment with an offset that is greater than 0 (all IP fragments except the first). When disabled, the rule will not apply to IP fragments with an offset greater than 0 because TCP/UDP port information can only be checked in initial fragments.

  7. Enable Stateful option.

    When this option is enabled, any traffic coming from the provider back to the consumer will always have to have the ACK bit set in the packet or else the packets will be dropped.

  8. Specify ARP flag (Address Resolution Protocol).

    The ARP Flag is used when creating a specific filter for ARP and allows you to specify ARP request or ARP reply.

  9. Choose the Source Port Range From and Source Port Range To.

    The start and end of the source ports range. You can define a single port by specifying the same value in both fields or you can define a range of ports from 0 to 65535. You can also choose to specify one of the server types instead of specific port numbers, for example http.

  10. Specify TCP session rules.

    TCP session rules are used when creating a filter for TCP traffic and allow you to configure stateful ACL behavior.

  11. Click Ok to save the filter.

  12. Repeat this step to create any additional filter entries for this filter.

    You can create and assign multiple filter entries for each filter.

Step 4

Create a contract

  1. In the main pane, select +Create Object > Contract.

    Alternatively, you can scroll down to the Contract area, mouse over the tile, and click Add Contract.

  2. In the right pane, provide the Display Name for the contract.

  3. (Optional) Provide a Description.

  4. Select the appropriate Scope for the contract.

    Contract scope limits the contract's accessibility; the contract will not be applied to any consumer EPG outside the scope of the provider EPG:

    • Application Profile

    • VRF

    • Tenant

    • Global

  5. Toggle the Apply both directions knob if you want the same filter to apply for both consumer-to-provider and provider-to-consumer directions.

    If you enable this option, you will need to provide the filters only once and they will apply for traffic in both directions. If you leave this option disabled, you will need to provide two sets of filter chains, one for each direction.

    Note

     

    If you create and deploy a contract with Apply both directions enabled, you cannot simply disable the option and re-deploy for the change to apply. To disable this option on an already deployed contract, you must delete the contract, deploy the template, then re-create the contract with the option disabled to correctly change the setting in your fabrics.

  6. (Optional) From the Service Graph dropdown, select a service graph for this contract.

  7. (Optional) From the QoS Level dropdown, select a value for this contract.

    This value specifies the ACI QoS Level that will be assigned to the traffic using this contract. For more information, see QoS Preservation Across IPN.

    If you leave this at Unspecified, the default QoS Level 3 is applied to the traffic.

Step 5

Assign the filters to the contract.

  1. In the main pane for template, select a contract. In the right pane, scroll down to the Filter Chain area and click + Add Filter to add a filter to the contract.

  2. In the Add Filter Chain window that opens, select the filter you added in previous step from the Name dropdown menu.

  3. Select the Action for the filter.

    When adding filters, you can choose whether to permit or deny traffic that matches the filter criteria. For deny filters, you can set the priority of the filter to one of four levels: default, low, medium, or high; the permit filters always have the default priority. For more information on ACI contracts and filters, see Cisco ACI Contract Guide.

  4. Click Ok to add the filter to the contract.

  5. If you disabled the Apply both directions option on the contract, repeat this step for the other filter chain.

  6. (Optional) You can create and assign multiple Filters to each Contract.

    If you want to create additional filter for the same contract:

    • Repeat Step 2 and Step 3 to create another filter along with its filter entries.

    • Then repeat this step to assign the new filter to this Contract.

Viewing Schemas

After you have created one or more schemas, they are displayed both on the Dashboard and the Schemas page.

You can use the functionality available on these two pages to monitor the usage and the health of your schemas when they are deployed. You can also access and edit specific areas of the implemented schema policies using the Nexus Dashboard Orchestrator GUI.

Cloning Schemas

This section describes how to create a copy of an existing schema and all its templates using the "Clone Schema" feature in the Schemas screen.


Note

If you clone a template and attempt to deploy it to the same site with a different configuration, the deployment may fail due to a duplicate name error. Changing the object name in the cloned template only updates the display name. It does not alter the database record, which causes the deployment to fail in this scenario.

Procedure

Step 1

Log in to your Nexus Dashboard Orchestrator GUI.

Step 2

Choose the schema to clone.

  1. From the left navigation menu, select Application Management > Schemas.

  2. From the Actions menu next to the name of the schema you want to clone, select Clone.

Step 3

Provide the name for the new schema and click Clone.

After you click Clone, the UI will display Cloning of <schema-name> was successful. message and the new schema will be listed in the Schemas screen.

The new schema is created with the exact same templates (and their tenants’ association), object, and policy configurations as the original schema.

Note that while the templates, objects, and configurations are copied, the site association is not preserved and you will need to re-associate the template in the cloned schema with any sites where you want to deploy them. Similarly, you will need to provide any site-specific configurations for the template objects after you associate it with the sites.

Step 4

(Optional) Verify that the schema and all its templates were copied.

You can verify the operation completed successfully by comparing the two schemas.