Sites Connected via SR-MPLS

SR-MPLS and Multi-Site

Starting with Orchestrator Release 3.0(1) and APIC Release 5.0(1), the Multi-Site architecture supports APIC sites connected via MPLS networks.

In a typical Multi-Site deployment, traffic between sites is forwarded over an intersite network (ISN) via VXLAN encapsulation:

Figure 1. Multi-Site and ISN

With Release 3.0(1), MPLS network can be used in addition to or instead of the ISN allowing inter-site communication via WAN:

Figure 2. Multi-Site and MPLS

The following sections describe guidelines, limitations, and configurations specific to managing Schemas that are deployed to these sites from the Multi-Site Orchestrator. Detailed information about MPLS hand off, supported individual site topologies (such as remote leaf support), and policy model is available in the Cisco APIC Layer 3 Networking Configuration Guide.

SR-MPLS Tenant Requirements and Guidelines

While the Infra MPLS configuration and requirements are described in the Day-0 operations chapter, the following restrictions apply for any user Tenants you will deploy to sites that are connected to SR-MPLS networks.

  • You must have created and configured the SR-MPLS Infra L3Outs, including the QoS policies, as described in the Day-0 operations chapter.

  • In case when traffic between two EPGs in the fabric needs to go through the SR-MPLS network:

    • Contracts must be assigned between each EPG and the external EPG defined on the local Tenant SR-MPLS L3Out.

    • If both EPGs are part of the same ACI fabric but separated by an SR-MPLS network (for example, in multi-pod or remote leaf cases), the EPGs must belong to different VRFs and not have a contract between them nor route-leaking configured.

    • If EPGs are in different sites, they can be in the same VRF, but there must not be a contract configured directly between them.

      Keep in mind, if the EPGs are in different sites, each EPG must be deployed to a single site only. Stretching EPGs between sites is not supported when using SR-MPLS L3Outs.

  • When configuring a route map policy for the SR-MPLS L3Out:

    • Each L3Out must have a single export route map. Optionally, it can also have a single import route map.

    • Routing maps associated with any SR-MPLS L3Out must explicitly define all the routes, including bridge domain subnets, which must be advertised out of the SR-MPLS L3Out.

    • If you configure a 0.0.0.0/0 prefixe and choose to not aggregate the routes, it will allow the default route only.

      However, if you choose to aggregate routes 0 through 32 for the 0.0.0.0/0 prefix, it will allow all routes.

    • You can associate any routing policy with any tenant L3Out.

  • Transit routing is supported, but with some restrictions:

    • Transit routing between two SR-MPLS networks using the same VRF is not supported. The following figure shows an example of this unsupported configuration.

      Figure 3. Unsupported Transit Routing Configuration Using Single VRF

    • Transit routing between two SR-MPLS networks using different VRFs is supported. The following figure shows an example of this supported configuration.

      Figure 4. Supported Transit Routing Configuration Using Different VRFs

Creating SR-MPLS Route Map Policy

This section describes how to create a route map policy. Route maps are sets of if-then rules that enable you to specify which routes are advertised out of the Tenant SR-MPLS L3Out. Route maps also enable you to specify which routes received from the DC-PE routers will be injected into the BGP VPNv4 ACI control plane.

If you have no sites connected to MPLS networks, you can skip this section.

Procedure

Step 1

Log in to the Cisco ACI Multi-Site Orchestrator GUI.

Step 2

In the Main menu, select Application Management > Policies.

Step 3

In the main pane, select Add Policy > Create Route Map Policy.

Step 4

In the Add Route Map Policy screen, select a Tenant and provide the name for the policy.

Step 5

Click Add Entry under Route-Map Entry Order to add a route map entry.

  1. Provide the Context Order and Context Action.

    Each context is a rule that defines an action based on one or more matching criteria.

    Context order is used to determine the order in which contexts are evaluated. The value must be in the 0-9 range.

    Action defines the action to perform (permit or deny) if a match is found.

  2. If you want to match an action based on an IP addres or prefix, click Add IP Address.

    In the Prefix field, provide the IP address prefix. Both IPv4 and IPv6 prefixes are supported, for example 2003:1:1a5:1a5::/64 or 205.205.0.0/16.

    If you want to aggregate IPs in a specific range, check the Aggregate checkbox and provide the range. For example, you can specify 0.0.0.0/0 prefix and choose to aggregate routes 0 through 32.

  3. If you want to match an action based on community lists, click Add Community.

    In the Community field, provide the community string. For example, regular:as2-nn2:200:300.

    Then choose the Scope.

  4. Click +Add Action to specify the action that will be taken should the context match.

    You can choose one of the following actions:

    • Set Community

    • Set Route Tag

    • Set Weight

    • Set Next Hop

    • Set Preference

    • Set Metric

    • Set Metric Type

    After you have configured the action, click the checkmark icon to save the action.

  5. (Optional) You can repeat the previous substeps to specify multiple match criteria and actions within the same Context entry.

  6. Click Save to save the Context entry.

Step 6

(Optional) Repeat the previous step if you want to add multiple entries to the same route policy.

Step 7

Click Save to save the route map policy.

Enabling Template for SR-MPLS

There is a number of template configuration settings that are unique when deploying them to sites connected via MPLS. Enabling SR-MPLS for a Tenant restricts and filters certain configurations that are not available for MPLS sites while bringing in additional configurations only available for such sites.

Before you can update MPLS-specific settings, you must enable the SR-MPLS knob in the template's Tenant properties.

Procedure

Step 1

Log in to the Cisco ACI Multi-Site Orchestrator GUI.

Step 2

In the main navigation menu, select Application Management > Schemas.

Step 3

Create a new or select an existing Schema where you will configure SR-MPLS Tenant.

Step 4

Select the Tenant.

If you created a new Schema, choose a Tenant as you typically would. Otherwise click an existing Template in the left sidebar.

Step 5

In the right sidebar Template properties, enable SR-MPLS knob.

Creating VRF and SR-MPLS L3Out

This section describes how to create the VRF, tenant SR-MPLS L3Out, and External EPG you will use to configure communication between application EPGs separated by an MPLS network.

Before you begin

You must have:

Procedure

Step 1

Select the template.

Step 2

Create a VRF.

  1. In the main pane, scroll down to the VRF area and click the + sign to add a VRF.

  2. In the right properties sidebar, provide the name for the VRF.

Step 3

Create an SR-MPLS L3Out.

  1. In the main pane, scroll down to the SR-MPLS L3Out area and click the + sign to add an L3Out.

  2. In the right properties sidebar, provide the name for the L3Out.

  3. From the Virtual Routing & Forwarding dropdown, select the same VRF you selected for the external EPG in the previous step.

Step 4

Create an external EPG.

  1. In the main pane, scroll down to the External EPG area and click the + sign to add an external EPG.

  2. In the right properties sidebar, provide the name for the external EPG.

  3. From the Virtual Routing & Forwarding dropdown, select the VRF you created in the previous step.

Configuring Site-Local VRF Settings

You must provide BGP route information for the VRF used by the SR-MPLS L3Out.

Before you begin

You must have:

Procedure

Step 1

Select the schema that contains your template.

Step 2

In the left sidebar of the schema view under Sites, select the template to edit its site-local properties.

Step 3

In the main pane, scroll down to VRF area and select the VRF.

Step 4

In the right properties sidebar, click +Add BGP Route Target Address.

Step 5

Configure the BGP settings.

  1. From the Address Family dropdown, select whether it is IPv4 or IPv6 address.

  2. In the Route Target field, provide the route string.

    For example, route-target:ipv4-nn2:1.1.1.1:1901.

  3. From the Type dropdown, select whether to import or export the route.

  4. Click Save to save the route information.

Step 6

(Optional) Repeat the previous step to add any additional BGP route targets.

Configuring Site-Local SR-MPLS L3Out Settings

Similar to how you configure site-local L3Out properties for typical external EPGs, you need to provide SR-MPLS L3Out details for external EPGs deployed to sites connected via MPLS.

Before you begin

You must have:

Procedure

Step 1

Select the schema that contains your template.

Step 2

In the left sidebar of the schema view under Sites, select the template to edit its site-local properties.

Step 3

In the main pane, scroll down to SR-MPLS L3Out area and select the MPLS L3Out.

Step 4

In the right properties sidebar, click +Add SR-MPLS Location.

Step 5

Configure the SR-MPLS Location settings.

  1. From the SR-MPLS Location dropdown, select the Infra SR-MPLS L3Out you created when configuring Infra for that site.

  2. Under External EPGs section, select an external EPG from the dropdown and click the checkmark icon to add it.

    You can add multiple external EPGs.

  3. Under Route Map Policy section, select a route map policy you created in previous section from the dropdown, specify whether you want to import or export the routes, then click the checkmark icon to add it.

    You must configure a single export route map policy. Optionally, you can configure an additional import route map policy.

  4. Click Save to add the location to the MPLS L3Out.

Step 6

(Optional) Repeat the previous step to add any additional SR-MPLS Locations for your SR-MPLS L3Out.

Communication Between EPGs Separated by MPLS Network

Typically, if you wanted to establish communication between two EPGs, you would simply assign the same contract to both EPGs with one EPG being the provider and the other one a consumer.

However, if the two EPGs are separated by an MPLS network, the traffic has to go through each EPG's MPLS L3Out and you establish the contracts between each EPG and its MPLS L3Out instead. This behavior is the same whether the EPGs are deployed to different sites or within the same fabric but separated by an SR-MPLS network, such as in Multi-Pod or Remote Leaf cases.

Before you begin

You must have:

  • Added one or more sites connected to MPLS network(s) to the Orchestrator.

  • Configured Infra MPLS settings, as described in "Day-0 Operations" chapter.

  • Created a schema, added a Tenant, and enabled the Tenant for SR-MPLS, as described in Enabling Template for SR-MPLS.

Procedure

Step 1

Log in to the Cisco ACI Multi-Site Orchestrator GUI.

Step 2

Create two application EPGs as you typically would.

For example, epg1 and epg2.

Step 3

Create two separate external EPGs

These EPGs can be part of the same template or different templates depending on the specific deployment scenario.

For example, mpls-extepg-1 and mpls-extepg-2

Step 4

Configure two separate Tenant SR-MPLS L3Outs.

For example, mpls-l3out-1 and mpls-l3out-2

For each Tenant SR-MPLS, configure the VRF, route map policies, and external EPGs as described in Configuring Site-Local VRF Settings and Configuring Site-Local SR-MPLS L3Out Settings.

Step 5

Create a contract you will use to allow traffic between the two application EPGs you created in Step 2.

You will need to create and define a filter for the contract just as you typically would.

Step 6

Assign the contracts to the appropriate EPGs.

In order to allow traffic between the two application EPGs you created, you will actually need to assign the contract twice: once between epg1 and its mpls-l3out-1 and then again between epg2 and its mpls-l3out-2.

As an example, if you want epg1 to provide a service to epg2, you would:

  1. Assign the contract to epg1 with type provider.

  2. Assign the contract to mpls-l3out-1 with type consumer.

  3. Assign the contract to epg2 with type consumer.

  4. Assign the contract to mpls-l3out-2 with type provider.

Deploying Configuration

You can deploy the configuration Template to an MPLS site as you typically would, with one exception: because you cannot stretch objects and policies between MPLS site and another site, you can only select a single site when deploying the template.

Procedure

Step 1

Add the site to which you want to deploy the template.

  1. In the left sidebar of the Schema view under Sites, click the + icon.

  2. In the Add Sites window, select the site where you want to deploy the Template.

    You can only select a single site if your template is MPLS-enabled.

  3. From the Assign to Template dropdown, select one or more Template you have created in this Schema.

  4. Click Save to add the site.

Step 2

Deploy the configuration

  1. In the main pain of the Schemas view, click Deploy to Sites.

  2. In the Deploy to Sites window, verify the changes that will be pushed to the site and click Deploy.