Endpoint Security Groups (ESGs) are the new network security component in Cisco Application Centric Infrastructure (Cisco ACI). Although the endpoint groups (EPGs) have been providing the network security in Cisco ACI, EPGs have to be associated to a single bridge domain (BD) and used to define security zones within a BD. This is because the EPGs define both forwarding and security segmentation at the same time. The direct relationship between the BD and an EPG limits the possibility of an EPG to spanning more than one BD. This limitation of EPGs is resolved by using the new ESG constructs.

The Application endpoint group (fvAEPg) object that represents an EPG has a direct relationship with the bridge domain object (fvBD) that represents the Layer 2 broadcast domain. This is illustrated in the above figure, in the first three columns.

An ESG is a logical entity that contains a collection of physical or virtual network endpoints. In addition, an ESG is associated to a single VRF (Virtual Routing and Forwarding) instead of BD. This allows the definition of a security zone that is independent of the BDs (the fourth column of Figure 1, illustrates this point). Just as the EPGs divide a BD into security zones, the ESGs divide the VRF into security zones.

The EPG policy embeds both forwarding and security logic. For example, an EPG provides not only a security zone based on VLAN, but also a VLAN binding on leaf node interfaces. Also, a contract on the EPG is used to enforce the security and determine which leaf nodes the BD subnet should be deployed on, and which subnets to be leaked to which VRF in the case of VRF route leaking (i.e. shared service). On the contrary, an ESG is used only to enforce security using the contracts while the forwarding logics are handled by other components. With an ESG, the routing logic such as BD subnets deployment and VRF route leaking are moved to VRF level. The VLAN binding on leaf node interfaces are still handled at EPG level.

An ESG is a security construct that has certain match criteria to define which endpoint belongs to the ESG, and uses contracts or policies to define the security stance. The match criteria are called the ESG selectors that are based on attributes, such as an IPv4 or IPv6 address spanning across BDs in the associated VRF. For Cisco APIC, release 5.0(1), the available matching criteria is the IP address or the subnet of an endpoint.

The contract usage in the ESGs is the same as the EPGs. Endpoints that belong to the same ESG can communicate without the need for a contract. To enable communication between endpoints that belong to different ESGs, you need to configure contracts between the ESGs. For the communication with devices outside of the Cisco ACI fabric, you need to configure a contract between the L3Out external EPG (l3extInstP) and the ESG. You can also use a Layer 4 to Layer 7 service graph in conjunction with a contract between the ESGs. However, contracts between an EPG and an ESG are not supported.

Contracts are the Cisco ACI equivalent of access control lists (ACLs). ESGs can only communicate with other ESGs according to the contract rules. The administrator uses a contract to select the types of traffic that can pass between ESGs, including the protocols and ports allowed. An ESG can be a provider, consumer, or both provider and consumer of a contract, and can consume multiple contracts simultaneously. ESGs can also be part of a preferred group so that multiple ESGs can talk freely with other ESGs that are part of the preferred group.

Supported Contracts relationship:

1. ESG ⇔ ESG

2. ESG ⇔ L3Out EPG

3. ESG ⇔ inband-EPG

4. ESG ⇔ vzAny

Contracts between the ESGs and the EPGs (or uSeg EPGs) are not supported. When an endpoint in an ESG needs to communicate with other endpoints in the EPG, the other endpoints need to be migrated to the ESGs first. vzAny or preferred group can be used as an alternative during the migration. Other contract-related features that are supported in a uSeg EPG, such as contract inheritance, an intra ESG contract, or intra ESG isolation, are also supported in an ESG. The exception is the Taboo Contract, which is not supported in an ESG.

When an endpoint needs a service that is shared by another VRF, there are two things required for the communication to happen. The first thing is the routing reachability. The second thing is security permission. In an EPG, these two are coupled closely in one set of configurations, such as the EPG subnet and contracts. In ESG, these two are decoupled in two different configurations:

1. The configuration of route leaking at the VRF level, which is independent of the ESG contract configuration.

2. The configuration of contracts between the ESGs.

With these two configurations completely decoupled, you do not need to configure a subnet or a subset of the subnet under the ESG as you must do for an EPG.

The following sections explain how to configure route leaking for the bridge domain subnets and external prefixes learned from external routers. After you finish configuring route leaking, you can configure a contract between two ESGs, or an ESG and L3Out EPG, to allow the communication. You must use a contract with a scope larger than VRF, such as global.

Note	The route leaking configuration at the VRF level is supported only for ESGs.

All the Layer 4 to Layer 7 service graph features that are available for the EPGs are supported for the ESGs. The use of service graph between ESGs of different VRFs is not supported in Cisco APIC 5.0.

Note

This note is an implementation detail for advanced user information. If a service graph is attached to a contract between ESGs, the Cisco APIC automatically creates hidden service EPGs where the Layer 4 to Layer 7 service device attaches, just as Cisco APIC does for a service graph between EPGs. Unlike a service graph between EPGs, in the case of ESGs, the hidden service EPGs get a global pcTag. Beginning with Cisco APIC release 5.0(1), all new service EPGs that are created for Layer 4 to Layer 7 Service deployments with vzAny-to-vzAny contracts will get global PcTag.

For more information on Layer 4 to Layer 7 services deployment, refer to Cisco APIC Layer 4 to Layer 7 Services Deployment Guide.

The Capacity Dashboard tab can be used to get a summary of critical fabric resource thresholds. This allows you to see quickly how close you are to reaching the approved scalability limits. Per leaf node usage is also shown, allowing you to see quickly which leaf node may be hitting resource constraints.

1. In the menu bar, choose Operations > Capacity Dashboard to launch the Capacity Dashboard troubleshooting tool.

2. In the Capacity Dashboard page, choose Fabric Capacity for the fabric resources. Scroll down for the Endpoint Security Groups tile and the Global pcTag tile to determine the available resources.

3. In the Capacity Dashboard page, choose Leaf Capacity for the leaf usage. Check the ESG tab for details on the resource usage for Endpoint Security Groups.

The Endpoint Tracker tab allows you to enter a fabric-attached endpoint IP or MAC address and quickly see the location of this endpoint, the endpoint group to which the endpoint belongs, the VLAN encapsulation used, and if any transitions (flaps) have occurred for this endpoint.

1. In the menu bar, click Operations > EP Tracker to launch the Endpoint Tracker troubleshooting tool.

2. In the End Point Search field, enter the IP address or MAC address of the endpoint and click Search.

3. Click on the endpoint after it is displayed.

The Endpoint Tracker tool displays the date and time of each state transition along with the IP address, MAC address, owning endpoint group, action (attached or detached), physical node, interface, and VLAN encapsulation during the event.

The Endpoint Tracker tool uses an object called the fvCEp to find the endpoints that are learned in the fabric, for an ESG and as well as an EPG. An endpoint that belongs to an ESG is represented by two fvCEp objects, one for the EPG that provides VLAN binding, another for the ESG that provides security.Therefore, the Endpoint Tracker tool shows two entries (one for an EPG, another for an ESG) when used for the ESG endpoints.

As of the Cisco APIC release 5.0(1), the following limitations apply:

• Contracts between ESGs and EPGs are not supported.

• The ESG feature is not integrated with Cisco ACI Multi-Site. Other topologies such as Multi-Pod, Multi-Tier, and Remote Leaf are supported.

• The supported ESG selector is the IP address. MAC addresses, VM tags, or other criteria are not yet supported.

• An ESG contract can be applied only for routed traffic with IP as the selector.

• Taboo contracts are not supported with ESGs.

• Inter-VRF service graphs between ESGs are not supported.

• ESGs are not supported as a source or destination of the following features:

  • On Demand Atomic Counter

  • On Demand Latency Measurement

  • SPAN

• The following features configured at the BD or EPG level are not supported when endpoints in the BD/EPG are classified to an ESG:

  • Endpoint Reachability (Static routes on BD/EPG)

  • Anycast Service

  • Microsoft NLB

  • First Hop Security (FHS)

  • Host Based Routing / Host Route Advertisement

• Only the EX and newer generation of leaf nodes are supported for ESG deployment.

• To prevent Layer 2 traffic (that is, non-routed traffic) from bypassing ESG security when IP is used as the selector, enable an intra EPG contract with a permit-all rule, such as the common default contract, on all of the EPGs that provide VLAN-to-interface binding for the ESG endpoints. If all the endpoints in the EPGs are classified to ESGs, you can alternatively enable intra EPG isolation with proxy ARP on the EPGs instead of the intra EPG contract. If the EPG is used only for VMM DVS integration, you can alternately enable the Allow Micro-Segmentation option instead of the other two options mentioned above. Either feature forces all communication between ESG endpoints to go through Layer 3 routing.

  Note

  This note explains the differences between an intra EPG contract with a permit-all rule and intra EPG isolation with proxy-ARP.The main purpose of both features is the same, which is to enforce all traffic to be routed on ACI leaf switches by using proxy-ARP. Note that proxy-ARP is enabled implicitly for the EPG when an intra EPG contract is used.The difference is when there are two or more endpoints that don’t belong to ESGs but learned in an EPG. With an intra EPG contract with a permit-all rule, such endpoints can still communicate freely within the same EPG due to the permit-all rule. However, with intra EPG isolation with proxy-ARP, such endpoints can no longer communicate even though they are in the same EPG.

• Label configurations are not supported when you add contracts to an ESG.

The contracts between an EPG to an ESG are not supported in the Cisco APIC, release 5.0(1). Therefore, to migrate the existing EPG deployments to an ESG with minimal impact, vzAny can be used to allow communication between an EPG and an ESG endpoints during the migration.

For migration to ESG within VRF

The following procedure explains how to migrate EPGs to ESGs within the same VRF, starting with the provider EPG, EPG4.

1. As shown in the example, firstly create an ESG Service.

2. Configure a new contract, with ESG Service as provider and vzAny as consumer. This will allow the communication between the existing consumer EPG endpoints in EPG1, EPG2 and EPG3 with the newly created ESG service.

3. Now migrate the provider EPG EPG4’s endpoints into ESG Service by configuring the appropriate endpoint selectors.

4. This completes the migration of the provider EPG EPG4 to ESG.

5. Prepare the migration of individual consumer EPGs, namely EPG1, EPG2 and EPG3, by creating ESG IT and Sales that consume the same contract as the VRF vzAny.

   Note: You can alternatively configure fine grain contracts for ESG IT and Sales respectively instead. In such a case, you must provide the new contract from ESG Service as well.

6. Now migrate the consumer endpoints to ESGs by configuring the selectors on ESG IT and Sales.

7. Once the migration of all the consumer EPGs are completed, the administrator can remove the VRF vzAny consumer configuration.

For migration to ESG across VRF

For shared services with EPGs, the administrator would have created a subnet under the provider EPG for route leaking across VRFs via contracts. As shown in the above example figure,

• provider EPG EPG4, part of provBD, has subnet 192.168.1.1/24 configured with Shared between VRFs scope enabled.

• Consumer EPGs EPG1, EPG2 and EPG3 are part of ConsBD that has subnet 172.16.1.1/24 with Shared between VRFs scope enabled.

The following procedure explains how to migrate EPGs to ESGs in the case of shared service, starting with the provider EPG, EPG4.

1. Create the subnet 192.168.1.1/24 on the ProvBD (provider EPG’s BD) if not existing already.

   Note: Shared between VRFs scope is not required.

2. Create a provider ESG Service .

3. Configure a new contract, with ESG Service as the provider and consumer VRF as vzAny consumer. This will allow the communication between the existing consumer EPG endpoints in EPG1, EPG2 and EPG3 with the newly created ESG Service.

4. Remove the EPG4’s subnet 192.168.1.1/24 .

5. Once this subnet is removed, the traffic flow across VRF will stop.

6. Proceed with the creation of ESG route-leak policy in the VRF level between provider and consumer VRF. On the consumer VRF leak subnet 172.16.1.0/24 to the provider VRF. On the provider VRF leak subnet 192.168.1.0/24 to the consumer VRF.

7. Now migrate the provider EPG EPG4’s endpoints into ESG Service by configuring the appropriate endpoint selectors.

8. Traffic flow across the VRFs will restore. This completes the migration of the provider EPG EPG4 to ESG.

9. Prepare the migration of individual consumer EPGs, namely EPG1, EPG2 and EPG3, by creating ESG IT and Sales that consume the same contract as the VRF vzAny.

   Note: You can alternatively configure fine grain contracts for ESG IT and Sales respectively instead. In such a case, you must provide the new contract from ESG Service as well. These contracts (the fine grain or the one consumed by vzAny) need to have a scope larger than VRF such as global.

10. Now migrate the consumer endpoints to ESGs by configuring the selectors on ESG IT and Sales.

11. Optionally, the administration can remove the Shared between VRFs scope on the consumer BD subnet 172.16.1.1/24 if this BD is not participating in any other shared service. Also cleanup the EPG contracts association used for shared services.

12. Once the migration of all the consumer EPGs are completed, the administrator can remove the VRF vzAny consumer configuration.

Guidelines for EPG shared service and ESG shared service

• If the subnet prefix under the provider EPG is exactly the same as the route leaked via ESG route-leak policy in VRF level, the administrator needs to make sure that the EPG subnet (or shared-service contracts) is deleted prior to configuring the ESG route-leak policy in VRF level. This will prevent overlapping configuration for the same prefix.

• If the subnet prefix under the provider EPG is a subset of the route leaked via ESG route-leak policy in VRF level, the administrator can create the ESG route-leak policy prior to deleting the provider EPG subnet. However, the traffic within the provider EPG subnet will not follow ESG security until the provider EPG subnet is deleted.

  Note	This configuration is valid only when the provider BD has the same subnet as the route leaked via ESG route-leak policy or a subnet larger than that.

• If the subnet prefix under the provider EPG is a superset of the route leaked via ESG route-leak policy in VRF level, the administrator can create the ESG route-leak policy prior to deleting the provider EPG subnet. The traffic within the route leaked via ESG route-leak policy follows the ESG security without deleting the provider EPG subnet.

Procedure:

<polUni> 
  <fvTenant name="t0"> 
    <fvAp name="ap0"> 
      <!-- ESG with the name ESG1 and Preferred Group as Exclude --> 
      <fvESg name="ESG1" prefGrMemb="exclude"> 
          <!-- The ESG is associated to VRFA --> 
          <fvRsScope tnFvCtxName="VRFA" /> 

          <!-- provided and consumed contracts --> 
          <fvRsProv tnVzBrCPName="provided_contract1" /> 
          <fvRsCons tnVzBrCPName="consumed_contract2" /> 

          <!-- Endpoint Selectors for the ESG --> 
          <fvEPSelector matchExpression="ip=='192.168.0.1/32'" /> 
          <fvEPSelector matchExpression="ip=='192.168.1.0/28'" /> 
          <fvEPSelector matchExpression="ip=='2001:23:45::0:0/64'" /> 
      </fvESg> 
    </fvAp> 
  </fvTenant> 
</polUni>

 

Before you begin:

You must have configured the BD subnet to be leaked or the BD subnet that includes the leaked subnet.

Procedure:

<polUni>
  <fvTenant name="t0">
    <fvCtx name="VRFA">
      <leakRoutes>
        <!--
            leak the BD subnet 192.168.1.0/24 with the Allow L3Out Advertisement
            False (i.e. scope private)
        -->
        <leakInternalSubnet ip="192.168.1.0/24" scope="private">
          <!--
              leak the BD subnet to Tenant t1 VRF VRFB with the
              Allow L3Out Advertisement configured in the parent
              scope (i.e. scope inherit)
          -->
          <leakTo ctxName="VRFB" tenantName="t1" scope="inherit" />
        </leakInternalSubnet>
      </leakRoutes>
    </fvCtx>
  </fvTenant>
</polUni>

 

Before you begin:

You must have configured an L3Out in the source VRF “VRFA” and external prefixes are learned.

Procedure:

<polUni>
  <fvTenant name="t0">
    <fvCtx name="VRFA">
      <leakRoutes>
        <!--
            leak the external prefixes in the range of
            10.20.0.0/17 and 10.20.0.0/30
        -->
        <leakExternalPrefix ip="10.20.0.0/16" ge="17" le="30">
          <!-- leak the external prefixes to Tenant t1 VRF VRFB -->
          <leakTo ctxName="VRFB" tenantName="t1" />
        </leakExternalPrefix>
      </leakRoutes>
    </fvCtx>
  </fvTenant>
</polUni>