Configuration and Verification of SDWAN Integration with ACI

Available Languages

Download Options

PDF (1.3 MB)
View with Adobe Reader on a variety of devices
ePub (1.2 MB)
View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi (Kindle) (1.0 MB)
View on Kindle device or Kindle app on multiple devices

Updated:August 6, 2021

Document ID:216942

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Acronyms

ACI - Application Centric Infrastructure

EPG - EndPoint Group

L3out - Layer 3 Out

AAR - Application Aware Routing

SLA - Service Level Agreements

DC - Data Center

WAN - Wide Area Network

SDN - Software Defined Networking

SD DC - Software Defined Data Center

SD WAN - Software Defined Wide Area Network

QOS - Quality of Service

VRF - Virtual Routing and Forwarding

Introduction

This document describes configuration steps to integrate Application Centric Infrastructure (ACI), Cisco's Software Defined - Data Center (SD-DC) solution with Software Defined - Wide Area Network (SD-WAN) and its verification.

Software Defined Networking (SDN) have been enhanced to accommodate specific network segment:

Software Defined -Data Center (SD-DC)
Software Defined - Wide Area Network (SD-WAN)

Cisco solution provides robust feature of QoS (Qualtiy of Service) in SD-DC (Application Centric Infrastructure ACI) and AAR (Application Aware Routing)/SLA (Service Level Agreements) profiles in SD-WAN.

As more and more customers are planning to integrate and want to have seamless traffic treatment across the path, Cisco has come up with SD-DC and SD-WAN Integration.

The integration focuses on two use cases:

Traffic from ACI (DC) to SDWAN (non ACI Branch)
Traffic from SDWAN (non ACI Branch) to ACI (DC)

Prerequisites

Requirements

As the integration with SD-WAN happens over the L3 out configured in ACI, so L3out with supported protocol must be configured.

Integration takes place over management network so Management reachability between ACI (APIC controllers) and vManage is required.

Components Used

ACI Fabric, SDWAN (vManage, vSmart Controller, vEdge)

This document is based on ACI version 4.2(3l)

Configuration

Network Diagram

Topology for reference:

In our topology, consider only ACI site A as DC and non-ACI Site C as SDWAN Branch site.

Topology

Configurations

Section A: Integration Configuration

Open the APIC Graphical User Interface (GUI) and navigate to Integrations tab under System tab.

Integrations

Create Integration Group

Create Group

Navigate to newly created Integration Group “SDWAN2” and right click on vManage

vManage

Right click on vManage and select Create Integration Manager

Create Integration Manager

Fill in the appropriate details like integration manager name, Device IP/FQDN, username, password

Create Integration

Ensure that registration is successful from status field. If it is not successfull or if any errors observed, verify if provided information is correct. Partner ID is identifier of vManage controller. You can navigate to Integrations -><Group Name>->vManage -> <Integration Manager Name> -> System info to verify the status.

System Info

Section B: Configuration of WAN SLA policy

Pre-configured WAN SLA profiles can be found under Tenants->common->Policies->Protocols->WAN SLA

This can be inherited in other tenant while configuring the contract using WAN SLA policy.

These are pre-configured SLAs and cannot be changed.

WAN SLA

VPN configured on SD-WAN side which is mapped to this ACI integration will also be reflected under Tenants->common->Policies->Protocols->WAN SLA

WAN VPN

Create the contract under the tenant/VRF where you want to map the WAN services.

The QoS Priority value must be set to any value other than Unspecified. The WAN SLA policies will not work if the QoS Priority value is set to Unspecified.

Please navigate to Tenants-><tenant name>->Contracts->Standard

Create Contract

Create the Contract Subject and Under Contract Subject, specify WAN SLA Policy.

The QoS Priority value must be set to any value other than Unspecified. The WAN SLA policies will not work if the QoS Priority value is set to Unspecified.

Create Contract Subject

Provide the contract from EPG.

Please navigate to Tenants-><tenant name>->Application Profiles->Application EPG->Contracts

Add Provided Contract

Consume the contract at L3out configured for SD-WAN

Please navigate to Tenants-><tenant name>->L3outs->External EPG->Consumed Contracts. It is also possible and valid to have contract provided by L3out External EPG and consumed by EPGs

Consumed Contracts

Match a WAN VPN to a tenant VRF

Please navigate to Tenants-><tenant name>->VRFs->Policy->WAN VPN

Policy

Verify

Section 3: Verification

Configuration verification

Configuration is pushed to both SDWAN devices according to configuration in ACI

DC end (connected to L3out) SDWAN route

ASR1001-X-DC#show sdwan policy from-vsmart
-->>> SLA Policy (parameters)
from-vsmart sla-class Bulk-Data
 loss    10
 latency 300
 jitter  100

from-vsmart sla-class Default
 loss    25
 latency 300
 jitter  100

from-vsmart sla-class Transactional-Data
 loss    5
 latency 50
 jitter  100

from-vsmart sla-class Voice-And-Video
 loss    2
 latency 45
 jitter  100

from-vsmart data-policy _vpn-10_data_policy
 direction from-service
 vpn-list vpn-10
  default-action accept

-->>> DSCP to SLA Mapping
from-vsmart app-route-policy _412898115_vpn_412898115
 vpn-list 412898115_vpn
  sequence 10
   match
    dscp 14
   action
    sla-class Default
    no sla-class strict
  sequence 20
   match
    dscp 18
   action
    sla-class Voice-And-Video
    no sla-class strict
  sequence 30
   match
    dscp 12
   action
    sla-class Transactional-Data
    no sla-class strict
  sequence 40
   match
    dscp 10
   action
    sla-class Bulk-Data
    no sla-class strict

from-vsmart lists vpn-list 412898115_vpn
 vpn 10

from-vsmart lists vpn-list vpn-10
 vpn 10

ASR1001-X-DC#

Branch end SDWAN router

ASR1001-X-Branch#show sdwan policy from-vsmart
-->>> SLA Policy (parameters)
from-vsmart sla-class Bulk-Data
 loss    10
 latency 300
 jitter  100

from-vsmart sla-class Default
 loss    25
 latency 300
 jitter  100

from-vsmart sla-class Transactional-Data
 loss    5
 latency 50
 jitter  100

from-vsmart sla-class Voice-And-Video
 loss    2
 latency 45
 jitter  100

-->>> DSCP to SLA Mapping
from-vsmart app-route-policy _412898115_vpn_412898115
 vpn-list 412898115_vpn
  sequence 10
   match
    dscp 14
   action
    sla-class Default
    no sla-class strict
  sequence 20
   match
    dscp 18
   action
    sla-class Voice-And-Video
    no sla-class strict
  sequence 30
   match
    dscp 12
   action
    sla-class Transactional-Data
    no sla-class strict
  sequence 40
   match
    dscp 10
   action
    sla-class Bulk-Data
    no sla-class strict

from-vsmart lists vpn-list 412898115_vpn
 vpn 10

ASR1001-X-Branch#

QoS verification

Example 1

WAN SLA Policy "Transactional-Data". Please navigate to Tenants-><tenant name>->Contracts->Standard-><Contract Name>-><Contract Subject>-> General- WAN SLA Policy

WAN SLA Policy 1

  sequence 30
   match
    dscp 12
   action
    sla-class Transactional-Data
    no sla-class strict

Direction:

1. Traffic from DC to SDWAN.

As can be seen in below captures, traffic originated from DC is with dscp 00 but the traffic reaching to SDWAN is with DSCP 12 (hex 0x0c).

This indicates DSCP value change according to WAN SLA Policy.

Packet capture performed at source (DC) reflecting original DSCP value to 00.

Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

0000 00.. = Differentiated Services Codepoint: Default (0x00)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa0d5 (41173)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 255

Protocol: ICMP (0x01)

Header checksum: 0x9016 [correct]

[Good: True]

[Bad : False]

Source: 192.168.10.2 (192.168.10.2)

Destination: 172.16.20.2 (172.16.20.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0xc16a [correct]

Identifier: 0x4158

Sequence number: 768 (0x0300)

Data (56 bytes)

Packet capture on destination (SDWAN Branch site) reflecting change in DSCP 12 (hex 0x0c) value according to WAN SLA Policy.

Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x30 (DSCP 0x0c: Assured Forwarding 12; ECN: 0x00)

0011 00.. = Differentiated Services Codepoint: Assured Forwarding 12 (0x0c)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa0d1 (41169)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 251

Protocol: ICMP (0x01)

Header checksum: 0x93ea [correct]

[Good: True]

[Bad : False]

Source: 192.168.10.2 (192.168.10.2)

Destination: 172.16.20.2 (172.16.20.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0x6e30 [correct]

Identifier: 0xc057

Sequence number: 1024 (0x0400)

Data (56 bytes)

2. Traffic from SDWAN to DC

As can be seen in below captures, traffic originated from SDWAN Branch site is with dscp 00 but the traffic reaching to DC is with DSCP 12 (hex 0x0c) reflecting the change in DSCP value according to WAN SLA Policy applied.

Packet capture performed at source (SDWAN Branch) reflecting original DSCP value to 00.

Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

0000 00.. = Differentiated Services Codepoint: Default (0x00)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa0c8 (41160)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 255

Protocol: ICMP (0x01)

Header checksum: 0x9023 [correct]

[Good: True]

[Bad : False]

Source: 172.16.20.2 (172.16.20.2)

Destination: 192.168.10.2 (192.168.10.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0xd3ff [correct]

Identifier: 0x5c79

Sequence number: 1 (0x0001)

Data (56 bytes)

Packet capture on destination (DC) reflecting change in DSCP 12 (hex 0x0c) value according to WAN SLA Policy.

Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x30 (DSCP 0x0c: Assured Forwarding 12; ECN: 0x00)

0011 00.. = Differentiated Services Codepoint: Assured Forwarding 12 (0x0c)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa073 (41075)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 251

Protocol: ICMP (0x01)

Header checksum: 0x9448 [correct]

[Good: True]

[Bad : False]

Source: 172.16.20.2 (172.16.20.2)

Destination: 192.168.10.2 (192.168.10.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0x741a [correct]

Identifier: 0x5c79

Sequence number: 43776 (0xab00)

Data (56 bytes)

Example 2

WAN SLA Policy "Voice-And-Video" Please navigate to Tenants-><tenant name>->Contracts->Standard-><Contract Name>-><Contract Subject>-> General- WAN SLA Policy

WAN SLA Policy 2

  sequence 20
   match
    dscp 18
   action
    sla-class Voice-And-Video
    no sla-class strict

1. Traffic from DC to SDWAN.

As can be seen in below captures, traffic originated from DC is with DSCP 00 but the traffic reaching to SDWAN is with DSCP 18 (hex 0x12).

This indicates DSCP value change according to WAN SLA Policy.

Packet capture performed at source (DC) reflecting original DSCP value to 00.

Internet Protocol, Src: 192.168.10.2 (192.168.10.2), Dst: 172.16.20.2 (172.16.20.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

0000 00.. = Differentiated Services Codepoint: Default (0x00)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa2b6 (41654)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 255

Protocol: ICMP (0x01)

Header checksum: 0x8e35 [correct]

[Good: True]

[Bad : False]

Source: 192.168.10.2 (192.168.10.2)

Destination: 172.16.20.2 (172.16.20.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0x3614 [correct]

Identifier: 0x8c5f

Sequence number: 512 (0x0200)

Data (56 bytes)

Packet capture on destination (SDWAN Branch site) reflecting change in DSCP value 18 (0x12) matching it with WAN SLA Policy.

Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x48 (DSCP 0x12: Assured Forwarding 21; ECN: 0x00)

0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (0x12)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa2b8 (41656)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 255

Protocol: ICMP (0x01)

Header checksum: 0x8deb [correct]

[Good: True]

[Bad : False]

Source: 172.16.20.2 (172.16.20.2)

Destination: 192.168.10.2 (192.168.10.2)

Internet Control Message Protocol

Type: 0 (Echo (ping) reply)

Code: 0 ()

Checksum: 0x8a13 [correct]

Identifier: 0x8c5f

Sequence number: 1024 (0x0400)

Data (56 bytes)

2. Traffic from SDWAN to DC.

Packet capture on source (SDWAN Branch) showing the original DSCP value (00).

Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)

0000 00.. = Differentiated Services Codepoint: Default (0x00)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa1bb (41403)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 255

Protocol: ICMP (0x01)

Header checksum: 0x8f30 [correct]

[Good: True]

[Bad : False]

Source: 172.16.20.2 (172.16.20.2)

Destination: 192.168.10.2 (192.168.10.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0x68e5 [correct]

Identifier: 0x1d03

Sequence number: 2048 (0x0800)

Data (56 bytes)

Packet capture on destination (DC) reflecting change in DSCP value 18 (0x12) according to WAN SLA Policy.

Internet Protocol, Src: 172.16.20.2 (172.16.20.2), Dst: 192.168.10.2 (192.168.10.2)

Version: 4

Header length: 20 bytes

Differentiated Services Field: 0x48 (DSCP 0x12: Assured Forwarding 21; ECN: 0x00)

0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (0x12)

.... ..0. = ECN-Capable Transport (ECT): 0

.... ...0 = ECN-CE: 0

Total Length: 84

Identification: 0xa1bb (41403)

Flags: 0x00

0.. = Reserved bit: Not Set

.0. = Don't fragment: Not Set

..0 = More fragments: Not Set

Fragment offset: 0

Time to live: 251

Protocol: ICMP (0x01)

Header checksum: 0x92e8 [correct]

[Good: True]

[Bad : False]

Source: 172.16.20.2 (172.16.20.2)

Destination: 192.168.10.2 (192.168.10.2)

Internet Control Message Protocol

Type: 8 (Echo (ping) request)

Code: 0 ()

Checksum: 0x68e5 [correct]

Identifier: 0x1d03

Sequence number: 2048 (0x0800)

Data (56 bytes)

Troubleshoot

Following log files are useful from troubleshooting perspective. .

Control path debugging

APIC techsupport files
PolicyDistributor Logs, PolicyManager Logs, PolicyElement, Edmgr logs can provide insight about relevant configuration getting pushed to leaves and spines.

Data path debugging

Packet captures on L3out interface and interfaces on vEdge routers.

ELAM can also help.

Revision History

Revision	Publish Date	Comments
1.0	06-Aug-2021	Initial Release

Contributed by Cisco Engineers

Bharatkumar Chaudhari
ACI TAC

Was this Document Helpful?

Feedback

Contact Cisco

Open a Support Case
(Requires a Cisco Service Contract)

This Document Applies to These Products

Application Policy Infrastructure Controller (APIC)

Configuration and Verification of SDWAN Integration with ACI

Available Languages

Download Options

Bias-Free Language

Contents

Acronyms

Introduction

Prerequisites

Requirements

Components Used

Configuration

Network Diagram

Configurations

Verify

Troubleshoot

Revision History

Contributed by Cisco Engineers

Was this Document Helpful?

Contact Cisco

This Document Applies to These Products