Security protocols derived from SSL provide the foundation for the Cisco Catalyst SD-WAN control plane infrastructure.

The first is the DTLS protocol, which is a transport privacy protocol for connectionless datagram protocols such as UDP, provides the foundation for the Cisco Catalyst SD-WAN control plane infrastructure. It is based on the stream-oriented Transport Layer Security (TLS) protocol, which provides security for TCP-based traffic. (TLS itself evolved from SSL.) The Cisco Catalyst SD-WAN infrastructure design uses DTLS running over UDP to avoid some of the issues with TCP, including the delays associated with stream protocols and some security issues. However, because UDP performs no handshaking and sends no acknowledgments, DTLS has to handle possible packet re-ordering, loss of datagrams, and data larger than the datagram packet size.

The control plane infrastructure can also be configured to run over TLS. This might be desirable in situations where the protections of TCP outweigh its issues. For example, firewalls generally offer better protection for TCP servers than for UDP servers.

The Cisco Catalyst SD-WAN software implements the standard version of DTLS with UDP, which is defined in RFC 6347. DTLS for use with other protocols is defined in a number of other RFCs. For TLS, the Cisco Catalyst SD-WAN software implements the standard version defined in RFC 5246. As described in the RFCs, Cisco Catalyst SD-WAN uses DTLS and TLS versions 1.2.

In the Cisco Catalyst SD-WAN architecture, the Cisco Catalyst SD-WAN devices use DTLS or TLS as a tunneling protocol, which is an application-level (Layer 4) tunneling protocol. When the Cisco SD-WAN Controller, Cisco SD-WAN Validator, Cisco SD-WAN Managers, and routers join the network, they create provisional DTLS or TLS tunnels between them as part of the device authentication process. After the authentication process completes successfully, the provisional tunnels between the routers and Cisco SD-WAN Controller, and those between the Cisco SD-WAN Validator and Cisco SD-WAN Controller, become permanent and remain up as long as the devices are active in the network. It is these authenticated, secure DTLS or TLS tunnels that are used by all the protocol applications running on the Cisco Catalyst SD-WAN devices to transport their traffic. For example, an OMP session on a router communicates with an OMP session on a Cisco SD-WAN Controller by sending plain IP traffic through the secure DTLS or TLS tunnel between the two devices. The Overlay Management Protocol is the Cisco Catalyst SD-WAN control protocol used to exchange routing, policy, and management information among Cisco Catalyst SD-WAN devices, as described in Overlay Routing Overview.

A Cisco Catalyst SD-WAN daemon running on each Cisco SD-WAN Controller and router creates and maintains the secure DTLS or TLS connections between the devices. This daemon is called vdaemon and is discussed later in this article. After the control plane DTLS or TLS connections are established between these devices, multiple protocols can create sessions to run and route their traffic over these connections—including OMP, Simple Network Management Protocol (SNMP), and Network Configuration Protocol (Netconf)—without needing to be concerned with any security-related issues. The session-related traffic is simply directed over the secure connection between the routers and Cisco SD-WAN Controller.

The Cisco SD-WAN control plane uses digital certificates with 2048-bit RSA keys to authenticate the Cisco vEdge devices in the network. The digital certificates are created, managed, and exchanged by standard components of the public key infrastructure (PKI):

• Public keys— These keys are generally known.

• Private keys— These keys are private. They reside on each Cisco vEdge device and cannot be retrieved from the Cisco vEdge device.

• Certificates signed by a root certification authority (CA)— The trust chain associated with the root CA needs to be present on all Cisco vEdge devices.

In addition to standard PKI components, the Cisco SD-WAN Controller serial numbers and the router chassis numbers are used in the authentication processes.

Let's first look at the PKI components that are involved in router authentication. On vEdge 100, 1000, and 2000 routers, the public and private keys and the certificates are managed automatically, by a Trusted Board ID chip that is built into the router. For vEdge 5000, instead of a Trusted Board ID chip, a Trusted Platform Module (TPM) is used. When the routers are manufactured, this chip is programmed with a signed certificate. This certificate includes the router's public key, its serial number, and the router's private key. When the routers boot up and join the network, they exchange their certificates (including the router's public key and serial number) with other Cisco Catalyst SD-WAN routers as part of the router authentication process. Note that the router's private key always remains embedded in the router's chip, and it is never distributed, nor can it ever be retrieved from the router. In fact, any brute-force attempt to read the private key causes the chip to fail, thereby disabling all access to the router.

For Cisco SD-WAN Controllers, Cisco Catalyst SD-WAN Validators, and Cisco SD-WAN Manager systems, the public and private keys and the certificates are managed manually. When you boot these routers for the first time, the Cisco SD-WAN Controller software generates a unique private key–public key pair for each software image. The public key needs to be signed by the CA root. The network administrator then requests a signed certificate and manually installs it and the certificate chains on the Cisco SD-WAN Controllers, Cisco Catalyst SD-WAN Validators, and Cisco SD-WAN Manager systems. A typical network might have only a small handful of Cisco SD-WAN Controllers, Cisco Catalyst SD-WAN Validators, and Cisco SD-WAN Managers, so the burden of manually managing the keys and certificates on these routers is small.

When you place an order with Cisco using your Smart and Virtual Account, Cisco updates the Cisco Plug and Play (PNP) Portal with the chassis and certificate serial numbers of the devices that you purchased. You can then use Cisco SD-WAN Manager to sync the device information from the PNP portal using your Smart Account credentials. Alternatively. you can also download the trusted WAN Edge serial file from the PNP portal and upload it manually to Cisco SD-WAN Manager. Cisco SD-WAN Manager then broadcasts this information to the other controllers. Both the authorized serial number file and the file listing the Cisco SD-WAN Controller serial numbers are uploaded and installed on Cisco Catalyst SD-WAN Validators. Then, during the automatic authentication process, as pairs of devices (routers and controllers) are establishing DTLS control connections, each device compares the serial numbers (and for routers, the chassis numbers) to those in the files installed on the router. A router allows a connection to be established only if the serial number or serial–chassis number combination (for a router) matches. Note that routers only make control connections to the controllers and not to other routers.

You can display the installed Cisco SD-WAN Controller authorized serial numbers using the show control valid-vsmarts command on a Cisco SD-WAN Controller or a router and the show orchestrator valid-vsmarts command on a Cisco Catalyst SD-WAN Validator. You can also run show sdwan control valid-vsmarts on Cisco IOS XE Catalyst SD-WAN devices. You can display the installed router authorized serial and chassis number associations using the show control valid-vedges command on a Cisco SD-WAN Controller and the show orchestrator valid-devices command on a Cisco Catalyst SD-WAN Validator.

Now, let's look at how the PKI authentication components and the router serial and chassis numbers are used to authenticate router on the Cisco SD-WAN Controller overlay network. When Cisco SD-WAN Controllers, Cisco Catalyst SD-WAN Validators, and routers first boot up, they establish secure DTLS or TLS connections between the Cisco SD-WAN Controllers and the routers. Over these connections, the devices authenticate each other, using the public and private keys, the signed certificates, and the routers serial numbers and performing a series of handshake operations to ensure that all the devices on the network are valid and not imposters. The following figure illustrates the key and certificate exchange that occurs when the Cisco SD-WAN Controller devices boot. For details about the authentication that occurs during the bringup process, see Bringup Sequence of Events.

Control plane encryption is done by either DTLS, which is based on the TLS protocol, or TLS. These protocol encrypt the control plane traffic that is sent across the connections between Cisco Catalyst SD-WAN devices to validate the integrity of the data. TLS uses asymmetric cryptography for authenticating key exchange, symmetric encryption for confidentiality, and message authentication codes for message integrity.

A single Cisco Catalyst SD-WAN device can have DTLS or TLS connections to multiple Cisco Catalyst SD-WAN devices, so vdaemon creates a kernel route for each destination. For example, a router would typically have one kernel route, and hence one DTLS or TLS connection, for each Cisco SD-WAN Controller. Similarly, a Cisco SD-WAN Controller would have one kernel route and one DTLS or TLS connection for each router in its domain.

The Cisco Catalyst SD-WAN design implements control plane integrity by combining two security elements: AES-GCM message digests, and public and private keys.

AES-GCM authenticated encryption provides high performance encryption that generates message digests (sometimes called simply digests) for each packet sent over a control plane connection. The receiver then generates a digest for the packet, and if the two match, the packet is accepted as valid. This encryption allows verification that the packet's contents have not been tampered with.

The second component of control plane integrity is the use of public and private keys. When a control plane connection is being established, a local Cisco Catalyst SD-WAN device sends a challenge to a remote device. The remote device encrypts the challenge by signing it with its private key, and returns the signed challenge to the local device. The local device then uses the remote device's public key to verify that the received challenge matches the sent challenge.

Then, once a control plane connection is up, keys are used to ensure that packets have been sent by a trusted host and were not inserted midstream by an untrusted source. The authenticity of each packet is verified through encryption and decryption with symmetric keys that were exchanged during the process of establishing the control connection.

During the bringup of the overlay, the Cisco Catalyst SD-WAN Controller establishes the information for edge routers to send data to each other. However before a pair of routers can exchange data traffic, they establish an IPsec connection between them, which they use as a secure communications channel. Since the Cisco Catalyst SD-WAN Controller has authenticated the devices, the devices do not further authenticate each other.

Control plane communications have allowed the edge device to have enough information to establish IPsec tunnels. Edge devices simply send data through the tunnels. There is no additional authentication step.

In a traditional IPsec environment, key exchange is handled by the Internet Key Exchange (IKE) protocol. IKE first sets up secure communications channels between devices and then establishes security associations (SAs) between each pair of devices that want to exchange data. IKE uses a Diffie-Hellman key exchange algorithm to generate a shared key that encrypts further IKE communication. To establish SAs, each device (n) exchanges keys with every other device in the network and creates per-pair keys, generating a unique key for each remote device. This scheme means that in a fully meshed network, each device has to manage n² key exchanges and (n-1) keys. As an example, in a 1,000-node network, 1,000,000 key exchanges are required to authenticate the devices, and each node is responsible for maintaining and managing 999 keys.

The discussion in the previous paragraph points out why an IKE-style key exchange does not scale as network size increases and why IKE could be a bottleneck in starting and in maintaining data exchange on a large network:

• The handshaking required to set up the communications channels is both time consuming and resource intensive.

• The processing required for the key exchange, especially in larger networks, can strain network resources and can take a long time.

The Cisco Catalyst SD-WAN implementation of data plane authentication and encryption establishes SAs between each pair of devices that want to exchange data, but it dispenses with IKE altogether. Instead, to provide a scalable solution to data plane key exchange, the Cisco Catalyst SD-WAN solution takes advantage of the fact that the DTLS control plane connections in the Cisco Catalyst SD-WAN overlay network are known to be secure. Because the Cisco Catalyst SD-WAN control plane establishes authenticated, encrypted, and tamperproof connections, there is no need in the data plane to set up secure communications channels to perform data plane authentication.

In the Cisco Catalyst SD-WAN network for unicast traffic, data plane encryption is done by AES-256-GCM, a symmetric-key algorithm that uses the same key to encrypt outgoing packets and to decrypt incoming packets. Each router periodically generates an AES key for its data path (specifically, one key per TLOC) and transmits this key to the Cisco SD-WAN Controller in OMP route packets, which are similar to IP route updates. These packets contain information that the Cisco SD-WAN Controller uses to determine the network topology, including the router's TLOC (a tuple of the system IP address and traffic color) and AES key. The Cisco SD-WAN Controller then places these OMP route packets into reachability advertisements that it sends to the other routers in the network. In this way, the AES keys for all the routers are distributed across the network. Even though the key exchange is symmetric, the routers use it in an asymmetric fashion. The result is a simple and scalable key exchange process that uses the Cisco Catalyst SD-WAN Controller.

In Cisco SD-WAN Release 19.2.x and Cisco IOS XE SD-WAN Release 16.12.x onwards, Cisco Catalyst SD-WAN supports IPSec pairwise keys that provide additional security. When IPSec pairwise keys are used, the edge router generates public and private Diffie-Hellman components and sends the public value to the Cisco SD-WAN Controller for distribution to all other edge devices. For more information, see IPsec Pairwise Keys

If control policies configured on a Cisco SD-WAN Controller limit the communications channels between network devices, the reachability advertisements sent by the Cisco SD-WAN Controller contain information only for the routers that they are allowed to exchange data with. So, a router learns the keys only for those routers that they are allowed to communicate with.

To further strengthen data plane authentication and encryption, routers regenerate their AES keys aggressively (by default, every 24 hours). Also, the key regeneration mechanism ensures that no data traffic is dropped when keys change.

In the Cisco Catalyst SD-WAN overlay network, the liveness of SAs between router peers is tracked by monitoring BFD packets, which are periodically exchanged over the IPsec connection between the peers. IPsec relays the connection status to the Cisco SD-WAN Controllers. If data connectivity between two peers is lost, the exchange of BFD packets stops, and from this, the Cisco SD-WAN Controller learns that the connection has been lost.

The IPsec software has no explicit SA idle timeout, which specifies the time to wait before deleting SAs associated with inactive peers. Instead, an SA remains active as long as the IPsec connection between two routers is up, as determined by the periodic exchange of BFD packets between them. Also, the frequency with which SA keys are regenerated obviates the need to implement an implicit SA idle timeout.

In summary, the Cisco Catalyst SD-WAN data plane authentication offers the following improvements over IKE:

• Because only n +1 keypaths are required rather than the n² required by IKE, the Cisco Catalyst SD-WAN solution scales better as the network grows large.

• Keys are generated and refreshed locally, and key exchange is performed over a secure control plane.

The following components contribute to the integrity of data packets in the Cisco Catalyst SD-WAN data plane:

• UDP – Encapsulate ESP within UDP packets per RFC 3948, UDP Encapsulation of IPsec ESP packets.

• ESP, which is a standard IPsec encryption protocol, protects (via encryption and authentication) the inner header, data packet payload, and ESP trailer in all data packets. SDWAN complies with RFC 4303, IP Encapsulating Security Payload (ESP).

• Enhancements to ESP, which protect (via authentication) the outer IP and UDP headers. This mimics the functionality of the AH protocol.

• Anti-replay, which is also part of the standard IPsec software suite, provides a mechanism to number all data packets and to ensure that receiving routers accept only packets with unique numbers.

The first of these components, ESP, is the standard IPsec encryption protocol. ESP protects a data packet’s payload and its inner IP header fields both by encryption, which occurs automatically, and authentication. For authentication, ESP performs a hash calculation on the data packet's payload and inner header fields using AES-GCM and places the resultant hash (also called a digest) into a field at the end of the packet. (A hash is a one-way compression.) The receiving device performs the same checksum and compares its calculated hash with that in the packet. If the two checksums match, the packet is accepted. Otherwise, it is dropped. In the figure below, the left stack illustrates the ESP/UDP encapsulation. ESP encrypts and authenticates the inner headers, payload, MPLS label (if present), and ESP trailer fields, placing the hash in the ICV checksum field at the end of the packet. The outer header fields added by ESP/UDP are neither encrypted nor authenticated.

In the Cisco Catalyst SD-WAN solution, there are also enhancements to ESP to enhance its behavior to cover more of the datagram. These enhancements are similar to the way that AH works. This enhancement performs a checksum that includes calculating the checksum over all the fields in the packet—the payload, the inner header, and also all the non-mutable fields in the outer IP header. AH places the resultant hash into the last field of the packet. The receiving device performs the same checksum, and accepts packets whose checksums match. In the figure below, the center stack illustrates the encapsulation performed by the enhanced version of ESP. ESP again encrypts the inner headers, payload, MPLS label (if present), and ESP trailer fields, and now mimics AH by authenticating the entire packet—the outer IP and UDP headers, the ESP header, the MPLS label (if present), the original packet, and the ESP trailer—and places its calculated hash into the ICV checksum field at the end of the packet.

For situations in which data packet authentication is not required, you can disable data packet authentication altogether. In this case, data packets are processed just by ESP, which encrypts the original packet, the MPLS label (if present), and the ESP trailer. This scheme is illustrated in the right stack in the figure below.

Note that Cisco Catalyst SD-WAN devices exchange not only the encryption key (which is symmetric), but also the authentication key that is used to generate the digest. Both are distributed as part of the TLOC properties for a router.

Even though the IPsec connections over which data traffic is exchanged are secure, they often travel across a public network space, such as the Internet, where it is possible for a hacker to launch a replay attack (also called a man-in-the-middle, or MITM, attack) against the IPsec connection. In this type of attack, an adversary tampers with the data traffic by inserting a copy of a message that was previously sent by the source. If the destination cannot distinguish the replayed message from a valid message, it may authenticate the adversary as the source or may incorrectly grant to the adversary unauthorized access to resources or services.

As a counter to such attacks, the Cisco Catalyst SD-WAN overlay network software implements the IPsec anti-replay protocol. This protocol consists of two components, both of which protect the integrity of a data traffic stream. The first component is to associate sequence numbers with each data packets. The sender inserts a sequence number into each IPsec packet, and the destination checks the sequence number, accepting only packets with unique, non-duplicate sequence numbers. The second component is a sliding window, which defines a range of sequence numbers that are current. The sliding window has a fixed length. The destination accepts only packets whose sequence numbers fall within the current range of values in the sliding window, and it drops all others. A sliding window is used rather than accepting only packets whose sequence number is larger than the last known sequence number, because packets often do not arrive in order.

When the destination receives a packet whose sequence number is larger than the highest number in the sliding window, it slides the window to the right, thus changing the range of valid sequences numbers it will accept. This scheme protects against an MITM type of attack because, by choosing the proper window size, you can ensure that if a duplicate packet is inserted into the traffic stream, its sequence number will either be within the current range but will be a duplicate, or it will be smaller than the lowest current value of the sliding window. Either way, the destination will drop the duplicate packet. So, the sequence numbering combined with a sliding window provide protection against MITM type of attacks and ensure the integrity of the data stream flowing within the IPsec connection.