Bringing Up the Overlay Network

The following table lists the tasks for bringing up the overlay network using Cisco SD-WAN Manager.

Generally, what you do to bring up the Cisco Catalyst SD-WAN overlay network is what you do to bring up any network. You plan out the network, create device configurations, and then deploy the network hardware and software components. These components include all the Cisco vEdge devices, all the traditional routers that participate in the overlay network, and all the network devices that provide shared services across the overlay network, such as firewalls, load balancers, and IDP systems.​

The following table summarizes the steps for the user portion of the Cisco Catalyst SD-WAN overlay network bring-up sequence. The details of each step are provided in the articles that are listed in the Procedure column. While you can bring up the Cisco vEdge devices in any order, it is recommended that you deploy them in the order listed below, which is the functional order in which the devices verify and authenticate themselves.

If your network has firewall devices, see Firewall Ports for Cisco Catalyst SD-WAN Deployments.

Table 3.

	Workflow	Procedure
1		Plan out your overlay network. See Components of the Cisco Catalyst SD-WAN Solution.
2		On paper, create device configurations that implement the desired architecture and functionality. See the Software documentation for your software release.
3		Download the software images.
4		Deploy Cisco SD-WAN Manager in the data center: Create a Cisco SD-WAN Manager VM instance, either on an ESXi or a KVM hypervisor. Create either a minimal or a full configuration for each Cisco SD-WAN Manager server. Configure certificate settings and generate a certificate for Cisco SD-WAN Manager. Create a Cisco SD-WAN Manager cluster.
5		Deploy the Cisco SD-WAN Validator: Create a Cisco SD-WAN Validator VM instance, either on an ESXi or a KVM hypervisor. Create a minimal configuration for the Cisco SD-WAN Validator. Add the Cisco SD-WAN Validator to the overlay network. During this process, you generate a certificate for the Cisco SD-WAN Validator. Create a full configuration for the Cisco SD-WAN Validator.
6		Deploy the Cisco SD-WAN Controller in the data center: Create a Cisco SD-WAN Controller VM instance, either on an ESXi or a KVM hypervisor. Create a minimal configuration for the Cisco SD-WAN Controller. Add the Cisco SD-WAN Controller to the overlay network. During this process, you generate a certificate for the Cisco SD-WAN Controller. Create a full configuration for the Cisco SD-WAN Controller.
7		Deploy the Cisco vEdge routers in the overlay network: For software vEdge Cloud routers, create a VM instance, either on an AWS server, or on an ESXi or a KVM hypervisor. For software vEdge Cloud routers, send a certificate signing request to Symantec and then install the signed certificate on the router. From Cisco SD-WAN Manager, send the serial numbers of all Cisco vEdge routers to the Cisco SD-WAN Controller and Cisco SD-WAN Validator in the overlay network. Create a full configuration for the Cisco vEdge routers.

After the Cisco vEdge devices boot and start running with their initial configurations, the second part of the bring-up process begins automatically. This automatic process is led by the Cisco SD-WAN Validator, as illustrated in the figure below. Under the leadership of the Cisco SD-WAN Validator software, the Cisco vEdge devices set up encrypted communication channels between themselves. Over these channels, the devices automatically validate and authenticate each other, a process that establishes an operational overlay network. Once the overlay network is running, the Cisco vEdge devices automatically receive and activate their full configurations from the Cisco SD-WAN Manager server. (The exception is the Cisco SD-WAN Manager. You must manually configure each Cisco SD-WAN Manager server itself).

The following sections explain what happens under the covers, during the automatic portion of the bring-up process. This explanation is provided to help you understand the detailed workings of the Cisco Catalyst SD-WAN software so that you can better appreciate the means by which the Cisco Catalyst SD-WAN solution creates a highly secure overlay framework to support your networking requirements.

The automatic validation and authentication of Cisco vEdge devices that occurs during the bringup process can happen only if Cisco SD-WAN Controllers and Cisco SD-WAN Validators know the serial and chassis numbers of the devices that are permitted in the network. Let's first define these two terms:

• Serial number—​Each Cisco vEdge device has a serial number, which is a 40-byte number that is included in the device's certificate. For Cisco SD-WAN Validator and Cisco SD-WAN Controller, the certificate can be provided by Symantec or by an enterprise root CA. For the vEdge routers, the certificate is provided in the hardware's trusted board ID chip.

• Chassis number—In addition to a serial number, each vEdge router is identified by a chassis number. Because the vEdge router is the only Cisco SD-WAN Controller manufactured hardware, it is the only Cisco vEdge device that has a chassis number. There is a one-to-one mapping between a vEdge router's serial number and its chassis number.

The Cisco SD-WAN Controllers and Cisco SD-WAN Validators learn the serial and chassis numbers during the initial configuration of these devices:

• Cisco SD-WAN Controller authorized serial numbers—The Cisco SD-WAN Manager learns the serial numbers for all Cisco SD-WAN Controllers that are allowed to be in the network while it is creating a CSR and installing the signed certificate. You download these serial numbers to Cisco SD-WAN Validator, and Cisco SD-WAN Validator pushes them to the Cisco Catalyst SD-WAN Controller during the automatic authentication process.

• vEdge authorized serial number file—This file contains the serial and chassis numbers of all the vEdge routers that are allowed to be in the network. You upload this file to Cisco SD-WAN Validators and Cisco SD-WAN Controllers.

In addition to the device serial and chassis numbers, the automatic validation and authentication procedure depends on having each device configured with the same organization name. You configure this name on Cisco SD-WAN Manager, and it is included in the configuration file on all devices. The organization name must be identical on all the devices that belong to a single organization (the name is case-sensitive). The organization name is also included in the certificate for each device, which is created either by Cisco Catalyst SD-WAN or by an enterprise root CA.

From a functional point of view, the first two devices on the Cisco Catalyst SD-WAN overlay network that validate and authenticate each other are Cisco SD-WAN Controller and Cisco SD-WAN Validator. This process is initiated by Cisco SD-WAN Controller.

When Cisco SD-WAN Controller comes up, it initiates a connection to Cisco SD-WAN Validator, which is how Cisco SD-WAN Validator learns about Cisco SD-WAN Controller. These two devices then automatically begin a two-way authentication process—Cisco SD-WAN Controller authenticates itself with , and Cisco SD-WAN Validator authenticates itself with Cisco SD-WAN Validator. The two-way handshaking between the two devices during the authentication process occurs in parallel. However, for clarity, the figure here, which is a high-level representation of the authentication steps, illustrates the handshaking sequentially. If the authentication handshaking succeeds, a permanent DTLS communication channel is established between the Cisco SD-WAN Controller and Cisco SD-WAN Validator devices. If any one of the authentication steps fails, the device noting the failure tears down the connection between the two devices, and the authentication attempt terminates.

The Cisco SD-WAN Controller knows how to reach Cisco SD-WAN Validator, because one of the parameters that you provision when you configure it is the IP address or DNS name of Cisco SD-WAN Validator. Cisco SD-WAN Validator is primed to respond to requests from Cisco SD-WAN Validator because:

• It knows that its role is to be the authentication system, because you included this information in the Cisco SD-WAN Validator configuration.

• You downloaded the Cisco SD-WAN Controller authorized serial numbers from Cisco SD-WAN Manager to Cisco SD-WAN Validator.

If Cisco SD-WAN Validator has not yet started when Cisco SD-WAN Controller initiates the authentication process, Cisco SD-WAN Controller periodically attempts to initiate a connection until it is successful.

Below is a more detailed step-by-step description of how the automatic authentication occurs between Cisco SD-WAN Controller and Cisco SD-WAN Validator.

To initiate a session between Cisco SD-WAN Controller and Cisco SD-WAN Validator, Cisco SD-WAN Controller initiates an encrypted DTLS connection to Cisco SD-WAN Validator. The encryption is provided by RSA. Each device automatically generates an RSA private key‒public key pair when it boots.

Over this encrypted channel, Cisco SD-WAN Controller and Cisco SD-WAN Validator authenticate each other. Each device authenticates the other in parallel. For our discussion, let's start with Cisco SD-WAN Controller authentication of Cisco SD-WAN Validator:

1. Cisco SD-WAN Validator sends its trusted root CA signed certificate to the Cisco SD-WAN Controller.

2. Cisco SD-WAN Validator sends the vEdge authorized serial number file to the Cisco SD-WAN Controller.

3. Cisco SD-WAN Controller uses its chain of trust to extract the organization name from the certificate and compares it to the organization name that is configured on Cisco SD-WAN Controller. If the two organization names match, Cisco SD-WAN Controller knows that the organization of Cisco SD-WAN Validator is proper. If they do not match, Cisco Catalyst SD-WAN Controller tears down the DTLS connection.

4. Cisco Catalyst SD-WAN Controller uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, Cisco Catalyst SD-WAN Controller knows that the certificate itself is valid. If the signature is incorrect, Cisco Catalyst SD-WAN Controller tears down the DTLS connection.

After performing these two checks, Cisco SD-WAN Controller authentication of Cisco SD-WAN Validator is complete.

In the other direction, Cisco SD-WAN Validator​ authenticates Cisco SD-WAN Controller:

1. Cisco SD-WAN Controller sends its trusted root CA signed certificate to Cisco SD-WAN Validator.

2. Cisco SD-WAN Validator uses its chain of trust to extract Cisco SD-WAN Controller serial number from the certificate. The number must match one of the numbers in the Cisco SD-WAN Controller authorized serial number file. If there is no match, Cisco SD-WAN Validator tears down the DTLS connection.

3. Cisco SD-WAN Validator uses its chain of trust to extract the organization name from the certificate and compares it to the organization name that is configured on Cisco SD-WAN Validator. If the two organization names match, the Cisco SD-WAN Validator knows that the organization of Cisco SD-WAN Controller is proper. If they do not match, Cisco SD-WAN Validator tears down the DTLS connection.

4. The Cisco SD-WAN Validator uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, Cisco SD-WAN Validator knows that the certificate itself is valid. If the signature is incorrect, Cisco SD-WAN Validator tears down the DTLS connection.

After performing these three checks, the Cisco SD-WAN Validator authentication of Cisco SD-WAN Validator is complete.

After the bidirectional authentication completes between the two devices, the DTLS connection between Cisco SD-WAN Validator and Cisco SD-WAN Controller transitions from being a temporary connection to being a permanent connection, and the two devices establish an OMP session over the connection.

In a domain that has multiple Cisco SD-WAN Controllers for redundancy, this process repeats between each pair of Cisco SD-WAN Controller and Cisco SD-WAN Validator devices. In coordination with Cisco SD-WAN Validator, Cisco SD-WAN Controllers learn about each other and they synchronize their route information. It is recommended that you connect the different Cisco SD-WAN Controller to the WAN network through different NAT devices for higher availability.

A Cisco SD-WAN Validator has only as many permanent DTLS connections as the number of Cisco SD-WAN Controllers in the network topology. These DTLS connections are part of the network's control plane; no data traffic flows over them. After all Cisco SD-WAN Controllers have registered themselves with Cisco SD-WAN Validator, Cisco SD-WAN Validator and Cisco SD-WAN Controllers are ready to validate and authenticate the vEdge routers in the Cisco Catalyst SD-WAN network.

In a domain with multiple Cisco SD-WAN Controllers, the controllers must authenticate each other so that they can establish a full mesh of permanent DTLS connection between themselves for synchronizing OMP routes. Cisco SD-WAN Controller learns the IP address of the other Cisco SD-WAN Controller from Cisco SD-WAN Validator.

Cisco SD-WAN Controller learns about the possibility of other Cisco SD-WAN Controllers being present on the network during the authentication handshaking with the Cisco SD-WAN Validator, when it receives a copy of the Cisco SD-WAN Controller authorized serial number file. If this file has more than one serial number, it indicates that the network may, at some point, have multiple Cisco SD-WAN Controllers.

As one Cisco SD-WAN Controller authenticates with Cisco SD-WAN Validator, Cisco SD-WAN Validator sends Cisco SD-WAN Controller the IP address of other Cisco SD-WAN Controllers it has authenticated with. If Cisco SD-WAN Validator later learns of another Cisco SD-WAN Controller, it sends that controller's address to the other already authenticated Cisco SD-WAN Controllers.

Then, Cisco SD-WAN Controllers perform the steps below to authenticate each other. Again, each device authenticates the other in parallel, but for clarity, we describe the process sequentially.

1. Cisco SD-WAN Controller1 (vSmart1) initiates an encrypted DTLS connection to Cisco SD-WAN Controller2 (vSmart2) and sends its trusted root CA signed certificate to Cisco SD-WAN Controller2.

2. Cisco SD-WAN Controller2 uses its chain of trust to extract the Cisco SD-WAN Controller1's serial number. The number must match one of the numbers in the Cisco SD-WAN Controller authorized serial number file. If there is no match, Cisco SD-WAN Controller2 tears down the DTLS connection.

3. Cisco SD-WAN Controller2 uses its chain of trust to extract the organization name from the certificate and compares it to the locally configured organization name. If the two organization names match, Cisco SD-WAN Controller2 knows that the organization of Cisco SD-WAN Controller1 is proper. If they do not match, Cisco SD-WAN Controller2 tears down the DTLS connection.

4. Cisco SD-WAN Controller2 uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, Cisco SD-WAN Controller2 knows that the certificate itself is valid. If the signature is incorrect, Cisco SD-WAN Controller2 tears down the DTLS connection.

After performing these three checks, Cisco SD-WAN Controller2​ authentication of Cisco SD-WAN Controller1 is complete.

Now, Cisco SD-WAN Controller1 authenticates Cisco SD-WAN Controller2, performing the same steps as above.

1. First, Cisco SD-WAN Controller2 sends its trusted root CA signed certificate to Cisco SD-WAN Controller1.

2. Cisco SD-WAN Controller1 uses its chain of trust to extract the Cisco SD-WAN Controller2's serial number. The number must match one of the numbers in the Cisco SD-WAN Controller authorized serial number file. If there is no match, Cisco SD-WAN Controller1 tears down the DTLS connection.

3. Cisco SD-WAN Controller1 uses its chain of trust to extract the organization name from the certificate and compares it to the locally configured organization name. If the two organization names match, Cisco SD-WAN Controller2 knows that the organization of Cisco SD-WAN Controller2 is proper. If they do not match, Cisco SD-WAN Controller1 tears down the DTLS connection.

4. Cisco SD-WAN Controller1 uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, Cisco SD-WAN Controller2 knows that the certificate itself is valid. If the signature is incorrect, Cisco SD-WAN Controller1 tears down the DTLS connection.

After performing these three checks, Cisco SD-WAN Controller1​ authentication of Cisco SD-WAN Controller2 is complete, and the temporary DTLS connection between the two devices becomes permanent.

After all the Cisco SD-WAN Controllers have registered themselves with , Cisco SD-WAN Validator and Cisco SD-WAN Controllers are ready to validate and authenticate the vEdge routers in the Cisco Catalyst SD-WAN network.

When you deploy a Cisco vEdge router in the network, it first needs to do two things:

• Establish a secure connection with Cisco SD-WAN Manager so that it can receive its full configuration.

• Establish a secure connection with Cisco Catalyst SD-WAN Controller can begin participating in the Cisco Catalyst SD-WAN overlay network.

When a Cisco vEdge device comes up, how does it automatically discover Cisco SD-WAN Manager and Cisco Catalyst SD-WAN Controller and establish connections with them? It does so with help from Cisco SD-WAN Validator. The initial configuration on the Cisco vEdge router contains the Cisco SD-WAN Validator system’s IP address (or DNS name). Using this information, the Cisco vEdge router establishes a DTLS connection with Cisco SD-WAN Validator, and the two devices authenticate each other to confirm that they are valid Cisco vEdge devices. Again, this authentication is a two-way process that happens automatically. When the authentication completes successfully, Cisco SD-WAN Validator sends the Cisco vEdge router the IP addresses of Cisco SD-WAN Manager and Cisco Catalyst SD-WAN Controller. Then, the Cisco vEdge router tears down its connection with Cisco SD-WAN Validator and begins establishing secure DTLS connections with the othe two devices.

After you boot Cisco vEdge routers and manually perform the initial configuration, they automatically start looking for their Cisco SD-WAN Validator. Cisco SD-WAN Validator and Cisco SD-WAN Controllers are able to recognize and authenticate the Cisco vEdge routers in part because you have installed the Cisco vEdge authorized device list file on both these devices.

After you boot a Cisco vEdge router, you manually perform the initial configuration, at a minimum setting the IP address of DNS name of Cisco SD-WAN Validator. The Cisco vEdge router uses this address information to reach Cisco SD-WAN Validator. Cisco SD-WAN Validator is primed to respond to requests from a Cisco vEdge router because:

• It knows that its role is to be the authentication system, because you included this information in the initial Cisco SD-WAN Validator configuration.

• As part of the initial configuration, you installed the Cisco vEdge authorized serial number file on Cisco SD-WAN Validator.

If Cisco SD-WAN Validator has not yet started when a Cisco vEdge router initiates the authentication process, the Cisco vEdge router periodically attempts to initiate a connection until the attempt succeeds.

Below is a more detailed step-by-step description of how the automatic authentication occurs between Cisco SD-WAN Validator and a Cisco vEdge router.

First, the Cisco vEdge router initiates an encrypted DTLS connection to the public IP address of Cisco SD-WAN Validator. The encryption is provided by RSA. Each device automatically generates an RSA private key‒public key pair when it boots. Cisco SD-WAN Validator receives the Cisco vEdge router's original interface address and uses the outer IP address in the received packet to determine whether the Cisco vEdge router is behind a NAT. If it is, Cisco SD-WAN Validator creates a mapping of the Cisco vEdge router's public IP address and port to its private IP address.

Over this encrypted DTLS channel, the Cisco vEdge router and Cisco SD-WAN Validator proceed to authenticate each other. As with other device authentication, the Cisco vEdge router and Cisco SD-WAN Validator authenticate each other in parallel. We start our discussion by describing how the Cisco vEdge router authenticates Cisco SD-WAN Validator:

1. Cisco SD-WAN Validator sends its trusted root CA signed certificate to the Cisco vEdge router.​

2. The Cisco vEdge router uses its chain of trust to extract the organization name from the certificate and compares it to the organization name that is configured on the router itself. If the two organization names match, the Cisco vEdge routers knows that the organization of Cisco SD-WAN Validator is proper. If they do not match, the Cisco vEdge router tears down the DTLS connection.

3. The Cisco vEdge router uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, the Cisco vEdge router knows that the certificate itself is valid. If the signature is incorrect, the Cisco vEdge router tears down the DTLS connection.

After performing these two checks, the Cisco vEdge router knows that Cisco SD-WAN Validator is valid, and its authentication of Cisco SD-WAN Validator is complete.

In the opposite direction, Cisco SD-WAN Validator authenticates the Cisco vEdge router:

1. Cisco SD-WAN Validator sends a challenge to the Cisco vEdge router. The challenge is a 256-bit random value.

2. The Cisco vEdge router sends a response to the challenge that includes the following:

   • Cisco vEdge serial number

   • Cisco vEdge chassis number

   • Cisco vEdge board ID certificate

   • 256-bit random value signed by the Cisco vEdge router's private key

3. Cisco SD-WAN Validator compares the serial and chassis numbers to the list in its Cisco vEdge authorized device list file. The numbers must match one of the number pairs in the file. If there is no match, Cisco SD-WAN Validator tears down the DTLS connection.

4. Cisco SD-WAN Validator checks that the signing of the 256-bit random value is proper. It does this using the Cisco vEdge router's public key, which it extracts from the router's board ID certificate. If the signing is not correct, Cisco SD-WAN Validator tears down the DTLS connection.

5. Cisco SD-WAN Validator uses the root CA chain from the Cisco vEdge routers board ID certificate to validate that the board ID certificate is itself valid. If the certificate is not valid, Cisco SD-WAN Validator tears down the DTLS connection.

After performing these three checks, Cisco SD-WAN Validator knows that Cisco vEdge router is valid, and its authentication of the router is complete.

When the two-way authentication succeeds, Cisco SD-WAN Validator performs the final step of its orchestration, sending messages to the Cisco vEdge router and Cisco Catalyst SD-WAN Controller in parallel. To the Cisco vEdge router, Cisco SD-WAN Validator sends the following:

• The IP addresses of Cisco SD-WAN Controllers in the network so that the Cisco vEdge router can initiate connections to them. The address can be public IP addresses, or for the controllers that are behind a NAT gateway, the addresses are a list of the public and private IP addresses and port numbers. If the Cisco vEdge router is behind a NAT gateway, Cisco SD-WAN Validator requests that the Cisco vEdge router initiate a session with Cisco Catalyst SD-WAN Controller.

• Serial numbers of Cisco SD-WAN Controllers that are authorized to be in the network.

To Cisco Catalyst SD-WAN Controller, Cisco SD-WAN Validator sends the following:

• A message announcing the new Cisco vEdge router in the domain.

• If the Cisco vEdge router is behind a NAT gateway, Cisco SD-WAN Validator sends a request to Cisco Catalyst SD-WAN Controller to initiate a session with the Cisco vEdge router.

Then, the Cisco vEdge router tears down the DTLS connection with the Cisco SD-WAN Validator.

After the Cisco vEdge router and Cisco SD-WAN Validator have authenticated each other, the Cisco vEdge router receives its full configuration over a DTLS connection with Cisco SD-WAN Manager:

1. The Cisco vEdge router establishes a DTLS connection with Cisco SD-WAN Manager.

2. Cisco SD-WAN Manager server sends the configuration file to the Cisco vEdge router.

3. When the Cisco vEdge router receives the configuration file and activates its full configuration.

4. The Cisco vEdge router starts advertising prefixes to Cisco SD-WAN Controller.

If you are not using Cisco SD-WAN Manager, you can log in to the Cisco vEdge router and either manually load its configuration file or manually configure the router.

Below is a more detailed step-by-step description of how the automatic authentication occurs between a Cisco vEdge router and Cisco SD-WAN Manager.

First, the Cisco vEdge router initiates an encrypted DTLS connection to the IP address of Cisco SD-WAN Manager. The encryption is provided by RSA. Each device automatically generates an RSA private key‒public key pair when it boots. Cisco SD-WAN Manager receives the Cisco vEdge router's original interface address and uses the outer IP address in the received packet to determine whether the Cisco vEdge router is behind a NAT. If it is, Cisco SD-WAN Manager creates a mapping of the Cisco vEdge router's public IP address and port to its private IP address.

Over this encrypted DTLS channel, the Cisco vEdge router and Cisco SD-WAN Manager proceed to authenticate each other. As with other device authentication, the Cisco vEdge router and Cisco SD-WAN Manager authenticate each other in parallel. We start our discussion by describing how the Cisco vEdge router authenticates Cisco SD-WAN Manager:

1. Cisco SD-WAN Manager sends its trusted root CA signed certificate to the Cisco vEdge router.​

2. The Cisco vEdge router uses its chain of trust to extract the organization name from the certificate and compares it to the organization name that is configured on the router itself. If the two organization names match, the Cisco vEdge routers knows that the organization of Cisco SD-WAN Manager is proper. If they do not match, the Cisco vEdge router tears down the DTLS connection.

3. The Cisco vEdge router uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, the Cisco vEdge router knows that the certificate itself is valid. If the signature is incorrect, the Cisco vEdge router tears down the DTLS connection.

After performing these two checks, the Cisco vEdge router knows that Cisco SD-WAN Manager is valid, and its authentication of Cisco SD-WAN Manager is complete.

In the opposite direction, Cisco SD-WAN Manager authenticates the Cisco vEdge router:

1. Cisco SD-WAN Manager sends a challenge to the Cisco vEdge router. The challenge is a 256-bit random value.

2. The Cisco vEdge router sends a response to the challenge that includes the following:

   • Cisco vEdge serial number

   • Cisco vEdge chassis number

   • Cisco vEdge board ID certificate (for a hardware Cisco vEdge router) or the signed certification (for a Cisco vEdge Cloud router)

   • 256-bit random value signed by the Cisco vEdge router's private key

3. Cisco SD-WAN Manager compares the serial and chassis numbers to the list in its Cisco vEdge authorized device list file. The numbers must match one of the number pairs in the file. If there is no match, Cisco SD-WAN Managerthe Cisco SD-WAN Manager NMS tears down the DTLS connection.

4. Cisco SD-WAN Manager checks that the signing of the 256-bit random value is proper. It does this using the Cisco vEdge router's public key, which it extracts from the router's board ID certificate. If the signing is not correct, Cisco SD-WAN Manager tears down the DTLS connection.

5. Cisco SD-WAN Manager uses the root CA chain from the Cisco vEdge routers board ID certificate to validate that the board ID certificate is itself valid. If the certificate is not valid, Cisco SD-WAN Manager tears down the DTLS connection.

After performing these three checks, Cisco SD-WAN Manager knows that Cisco vEdge router is valid, and its authentication of the router is complete.

When the two-way authentication succeeds, Cisco SD-WAN Manager server sends the configuration file to the Cisco vEdge router. When the Cisco vEdge router receives the configuration file, it activates its full configuration and starts advertising prefixes to Cisco SD-WAN Controller.

The last step in the automatic authentication process is for Cisco SD-WAN Controller and the Cisco vEdge router to authenticate each other. In this step, Cisco SD-WAN Controller performs authentication to ensure that the Cisco vEdge router belongs in its network, and the Cisco vEdge router also authenticates Cisco SD-WAN Controller. When the authentication completes, the DTLS connection between the two devices becomes permanent, and Cisco SD-WAN Controller establishes an OMP peering session running over the DTLS connection. Then, the Cisco vEdge router starts sending data traffic over the Cisco Catalyst SD-WAN overlay network.

In this section below, is a more detailed step-by-step description of how the automatic authentication occurs between Cisco SD-WAN Controller and a Cisco vEdge router.

To initiate a session between Cisco SD-WAN Controller and a Cisco vEdge router, one of the two devices initiates an encrypted DTLS connection to the other. The encryption is provided by RSA. Each device automatically generates an RSA private key‒public key pair when it boots.

The authentication between Cisco SD-WAN Controller and a Cisco vEdge router is a two-way process that occurs in parallel. Let's start our discussion with how Cisco SD-WAN Controller authenticates a Cisco vEdge router:

1. Cisco SD-WAN Controller sends a challenge to the Cisco vEdge router. The challenge is a 256-bit random value.

2. The Cisco vEdge router sends a response to the challenge that includes the following:

   • Cisco vEdge serial number

   • Cisco vEdge chassis number

   • Cisco vEdge board ID certificate

   • 256-bit random value signed by the Cisco vEdge router's private key

3. Cisco SD-WAN Controller compares the serial and chassis numbers to the list in its Cisco vEdge authorized device list file. The numbers must match one of the number pairs in the file. If there is no match, Cisco SD-WAN Controller tears down the DTLS connection.

4. Cisco SD-WAN Controller checks that the signing of the 256-bit random value is proper. It does this using the Cisco vEdge router's public key, which it extracts from the router's board ID certificate. If the signing is not correct, Cisco SD-WAN Controller tears down the DTLS connection.

5. Cisco SD-WAN Controller uses the root CA chain from the Cisco vEdge routers board ID certificate to validate that the board ID certificate is itself valid. If the certificate is not valid, Cisco SD-WAN Controller tears down the DTLS connection.

6. Cisco SD-WAN Controller compares the response with the original challenge. If the response matches the challenge that Cisco SD-WAN Validator issued, authentication between the two devices occurs. Otherwise, Cisco SD-WAN Controller tears down the DTLS connection.

After performing these three checks, Cisco SD-WAN Controller knows that Cisco vEdge router is valid, and its authentication of the router is complete.

In the other direction, the Cisco vEdge router authenticates Cisco SD-WAN Controller:

1. Cisco SD-WAN Controller sends its trusted root CA signed certificate to the Cisco vEdge router.

2. The Cisco vEdge router uses its chain of trust to extract Cisco SD-WAN Controller's serial number from the certificate. The number must match one of the numbers in the Cisco SD-WAN Controller authorized serial number file. If there is no match, the Cisco vEdge router tears down the DTLS connection.

3. The Edge router uses its chain of trust to extract the organization name from the certificate and compares it to the organization name that is configured on the Cisco vEdge router. If the two organization names match, the Cisco vEdge router knows that the organization of Cisco SD-WAN Controller is proper. If they do not match, the Cisco vEdge router tears down the DTLS connection.

4. The Cisco vEdge router uses the root CA chain to verify that the certificate has indeed been signed by the root CA (either Symantec or the enterprise CA). If the signature is correct, the Cisco vEdge router knows that the certificate itself is valid. If the signature is incorrect, the Cisco vEdge router tears down the DTLS connection.

After performing these three checks, the Cisco vEdge authentication of Cisco SD-WAN Controller is complete. The DTLS connection that is used for authentication now becomes a permanent (nontransient) connection, and the two devices establish an OMP session over it that is used to exchange control plane traffic.

This authentication procedure repeats for each Cisco SD-WAN Controller and each Cisco vEdge router that you introduce into the overlay network.

Each Cisco vEdge router in the network must connect to at least one Cisco SD-WAN Controller. That is, a DTLS connection must be successfully established between each Cisco vEdge router and one Cisco SD-WAN Controller. The Cisco SD-WAN network has the notion of a domain. Within a domain, it is recommended that you have multiple Cisco SD-WAN Controllers for redundancy. Then each Cisco vEdge router can connect to more than one Cisco SD-WAN Controller.

Over the OMP session, a Cisco vEdge router relays various control plane–​related information to Cisco SD-WAN Controller so that Cisco SD-WAN Controller can learn the network topology:

• The Cisco vEdge router advertises the service-side prefixes and routes that it has learned from its local static and dynamic (BGP and OSPF) routing protocols.

• Each Cisco vEdge router has a transport address, called a TLOC, or transport location, which is the address of the interface that connects to the WAN transport network (such as the Internet) or to the NAT gateway that connects to the WAN transport. Once the DTLS connection comes up between the Cisco vEdge router and Cisco SD-WAN Controller, OMP registers the TLOCs with Cisco SD-WAN Controller.

• The Cisco vEdge router advertises the IP addresses of any services that are located on its service-side network, such as firewalls and intrusion detection devices.

Cisco SD-WAN Controller installs these OMP routes in its routing database and advertises them to the other Cisco vEdge routers in the Cisco Catalyst SD-WAN overlay network. Cisco SD-WAN Controller also updates the Cisco vEdge router with the OMP route information that it learns from other Cisco vEdge routers in the network. Cisco SD-WAN Controller can apply inbound policy on received routes and prefixes before installing them into its routing table, and it can apply outbound policy before advertising routes from its routing table.

This article describes which ports Cisco Catalyst SD-WAN devices use. If your network has firewall devices, you must open these ports on the firewalls so that devices in the Cisco Catalyst SD-WAN overlay network can exchange traffic.

Before You Begin

To run Cisco SD-WAN Manager, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This topic describes how to create a virtual machine on a server running the VMware vSphere ESXi Hypervisor. You can also create the virtual machine on a server running the Kernel-based Virtual Machine (KVM) hypervisor.

For server requirements, see Server Hardware Recommendations.

From Cisco Catalyst SD-WAN Control Components Release 20.14.1, you can enable disk encryption on the hypervisor.

Create Cisco Catalyst SD-WAN Manager VM Instance

1. Start the vSphere Client and create a Cisco SD-WAN Manager VM instance.

2. Create a new virtual disk that has a volume of at least 100 GB for the Cisco SD-WAN Manager database.

3. Add another vNICs.

4. Start the Cisco SD-WAN Manager VM instance and connect to the Cisco SD-WAN Manager console.

5. To create a Cisco SD-WAN Manager cluster, repeat Steps 1 through 4 to create a VM for each Cisco SD-WAN Manager instance.

If you are using the VMware vCenter Server to create the Cisco SD-WAN Manager VM instance, follow the same procedure.

To run Cisco SD-WAN Manager, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This topic describes the process for creating a VM on a server running VMware Kernel-based Virtual Machine (KVM) Hypervisor. You can also create the VM on a server running VMware vSphere ESXi Hypervisor.

For server requirements, see Server Hardware Requirements.

You can configure Cisco SD-WAN Manager using device templates. However, Cisco recommends that you use the CLI mode to configure Cisco SD-WAN Manager instead of using the device templates.

Once you have set up and started the virtual machines (VMs) for Cisco SD-WAN Manager, they come up with a factory-default configuration. You then configure each Cisco SD-WAN Manager instance directly from the Cisco SD-WAN Manager server itself using CLI mode or ESXi console so that each Cisco SD-WAN Manager can be authenticated and verified and can join the overlay network. At a minimum, you must configure the IP address of your network's Cisco SD-WAN Validator, the device's system IP address, and a tunnel interface in VPN 0 to use for exchanging control traffic among the network controller devices (the Cisco SD-WAN Validator, Cisco SD-WAN Manager, and Cisco SD-WAN Controller devices).

For the overlay network to be operational and for Cisco SD-WAN Manager instances to participate in the overlay network, you must do the following:

• Configure a tunnel interface on at least one interface in VPN 0. This interface must connect to a WAN transport network that is accessible by all Cisco vEdge devices. VPN 0 carries all control plane traffic among the Cisco vEdge devices in the overlay network.

• Ensure that the Overlay Management Protocol (OMP) is enabled. OMP is the protocol responsible for establishing and maintaining the Cisco Catalyst SD-WAN control plane. OMP is enabled by default, and you cannot disable it. If you edit the configuration from the CLI, do not remove the omp configuration command.

Note	For a Cisco SD-WAN Managercluster, you must configure each Cisco SD-WAN Manager instance in the cluster individually, from the Cisco SD-WAN Manager server itself using CLI mode or ESXi console.

Configure Cisco Catalyst SD-WAN Manager

To configure Cisco SD-WAN Manager, create a device configuration template:

1. Configure the address of Cisco SD-WAN Validator:

   1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

   2. Click Validator. (If you are using Cisco Catalyst SD-WAN Manager Release 20.12.1 or earlier, click Edit.)

   3. In the Validator DNS/IP Address: Port field, enter the DNS name that points to Cisco SD-WAN Validator or the IP address of Cisco SD-WAN Validator and the port number to use to connect to it.

   4. Click Save.

2. Configure Cisco SD-WAN Manager Using CLI

   Use CLI mode to configure Cisco SD-WAN Manager. You can access the CLI by using the ESXi console using a separate SSH client or by establishing an SSH session through the Cisco SD-WAN Manager Graphical User Interface (GUI).

   To establish an SSH session to a device:

   1. From the Cisco SD-WAN Manager menu, choose Tools > SSH Terminal.

   2. From the left pane, click to select a device.

   3. Log in as the user admin, using the default password, admin. The CLI prompt is displayed.

   4. Enter configuration mode:

      Device# config

      Device(config)#

   You can now issue CLI commands to configure Cisco SD-WAN Manager.

   The following features are mandatory for Cisco SD-WAN Manager operation. Configure these features from the CLI mode.

   • Authentication, Authorization, and Accounting (AAA)

   • Security

   • System-wide parameters

   • Transport VPN (VPN 0)

   • Management VPN (for out-of-band management traffic)

Sample CLI Configuration

This section provides sample CLI configurations to configure Cisco SD-WAN Manager using CLI.

Note that this configuration includes a number of settings from the factory-default configuration and shows a number of default configuration values.

vManage# show running-config 
system
 host-name         vManage
 gps-location latitude 40.7127837
 gps-location longitude -74.00594130000002
 system-ip         172.16.255.22
 site-id           200
 organization-name "Cisco"
 clock timezone America/Los_Angeles
 vbond 10.1.14.14
 aaa
  auth-order local radius tacacs
  usergroup basic
   task system read write
   task interface read write
  !
  usergroup netadmin
  !
  usergroup operator
   task system read
   task interface read
   task policy read
   task routing read
   task security read
  !
  user admin
   password encrypted-password
  !
 !
 logging
  disk
   enable
  !
 !
!
snmp
 no shutdown
 view v2
  oid 1.3.6.1
 !
 community private
  view          v2
  authorization read-only
 !
 trap target vpn 0 10.0.1.1 16662
  group-name     Cisco
  community-name private
 !
 trap group test
  all
   level critical major minor
  exit
 exit
!
vpn 0
 interface eth1
  ip address 10.0.12.22/24
  tunnel-interface
   color public-internet
   allow-service dhcp
   allow-service dns
   allow-service icmp
   no allow-service sshd
   allow-service netconf
   no allow-service ntp
   no allow-service stun
   allow-service https
  !
  no shutdown
 !
 ip route 0.0.0.0/0 10.0.12.13
!
vpn 512
 interface eth0
  ip 172.16.14.145/23
  no shutdown
 !
 ip route 0.0.0.0/0 172.16.14.1
!

New controller devices in the overlay network—Cisco SD-WAN Manager instances, Cisco SD-WAN Validators, and Cisco SD-WAN Controllers—are authenticated using signed certificates. From Cisco SD-WAN Manager, you can automatically generate the certificate signing requests (CSRs), retrieve the generated certificates, and install them on all controller devices when they are added to the network.

Note	All controller devices must have a certificate installed on them to be able to join the overlay network.

To automate the certificate generation and installation process, configure the name of your organization and certificate authorization settings before adding the controller devices to the network.

For more information on configuring certificate settings, see Certificates.

For Cisco SD-WAN Manager to be able to join the overlay network, you must generate a certificate signing request (CSR) for Cisco SD-WAN Manager instance. Cisco SD-WAN Manager automatically retrieves the generated certificate and installs it.

For more information on generating Cisco SD-WAN Manager certificate, see Certificates.

A Cisco SD-WAN Manager cluster is a collection of three or more Cisco SD-WAN Manager instances in a Cisco Catalyst SD-WAN overlay network domain. The cluster collectively provides network management services to all Cisco vEdge devices in the network. Some of the services, such as determining which Cisco SD-WAN Manager instance connects to and handles requests for a router, are distributed automatically, while for others (the statistics and configuration databases, and the messaging server), you configure which Cisco SD-WAN Manager instance handles the service.

For more information on creating Cisco SD-WAN Manager cluster, refer to Cluster Management.

By default, a user's session to a Cisco SD-WAN Manager client remains established indefinitely and never times out.

1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

2. Click User Sessions. In the Client Session Timeout option, enable the Session Timeout. (If you are using Cisco Catalyst SD-WAN Manager Release 20.12.1 or earlier, click Edit.)

3. Enter the timeout value, in minutes. This value can be from 10 to 180 minutes.

4. Click Save.

The client session timeout value applies to all Cisco SD-WAN Manager servers in a Cisco SD-WAN Manager cluster.

Before You Begin

To start Cisco SD-WAN Validator, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This article describes how to create a VM on a server running the VMware vSphere ESXi Hypervisor. You can also create the VM on a server running the Kernel-based Virtual Machine (KVM) Hypervisor software.

For server information, see Server Hardware Recommendations .

From Cisco Catalyst SD-WAN Control Components Release 20.14.1, you can enable disk encryption on the hypervisor.

Create Cisco Catalyst SD-WAN Validator VM Instance

1. Launch the vSphere client and create a Cisco SD-WAN Validator VM instance.

2. Add a vNIC for the tunnel interface.

3. Start the Cisco SD-WAN Validator VM instance and connect to the console.

The details of each step are provided below.

If you are using the VMware vCenter Server to create the Cisco SD-WAN Validator VM instance, follow the same procedure. Note, however, that the vCenter Server pages look different than the vSphere Client pages shown in the procedure.

To start Cisco SD-WAN Validator, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This article describes how to create a VM on a server running the Kernel-based Virtual Machine (KVM) Hypervisor. You can also create the VM on a server running the vSphere ESXi Hypervisor software.

For server information, see Server Hardware Recommendations.

To create a Cisco SD-WAN Validator VM instance on the KVM hypervisor:

1. Launch the Virtual Machine Manager (virt-manager) client application. The system displays the Virtual Machine Manager page.

2. Click New to deploy the virtual machine. The system opens the Create a new virtual machine page.

3. Enter the name of the virtual machine. The figure below specifies a name for the Cisco SD-WAN Validator instance.

   1. Choose Import existing disk image option to install the operating system.

   2. Click Forward.

4. For Provide the existing storage path, click Browse to find the Cisco SD-WAN Validator software image.

   1. For OS Type, choose Linux.

   2. For Version, choose the Linux version that you are running.

   3. Click Forward.

5. Specify Memory and CPU based on your network topology and the number of sites, and click Forward.

6. Check Customize configuration before install. Then click Finish.

7. Choose Disk 1 in the left navigation bar. Then:

   1. Click Advanced Options.

   2. For Disk Bus, choose IDE.

   3. For Storage Format, choose qcow2.

   4. Click Apply to create the VM instance with the parameters you had defined. By default, this includes one vNIC. This vNIC is used for the management interface.

   Note	The software supports only VMXNET3 vNICs.

8. In the vEdge Cloud Virtual Machine page, click Add Hardware to add a second vNIC for the tunnel interface.

9. In the Add New Virtual Hardware page, click Network.

   1. In the Host Device, choose an appropriate Host device.

   2. Click Finish.

   The newly created vNIC is listed in the left pane. This vNIC is used for the tunnel interface.

10. In the Cisco SD-WAN Validator Virtual Machine page, click Begin Installation in the top upper-left corner of the page.

11. The system creates the virtual machine instance and displays the Cisco SD-WAN Validator console.

12. In the login page, log in with the default username, which is admin, and the default password, which is admin.

What's Next

See Configure Cisco Catalyst SD-WAN Validator.

Once you have set up and started the virtual machine (VM) for Cisco SD-WAN Validator in your overlay network, Cisco SD-WAN Validator comes up with a factory-default configuration. You then need to manually configure few basic features and functions so that the devices can be authenticated and verified and can join the overlay network. Among these features, you configure the device as Cisco SD-WAN Validator providing the system IP address, and you configure a WAN interface that connects to the Internet. This interface must have a public IP address so that all Cisco vEdge devices in the overlay network can connect to Cisco SD-WAN Validator.

You create the initial configuration by using SSH to open a CLI session to Cisco SD-WAN Validator.

After you have created the initial configuration, you create the full configuration by creating configuration templates on Cisco SD-WAN Manager and then attach the templates to Cisco SD-WAN Validator. When you attach the configuration templates to Cisco SD-WAN Validator, the configuration parameters in the templates overwrite the initial configuration.

Create Initial Configuration for Cisco Catalyst SD-WAN Validator

To create the initial configuration on Cisco SD-WAN Validator using a CLI session:

1. Open a CLI session to Cisco vEdge device via SSH.

2. Log in as the user admin, using the default password, admin. The CLI prompt is displayed.

3. Enter configuration mode:

   For Cisco Catalyst SD-WAN Control Components Release 20.14.x and later releases:

   vSmart# config
   vSmart(config)# 

   For releases before Cisco Catalyst SD-WAN Control Components Release 20.14.x:

   vBond# config 
   vBond(config)# 

4. Configure the hostname:

   For Cisco Catalyst SD-WAN Control Components Release 20.14.x and later releases:

   vSmart(config)# system host-name vBond

   For releases before Cisco Catalyst SD-WAN Control Components Release 20.14.x:

   vBond(config)# system host-name hostname

   Configuring the hostname is optional, but is recommended because this name in included as part of the prompt in the CLI and it is used on various Cisco SD-WAN Manager screens to refer to the device.

5. Configure the system IP address:

   vBond(config-system)#system-ip ip-address

   Cisco SD-WAN Manager uses the system IP address to identify the device so that the NMS can download the full configuration to the device.
6. Configure the IP address of Cisco SD-WAN Validator. Cisco SD-WAN Validator's IP address must be a public IP address, to allow all Cisco vEdge devices in the overlay network to reach Cisco SD-WAN Validator:

   vBond(config-system)#vbond ip-address local

   In Releases 16.3 and later, the address can be an IPv4 or an IPv6 address. In earlier releases, it must be an IPv4 address. ​A Cisco SD-WAN Manager is effectively a vEdge router that performs only the orchestrator functions. The local option designates the device to be Cisco SD-WAN Validator, not a vEdge router. Cisco SD-WAN Validator must run on a standalone virtual machine (VM) or hardware router; it cannot coexist in the same device as a software or hardware vEdge router.
7. Configure a time limit for confirming that a software upgrade is successful:

   vBond(config-system)#upgrade-confirm minutes

   The time can be from 1 through 60 minutes. If you configure this time limit, when you upgrade the software on the device, Cisco SD-WAN Manager (when it comes up) or you must confirm that a software upgrade is successful within the configured number of minutes. If the device does not received the confirmation within the configured time, it reverts to the previous software image.
8. Change the password for the user "admin":

   vBond(config-system)#user admin password password

   The default password is "admin".
9. Configure an interface in VPN 0, to connect to the Internet or other WAN transport network. In Releases 16.3 and later, the IP address can be an IPv4 or an IPv6 address. In earlier releases, it must be an IPv4 address. Ensure that the prefix you configure for the interface contains the IP address that you configure in the vbond local command.

   vBond(config)#vpn 0 interface interface-name
   vBond(config-interface)#ip address ipv4-prefix/length
   vBond(config-interface)#ipv6 address ipv6-prefix/length
   vBond(config-interface)#no shutdown

   Note	The IP address must be a public address so that all devices in the overlay network can reach Cisco SD-WAN Validator.

10. Commit the configuration:

    vBond(config)#commit and-quit
    vBond#

11. Verify that the configuration is correct and complete:

    vBond#show running-config 

After the overlay network is up and operational, create a Cisco SD-WAN Validator configuration template on the Cisco SD-WAN Manager that contains the initial configuration parameters. Use the following Cisco SD-WAN Manager feature templates:

• System feature template to configure the hostname, system IP address, and Cisco SD-WAN Validator functionality.

• AAA feature template to configure a password for the "admin" user.

• VPN Interface Ethernet feature template to configure the interface in VPN 0.

In addition, it is recommended that you configure the following general system parameters:

• From the Cisco SD-WAN Manager menu, choose Administration > Settings and configure Organization name.

• From the Cisco SD-WAN Manager menu, choose Configuration > Templates. From System configuration template drop-down, select create template and configure Timezone, NTP servers, and device physical location.

• Click Additional Templates and from banner feature template drop-down, select Create Template. Configure Login banner.

• From System feature configuration template drop-down, select Create Template and configure disk and server parameters.

• From AAA feature configuration template drop-down, select Create Template and configure AAA, RADIUS and TACACS servers.

• Click Additional Templates and from SNMP feature template drop-down, select Create Template and configure SNMP.

Sample Initial CLI Configuration

Below is an example of a simple configuration on Cisco SD-WAN Validator. Note that this configuration includes a number of settings from the factory-default configuration and shows a number of default configuration values.

vBond#show running-config 
system
 host-name         vBond
 gps-location latitude 40.7127837
 gps-location longitude -74.00594130000002
 system-ip         172.16.240.161
 organization-name "Cisco"
 clock timezone America/Los_Angeles
 vbond 11.1.1.14 local
 aaa
  auth-order local radius tacacs
  usergroup basic
   task system read write
   task interface read write
  !
  usergroup netadmin
  !
  usergroup operator
   task system read
   task interface read
   task policy read
   task routing read
   task security read
  !
  user admin
   password encrypted-password
  !
 !
 logging
  disk
   enable
  !
!
vpn 0
 interface ge0/0
  ip address 11.1.1.14/24
  no shutdown
 !
 ip route 0.0.0.0/0 11.1.1.1
!
vpn 512
 interface eth0
  ip dhcp-client
  no shutdown
 !
!

What's Next

See Add Cisco SD-WAN Validator to the Overlay Network.

This article describes how to configure Cisco SD-WAN Validators that are being managed by Cisco SD-WAN Manager. These devices must be configured from Cisco SD-WAN Manager. If you configure them directly from the CLI on the router, Cisco SD-WAN Manager overwrites the configuration with the one stored on the NMS system.

After you create a minimal configuration for Cisco SD-WAN Validator, you must add it to overlay network by making Cisco SD-WAN Manager aware of Cisco SD-WAN Validator. When you add Cisco SD-WAN Validator, a signed certificate is generated and is used to validate and authenticate the orchestrator.

Add Cisco Catalyst SD-WAN Validator and Generate Certificate

To add Cisco SD-WAN Validator to the network, automatically generate the CSR, and install the signed certificate:​

1. From the Cisco SD-WAN Manager menu, choose Configuration > Devices.

2. Click Control Components and click Add Validator.

3. In the Add Validator window:

   1. Enter the VPN 0 IP address.

   2. Enter the username and password to access Cisco SD-WAN Validator.

   3. Choose the Generate CSR check box to allow the certificate-generation process to occur automatically.

   4. Click Add.

Cisco SD-WAN Manager generates the CSR, retrieves the generated certificate, and automatically installs it on Cisco SD-WAN Validator. The new controller device is listed in the Controller table with the controller type, hostname of the controller, IP address, site ID, and other details.

Verify Certificate Installation

To verify that the certificate is installed on Cisco SD-WAN Validator:

1. From the Cisco SD-WAN Manager menu, choose Configuration > Devices.

2. Choose the new device listed, and check in the Certificate Status column to ensure that the certificate has been installed.

The ZTP server must be configured before the ZTP workflow starts.

If you are hosting the Cisco Catalyst SD-WAN zero-touch-provisioning (ZTP) server in your enterprise, you must configure one Cisco SD-WAN Validator to perform this role. This Cisco SD-WAN Validator provides the Cisco vEdge devices in the overlay network with the IP address of your enterprise Cisco SD-WAN Validator and with the enterprise root CA chain. You can think of this Cisco SD-WAN Validator server as a top-level Cisco SD-WAN Validator, analogous to a top-level domain server in the Internet.

If you are using the Cisco Catalyst SD-WAN ZTP hosted service, there is no need to set up a top-level Cisco SD-WAN Validator.

This section provides step-by-step instructions on how to start the Cisco SD-WAN Validator and perform initial configuration.

Before You Begin

To start the Cisco SD-WAN Controller, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This article describes how to create a VM on a server running the VMware vSphere ESXi Hypervisor software. You can also create the VM on a server running the Kernel-based Virtual Machine (KVM) Hypervisor software.

For server requirements, see Server Hardware Recommendations.

From Cisco Catalyst SD-WAN Control Components Release 20.14.1, you can enable disk encryption on the hypervisor.

Create Cisco Catalyst SD-WAN Controller VM Instance

1. Launch the vSphere Client and create a Cisco SD-WAN Controller VM instance.

2. Add a vNIC for the management interface.

3. Start the Cisco SD-WAN Controller VM instance and connect to the console.

The details of each step are provided below.

If you are using the VMware vCenter Server to create the Cisco SD-WAN Controller VM instance, follow the same procedure. Note, however, that the vCenter Server pages look different than the vSphere Client pages shown in the procedure.

To start the Cisco SD-WAN Controller, you must create a virtual machine (VM) instance for it on a server that is running hypervisor software. This article describes how to create a VM on a server running the Kernel-based Virtual Machine (KVM) Hypervisor software. You can also create the VM on a server running the VMware vSphere ESXi Hypervisor software.

For server requirements, see Server Hardware Recommendations.

To create a Cisco SD-WAN Controller VM instance on the KVM hypervisor:

1. Launch the Virtual Machine Manager (virt-manager) client application. The system displays the Virtual Machine Manager page.

2. Click New to deploy the virtual machine. The system opens the Create a new virtual machine page.

3. Enter the name of the virtual machine. The figure below specifies a name for the Cisco SD-WAN Controller instance.

   1. Select Import existing disk image.

   2. Click Forward.

4. In Provide the existing storage path field, click Browse to find the Cisco SD-WAN Controller software image.

   1. For OS Type, select Linux.

   2. For Version, select the Linux version you are running.

   3. Click Forward.

5. Specify Memory and CPU based on your network topology and the number of sites, and click Forward.

6. Select Customize configuration before install. Then click Finish.

7. Select Disk 1 in the left navigation bar. Then:

   1. Click Advanced Options.

   2. In the Disk Bus field, select IDE.

   3. In the Storage Format field, select qcow2.

   4. Click Apply to create the VM instance with the parameters you just defined. By default, this includes one vNIC. This vNIC is used for the tunnel interface.

      Note	The software supports only VMXNET3 vNICs.

8. In the Cisco SD-WAN Controller Virtual Machine page, click Add Hardware to add a second vNIC for the management interface.

9. In the Add New Virtual Hardware page, click Network.

   1. In the Host Device field, select an appropriate host device.

   2. Click Finish.

      The newly created vNIC is listed in the left pane. This vNIC is used for the management interface.

10. In the Cisco SD-WAN Controller Virtual Machine page, click Begin Installation in the top upper-left corner of the screen.

11. The system creates the virtual machine instance and displays the Cisco SD-WAN Controller console.

12. At the login prompt, log in with the default username, which is admin, and the default password, which is admin.

What's Next

See Configure Cisco Catalyst SD-WAN Controller.

Once you have set up and started the virtual machines (VMs) for the Cisco SD-WAN Controllers in your overlay network, they come up with a factory-default configuration. You then need to manually configure a few basic features and functions so that the devices can be authenticated and verified and can join the overlay network. These features include the IP address of your network's Cisco SD-WAN Validator, the device's system IP address, and a tunnel interface in VPN 0 to use for exchanging control traffic among the network controller devices (the Cisco SD-WAN Validator, Cisco SD-WAN Manager, and Cisco SD-WAN Controller devices).

For the overlay network to be operational and for the Cisco SD-WAN Controllers to participate in the overlay network, do the following:

• Configure a tunnel interface on at least one interface in VPN 0. This interface must connect to a WAN transport network that is accessible by all Cisco vEdge devices. VPN 0 carries all control plane traffic among the Cisco vEdge devices in the overlay network.

• Ensure that the Overlay Management Protocol (OMP) is enabled. OMP is the protocol responsible for establishing and maintaining the Cisco Catalyst SD-WAN control plane. It is enabled by default, and you cannot disable it. When you edit the configuration from the CLI, do not remove the omp configuration command.

You create these initial configuration by using SSH to open a CLI session to the the Cisco SD-WAN Controller.

After you have created the initial configuration, you create the full configuration by creating configuration templates on the Cisco SD-WAN Manager NMS and then attaching them to the Cisco SD-WAN Controllers. When you attach the configuration template to the Cisco SD-WAN Controllers, the configuration parameters in the templates overwrite the initial configuration.

In this initial configuration, you should assign a system IP address to the Cisco SD-WAN Controller. This address, which is similar to the router ID on non-Cisco SD-WAN routers, is a persistent address that identifies the controller independently of any interface addresses. The system IP is a component of the device's TLOC address. Setting the system IP address for a device allows you to renumber interfaces as needed without affecting the reachability of the Cisco vEdge device. Control traffic over secure DTLS or TLS connections between Cisco SD-WAN Controllers and vEdge routers and between Cisco SD-WAN Controllers and Cisco SD-WAN Validator is sent over the system interface identified by the system IP address. In the transport VPN (VPN 0), the system IP address is used as the device's loopback address. You cannot use this same address for another interface in VPN 0.

Note	For the overlay network to function properly and predictably, the policies configured on all Cisco SD-WAN Controllers must be identical.

Create Initial Configuration for the Cisco Catalyst SD-WAN Controller

To create the initial configuration on a Cisco SD-WAN Controller from a CLI session:

1. Open a CLI session to the Cisco vEdge device via SSH.

2. Log in as the user admin, using the default password, admin. The CLI prompt is displayed.

3. Enter configuration mode:

   vSmart#  config
   vSmart(config)#

4. Configure the hostname:

   Cisco(config)# system host-name hostname
   

   Configuring the hostname is optional, but is recommended because this name in included as part of the prompt in the CLI and it is used on various Cisco SD-WAN Manager pages to refer to the device.

5. Configure the system IP address. In Releases 16.3 and later, the IP address can be an IPv4 or an IPv6 address. In earlier releases, it must be an IPv4 address. Releases 19.1 and later do not allow the configuration of IPv6 unique local addresses. In these releases, configure IPv6 addresses from the FC00::/7 prefix range.

   Note	Starting from Cisco Catalyst SD-WAN Control Components Release 20.9.x release, you can configure unique local IPv6 addresses. Prior to this release, you can configure IPv6 addresses from the FC00::/7 prefix range.

   vSmart(config-system)#system-ip ip-address
   

   The Cisco SD-WAN Manager uses the system IP address to identify the device so that the NMS can download the full configuration to the device.
6. Configure the numeric identifier of the site where the device is located:

   vSmart(config-system)# site-id site-id
   

7. Configure the numeric identifier of the domain in which the device is located:

   vSmart(config-system)# domain-id domain-id
   

8. Configure the IP address of the Cisco Catalyst SD-WAN Validator or a DNS name that points to the Cisco Catalyst SD-WAN Validator. The Cisco Catalyst SD-WAN Validator's IP address must be a public IP address, to allow all Cisco vEdge devices in the overlay network to reach it.

   vSmart(config-system)# vbond  (dns-name | ip-address)

9. Configure a time limit for confirming that a software upgrade is successful:

   vSmart(config-system)# upgrade-confirm minutes
   

   The time can be from 1 through 60 minutes. If you configure this time limit, when you upgrade the software on the device, Cisco SD-WAN Manager (when it comes up) or you must confirm that a software upgrade is successful within the configured number of minutes. If the device does not receive the confirmation within the configured time, it reverts to the previous software image.
10. Change the password for the user "admin":

    vSmart(config-system)# user admin password password
    

    The default password is "admin".

11. Configure an interface in VPN 0 to be used as a tunnel interface. VPN 0 is the WAN transport VPN, and the tunnel interface carries the control traffic among the devices in the overlay network. The interface name has the format eth number. You must enable the interface and configure its IP address, either as a static address or as a dynamically assigned address received from a DHCP server. In Releases 16.3 and later, the address can be an IPv4 or an IPv6 address, or you can configure both to enable dual-stack operation. In earlier releases, it must be an IPv4 address.

    ​vSmart(config)# vpn 0
    ​vSmart(config-vpn-0)# interface interface-name
    vSmart(config-interface)# ( ip dhcp-client | ip address prefix/length)
    vSmart(config-interface)# (ipv6 address ipv6-prefix/length​​​​​​​ | ipv6 dhcp-client [​​​​​​​dhcp-distance ​​​​​​​number | ​​​​​​​dhcp-rapid-commit​​​​​​​])
    vSmart(config-interface)# no shutdown
    vSmart(config-interface)# tunnel-interface
    ​​vSmart(config-tunnel-interface)# allow-service netconf
    

    Note

    You must configure a tunnel interface on at least one interface in VPN 0 in order for the overlay network to come up and for Cisco SD-WAN Controller to be able to participate in the overlay network. This interface must connect to a WAN transport network that is accessible by all Cisco vEdge devices. VPN 0 carries all control plane traffic among the Cisco vEdge devices in the overlay network.

12. Configure a color for the tunnel to identify the type of WAN transport. You can use the default color (default), but you can also configure a more appropriate color, such as mpls or metro-ethernet, depending on the actual WAN transport.

    vSmart(config-tunnel-interface)# color color
    

13. Configure a default route to the WAN transport network:

    vSmart(config-vpn-0)# ip route 0.0.0.0/0 next-hop
    

14. Commit the configuration:

    vSmart(config)# commit and-quit
    vSmart#

15. Verify that the configuration is correct and complete:

    vSmart# show running-config 

After the overlay network is up and operational, create a Cisco SD-WAN Controller configuration template on the Cisco SD-WAN Manager that contains the initial configuration parameters. Use the following Cisco SD-WAN Manager feature templates:

• System feature template to configure the hostname, system IP address, and Cisco SD-WAN Validator functionality.

• AAA feature template to configure a password for the "admin" user.

• VPN Interface Ethernet feature template to configure the interface, default route, and DNS server in VPN 0.

In addition, it is recommended that you configure the following general system parameters:

• From the Cisco SD-WAN Manager menu, select Administration > Settings and configure Organization name.

• From the Cisco SD-WAN Manager menu, select Configuration > Templates and configure the following:

• For NTP and System feature configuration template, configure Timezone, NTP servers, and device physical location.

• For Banner feature template, configure Login banner.

• For Logging feature configuration template, configure Logging parameters.

• For AAA feature configuration template, configure AAA, and RADIUS and TACACS+ servers.

• For SNMP feature configuration template, configure SNMP.

Sample Initial CLI Configuration

Below is an example of a simple configuration on a Cisco SD-WAN Controller. Note that this configuration includes a number of settings from the factory-default configuration and shows a number of default configuration values.

vSmart# show running-config 
system
 host-name         vSmart
 gps-location latitude 40.7127837
 gps-location longitude -74.00594130000002
 system-ip         172.16.240.172
 site-id           200
 organization-name "Cisco"
 clock timezone America/Los_Angeles
 upgrade-confirm   15
 vbond 184.122.2.2
 aaa
  auth-order local radius tacacs
  usergroup basic
   task system read write
   task interface read write
  !
  usergroup netadmin
  !
  usergroup operator
   task system read
   task interface read
   task policy read
   task routing read
   task security read
  !
  user admin
   password encrypted-password
  !
 !
 logging
  disk
   enable
  !
  server 192.168.48.11
   vpn      512
   priority warm
  exit
 !
!
omp
 no shutdown
 graceful-restart
!
snmp
 no shutdown
 view v2
  oid 1.3.6.1
 !
 community private
  view          v2
  authorization read-only
 !
 trap target vpn 0 10.0.1.1 16662
  group-name     Cisco
  community-name private
 !
 trap group test
  all
   level critical major minor
  exit
 exit
!
vpn 0
 interface eth1
  ip address 10.0.12.22/24
  tunnel-interface
   color public-internet
   allow-service dhcp
   allow-service dns
   allow-service icmp
   no allow-service sshd
   allow-service netconf
   no allow-service ntp
   no allow-service stun
 !
  no shutdown
 !
vpn 512
 interface eth0
  ip dhcp-client
  no shutdown
 !
!

What's Next

See Add the Cisco SD-WAN Controller to the Overlay Network.

For Cisco SD-WAN Controllers that are being managed by a Cisco SD-WAN Manager, you must configure them from Cisco SD-WAN Manager. If you configure them directly from the CLI on Cisco Catalyst SD-WAN Controller, Cisco SD-WAN Manager overwrites the configuration with the one stored on Cisco SD-WAN Manager.

After you create a minimal configuration for Cisco SD-WAN Controller, you must add it to an overlay network by making Cisco SD-WAN Manager aware of the controller. When you add Cisco SD-WAN Controller, a signed certificate is generated and is used to validate and authenticate the controller.

Cisco SD-WAN Manager can support up to 20 Cisco SD-WAN Controllers in the network.

Add a Cisco Catalyst SD-WAN Controller and Generate Certificate

To add a Cisco SD-WAN Controller to the network, automatically generate the CSR, and install the signed certificate:​

1. From the Cisco SD-WAN Manager menu, choose Configuration > Devices.

2. Click Controllers, and from the Add Controller drop-down menu.

   Note	Starting from Cisco IOS XE Catalyst SD-WAN Release 17.13.1a, the Controllers tab is renamed as the Control Components tab to stay consistent with Cisco Catalyst SD-WAN rebranding.

3. In the Add Controller window:

   1. Enter the system IP address of Cisco SD-WAN Controller.

   2. Enter the username and password to access Cisco SD-WAN Controller.

   3. Choose the protocol to use for control-plane connections. The default is DTLS.

   4. If you select TLS, enter the port number to use for TLS connections. The default is 23456.

   5. Check the Generate CSR check box to allow the certificate-generation process to occur automatically.

   6. Click Add.

Cisco SD-WAN Manager automatically generates the CSR, retrieves the generated certificate, and installs it on Cisco SD-WAN Controller. The new controller is listed in the Controller table with the controller type, hostname of the controller, IP address, site ID, and other details.

Verify Certificate Installation

To verify that the certificate is installed on a Cisco SD-WAN Controller:

1. From the Cisco SD-WAN Manager menu, choose Configuration > Devices.

2. Choose the new controller listed and check in the Certificate Status column to ensure that the certificate has been installed.

Note

If Cisco SD-WAN Controller and Cisco SD-WAN Validator have the same system IP addresses, they do not appear in Cisco SD-WAN Manager as devices or controllers. The certificate status of Cisco SD-WAN Controller and Cisco SD-WAN Validator is also not displayed. However, the control connections still successfully comes up.

What's Next

See Deploy the vEdge Routers.

To start a software vEdge Cloud router, you must create a virtual machine (VM) instance for it. This article describes how to create a VM instance on Amazon AWS. You can also create the VM on a server running the vSphere ESXi Hypervisor software or the Kernel-based Virtual Machine (KVM) Hypervisor software.

To start the vEdge Cloud router virtual machine (VM) instance on Amazon AWS, first create a Virtual Private Cloud (VPC). The VPC is a self-contained environment in which you build the infrastructure you need in order to build your network.

Plan your network addressing carefully before creating the VPC. The VPC can use addresses only in the range you specify, and once you create a VPC, you cannot modify it. If your network addressing requirements change, you must delete the VPC and create a new one.

Starting Cisco SD-WAN 18.4 Release, Cisco Cloud Services 1000v (CSR 1000v) Router Cisco Catalyst SD-WAN version is supported on AWS.

To start a vEdge Cloud router on Amazon AWS:

1. Create a VPC.

2. Set up the vEdge Cloud router VM instance.

3. Define additional interfaces.

Create a VPC

Plan your network address blocks carefully before creating the VPC. Once you create a VPC, you cannot modify it. To make any changes to the network addressing, you must delete the VPC and create a new one.

1. Log in to AWS. In the Networking section of the AWS home page, click VPC.

2. On the page that opens, click Start VPC.

3. On the Select a VPC Configuration page, select VPC with Public and Private Subnets.

4. On the VPC with Public and Private Subnets screen:

   1. In IP CIDR Block, enter the desired IP addressing block. The VPC can use addresses only in this ange.

   2. Specify a public subnet and a private subnet from within the IP CIDR block.

   3. In Elastic IP Allocation ID, enter the address of your Internet gateway. This gateway translates internal traffic for delivery to the public Internet.

   4. Add endpoints for S3 only if you need extended storage space, such as for a large database.

   5. To use the AWS automatic registration of IP addresses to DNS, enable DNS hostnames.

   6. Select the desired Hardware tenancy, either shared or dedicated. You can share your AWS hardware with other AWS clients, or you can have dedicated hardware. With dedicated hardware, the device assigned to you can host only your data. However, the cost is higher.

   7. Click Create VPC.

Wait a few minutes until the VPC Dashboard displays the VPC Successfully Created message.

The infrastructure is now complete and ready for you to deploy applications, appliances, and the vEdge Cloud router. Click the links on the left to see the subnets, route tables, internet gateways, and NAT address translation points in the VPC.

Set Up the vEdge Cloud router VM Instance

1. Click Services > EC2 to open the EC2 Dashboard, and then click Launch Instance.

1. Choose an Amazon Machine Image (AMI). The Cisco Catalyst SD-WAN AMI has a name in the format release-number-vEdge; for example, 16.1.0-vEdge. The Cisco Catalyst SD-WAN AMI is private. Contact your Cisco Catalyst SD-WAN sales representative, who can share it with you.

2. Choose the Cisco Catalyst SD-WAN AMI, then click Select.

3. The Choose an Instance Type screen appears. Determine which instance type best meets your needs, according to the following table. The minimum requirement is 2 vCPUs.

   Table 6. Table 1: EC2 Instance Types that Support the vEdge Cloud router

   	vCPU	Memory (GB)	Instance Storage (GB)
   General Purpose — Current Generation
   m4.large	2	8	EBS only
   m4.xlarge	4	16	EBS only
   m4.2xlarge	8	32	EBS only
   m4.4xlarge	16	64	EBS only
   m4.10xlarge	40	160	EBS only
   Compute Optimized — Current Generation
   c4.large	2	3.75	EBS only
   c4.xlarge	4	7.5	EBS only
   c4.2xlarge	8	15	EBS only
   c4.4xlarge	16	30	EBS only
   c4.8xlarge	36	60	EBS only
   c3.large	2	3.75	2 x 16 SSD
   c3.xlarge	4	7.5	2 x 40 SSD
   c3.2xlarge	8	15	2 x 80 SSD
   c3.4xlarge	16	30	2 x 160 SSD
   c3.8xlarge	32	60	2 x 320 SSD

4. Select the preferred instance type, then click Next: Configure Instance Details.

Configure Instance Details

On the Configure Instance Details screen:

1. In Network, select the VPC you just created.

2. In Subnet, select the subnet for your first interface.

3. In Network Interfaces, click Add Device and select a subnet for each additional interface.

4. Note

   Starting from Cisco SD-WAN Release 20.5.1, a Cisco vEdge Cloud router VM with the default username and password (admin/admin) cannot be deployed on AWS. Therefore, when you deploy a Cisco vEdge Cloud router VM using a third-party cloud provider, ensure that you use the following cloud configuration to continue using the default credentials.

   In the User Data field, enter the following cloud configuration:

   #cloud-config
    
   hostname: vedge
   write_files:
   - content: "vedge\n"
     owner: root:root
     path: /etc/default/personality
     permissions: '0644'
   - content: "1\n"
     owner: root:root
     path: /etc/default/inited
     permissions: '0600'
   - path: /etc/confd/init/zcloud.xml
     content: |
       <config xmlns="http://tail-f.com/ns/config/1.0">
         <system xmlns="http://viptela.com/system">
           <aaa>
             <user>
               <name>admin</name>
               <password>$6$9ac6af765f1cd0c0$jRM/rCPsQ56JlDU/1s9H7zhhksy/FZHv37zDJkzM6h/IU/FsnTcBuLwV3AVI5kCnfX9wYmqP8CsGk.4PrjC22/</password>
               <group>netadmin</group>
             </user>
           </aaa>
         </system>
       </config>

   This cloud configuration configures the VM with admin/admin credentials, and forces a password change on your first login.

5. Click Next: Add Storage.

6. The Add Storage page opens. You do not need to change any settings on this screen. Click Next: Tag Instance.

    

7. The Tag Instance page opens. Enter your desired Key and Value, and then click Next: Configure Security Group.

8. The Configure Security Group page opens. Add rules to configure your firewall settings. These rules apply to outside traffic coming into your vEdge Cloud router.

   1. Below Type, select SSH.

   2. Below Source, select My IP.

9. Click Add Rule, then fill out the fields as follows:

   1. Below Type, select Custom UDP Rule.

   2. Below Port Range, enter 12346.

   3. Below Source, select Anywhere. 12346 is the default port for IPSec.

   4. If port hopping is enabled, you may need to add more rules.

10. Click Review and Launch. The Review Instance Launch screen opens. Click Launch.

11. Select Proceed without a key pair, click the acknowledgement check box, then click Launch Instances.

12. Wait a few minutes, the instance initializes. The vEdge Cloud router is now running. The first interface, eth0, is always the management interface. The second interface, ge0/0, appears in VPN 0, but you can configure it to be in a different VPN.

Define Additional Interfaces

The vEdge Cloud router supports a total of nine interfaces. The first is always the management interface, and the remaining eight are transport and service interfaces. To configure additional interfaces:

1. In the left pane, click Network Interfaces.

2. Click Create Network Interface. Select the Subnet and Security group, and then click Yes, Create. Note that two interfaces in the same routing domain cannot be in the same subnet.

3. Select the check box to the left of the new interface, and click Attach.

4. Select the vEdge Cloud router, and click Attach.

5. Reboot the vEdge Cloud router, because the vEdge Cloud router detects interfaces only during the boot process.

   The new interface is now up. The interface in VPN 0 connects to a WAN transport, such as the internet. The interface in VPN 1 faces a service-side network and can be used for appliances and applications. The interface in VPN 512 is dedicated to out-of-band management.

6. To allow the interface to carry jumbo frames (packets with an MTU of 2000 bytes), configure the MTU from the CLI. For example:

   Router# show interface
                                           IF      IF                                                                TCP                                   
                   AF                      ADMIN   OPER    ENCAP                                      SPEED          MSS                 RX       TX       
   VPN  INTERFACE  TYPE  IP ADDRESS        STATUS  STATUS  TYPE   PORT TYPE  MTU   HWADDR             MBPS   DUPLEX  ADJUST  UPTIME      PACKETS  PACKETS  
   --------------------------------------------------------------------------------------------------------------------------------------------------------
   0    ge0/0      ipv4  10.66.15.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:62  1000   full    1420    0:14:05:07  545682   545226   
   0    ge0/1      ipv4  10.1.17.15/24     Up      Up      null   service    1500  00:0c:29:db:f0:6c  1000   full    1420    0:14:21:19  0        10       
   0    ge0/2      ipv4  -                 Down    Up      null   service    1500  00:0c:29:db:f0:76  1000   full    1420    0:14:21:47  0        0        
   0    ge0/3      ipv4  10.0.20.15/24     Up      Up      null   service    1500  00:0c:29:db:f0:80  1000   full    1420    0:14:21:19  0        10       
   0    ge0/6      ipv4  172.17.1.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:9e  1000   full    1420    0:14:21:19  0        10       
   0    ge0/7      ipv4  10.0.100.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:a8  1000   full    1420    0:14:21:19  770      705      
   0    system     ipv4  172.16.255.15/32  Up      Up      null   loopback   1500  00:00:00:00:00:00  0      full    1420    0:14:21:30  0        0        
   0    loopback3  ipv4  10.1.15.15/24     Up      Up      null   transport  2000  00:00:00:00:00:00  10     full    1920    0:14:21:22  0        0        
   1    ge0/4      ipv4  10.20.24.15/24    Up      Up      null   service    2000  00:0c:29:db:f0:8a  1000   full    1920    0:14:21:15  52014    52055    
   1    ge0/5      ipv4  172.16.1.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:94  1000   full    1420    0:14:21:15  0        8        
   512  eth0       ipv4  10.0.1.15/24      Up      Up      null   service    1500  00:50:56:00:01:05  0      full    0       0:14:21:16  28826    29599    
   
   Router# config 
   Entering configuration mode terminal
   Router(config)# vpn 0 interface ge0/3 mtu 2000
   Router(config-interface-ge0/3)# commit 
   Commit complete.
   vEdge(config-interface-ge0/3)# end
   vEdge# show interface
                                           IF      IF                                                                TCP                                   
                   AF                      ADMIN   OPER    ENCAP                                      SPEED          MSS                 RX       TX       
   VPN  INTERFACE  TYPE  IP ADDRESS        STATUS  STATUS  TYPE   PORT TYPE  MTU   HWADDR             MBPS   DUPLEX  ADJUST  UPTIME      PACKETS  PACKETS  
   --------------------------------------------------------------------------------------------------------------------------------------------------------
   0    ge0/0      ipv4  10.66.15.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:62  1000   full    1420    0:14:05:30  546018   545562   
   0    ge0/1      ipv4  10.1.17.15/24     Up      Up      null   service    1500  00:0c:29:db:f0:6c  1000   full    1420    0:14:21:42  0        10       
   0    ge0/2      ipv4  -                 Down    Up      null   service    1500  00:0c:29:db:f0:76  1000   full    1420    0:14:22:10  0        0        
   0    ge0/3      ipv4  10.0.20.15/24     Up      Up      null   service    2000  00:0c:29:db:f0:80  1000   full    1920    0:14:21:42  0        10       
   0    ge0/6      ipv4  172.17.1.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:9e  1000   full    1420    0:14:21:42  0        10       
   0    ge0/7      ipv4  10.0.100.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:a8  1000   full    1420    0:14:21:42  773      708      
   0    system     ipv4  172.16.255.15/32  Up      Up      null   loopback   1500  00:00:00:00:00:00  0      full    1420    0:14:21:54  0        0        
   0    loopback3  ipv4  10.1.15.15/24     Up      Up      null   transport  2000  00:00:00:00:00:00  10     full    1920    0:14:21:46  0        0        
   1    ge0/4      ipv4  10.20.24.15/24    Up      Up      null   service    2000  00:0c:29:db:f0:8a  1000   full    1920    0:14:21:38  52038    52079    
   1    ge0/5      ipv4  172.16.1.15/24    Up      Up      null   service    1500  00:0c:29:db:f0:94  1000   full    1420    0:14:21:38  0        8        
   512  eth0       ipv4  10.0.1.15/24      Up      Up      null   service    1500  00:50:56:00:01:05  0      full    0       0:14:21:39  28926    29663  

The following instances support jumbo frames:

• Accelerated computing—CG1, G2, P2

• Compute optimized—C3, C4, CC2

• General purpose—M3, M4, T2

• Memory optimized—CR1, R3, R4, X1

• Storage optimized—D2, HI1, HS1, I2

What's Next

See Install Signed Certificates on vEdge Cloud Routers.

To start a software vEdge Cloud router, you must create a virtual machine (VM) instance for it. This article describes how to create a VM instance on Microsoft Azure. You can also create the VM on Amazon AWS or on a server running the vSphere ESXi Hypervisor software or the Kernel-based Virtual Machine (KVM) Hypervisor software.

Note: Cisco Catalyst SD-WAN offers only a Bring Your Own License (BYOL) for the vEdge Cloud router, so you are not actually purchasing the Cisco Catalyst SD-WAN product. You are charged hourly for the VNET instance.

For server requirements, see Server Hardware Recommendations.

Launch Azure Marketplace and Create a vEdge Cloud router VM Instance

1. Launch the Azure Marketplace application:

   1. In the left pane, click New to create a new vEdge Cloud router VM instance.

   2. In the Search box, search for Cisco.

2. In the right pane, select Cisco vEdge Cloud router (3 NICs) (Staged).

3. In the Cisco vEdge Cloud router (3 NICs) (Staged) screen, click Basics in the left pane to configure basic settings for the vEdge Cloud router VM:

   1. In the VM Name field, enter a name for the vEdge Cloud router VM instance.

   2. In the Username field, enter the name of a user who can access the VM instance.

   3. In the Authentication type field, select either Password or SSH public key.

   4. If you selected password, enter, and then confirm, your password. You use the username and password to open SSH session to the VM instance.

   5. If you selected SSH public key, see https://docs.microsoft.com/en-us/azu...reate-ssh-keys for instructions about how to generate an SSH key pair for Linux VMs.

   6. In the Subscription field, select Pay-As-You-Go from the drop-down menu.

   7. In the Resource Group field, click Create new to create a new resource group, or click Use existing to select an existing resource group from the drop-down menu.

   8. In the Location field, select the location in which you wish to bring up the vEdge Cloud router VM instance.

   9. Click OK.

4. In the left pane, click vEdge Settings to configure the vEdge Cloud router infrastructure settings.

5. In the Infrastructure Settings pane:

   1. Click Size. In the Choose a size pane, select D3_V2 Standard for the instance type and click Select. This is the recommended instance type.

   2. Click Storage Account. In the Choose storage account pane, click Create New to create a new storage account or select one of the listed storage accounts. Then click OK.

   3. Click Public IP Address. In the Choose public IP address pane, click Create New to create a new public IP address, or select one of the listed public IP address to use for the public IP subnet. Then click OK.

   4. In the Domain Name field, select vedge from the drop-down menu.

   5. Click Virtual Network. In the Choose virtual network pane, click Create New to create a new virtual network (VNET), or select an existing VNET to launch the vEdge Cloud instance in. Then click OK.

   6. If you selected an existing VNET, use the drop-down menu to choose available subnets within the VNET. Then click OK.

      You must have three subnets available within the VNET; otherwise, the vEdge Cloud router VM instance will fail to launch. Also, ensure that route tables associated with your VM subnets have a user-defined route (UDR) towards the service subnet of the vEdge Cloud router. The UDR ensures that the VM subnets use the vEdge Cloud router as the gateway. See the example topology below.

      

   7. If you created a new VNET, define the address space within that VNET. Then click OK in the Subnets pane.

      Cisco Catalyst SD-WAN prepopulates subnet names and assigns IP addresses per subnet from the VNET address space you defined. If you plan to connect your VNET instances through the service subnet associated to the vEdge Cloud router, you do not need to make updates to the route table.

6. In the Summary pane, click OK. The Summary pane validates and displays the configuration you defined for the vEdge Cloud router VM instance.

7. Click Buy to purchase. Then click Purchase in the Purchase pane.

   Note	Cisco Catalyst SD-WAN offers only a Bring Your Own License (BYOL) for the vEdge Cloud router, so you are not actually purchasing the Viptela product. You are charged hourly for the VNET instance.

   The system creates the vEdge Cloud router VM instance and notifies you that the deployment has succeeded.

8. Click the vEdge VM instance you just created.

   The system displays the public IP address and DNS name of the vEdge Cloud router VM instance.

9. SSH into the public IP address of the vEdge Cloud router VM instance.

10. At the login prompt, log in with the username and password you created in Step 3. To view the vEdge Cloud router default configuration, enter the following command:

    vEdge# show running-config

    When you create a vEdge Cloud router VM, the security group configuration shown below is applied to the NIC associated with the public subnet. This security group does not restrict traffic from specific sources, but it does restrict specific services. Custom services for TCP and UDP that need to be enabled for Cisco Catalyst SD-WAN control protocols are also automatically configured. You can change the security group configuration to suit your requirements.

vEdge Cloud Router Interface and Subnet Mapping

To create a vEdge Cloud router VM instance on Azure Marketplace, a minimum of three NICs are required—one each for management, service, and transport. The table below shows the mapping of the vEdge Cloud router interface with the subnet associated to these NICs.

What's Next

See Install Signed Certificates on vEdge Cloud Routers.

To start a software vEdge Cloud router, you must create a virtual machine (VM) instance for it. This article describes how to create a VM instance on a server running the vSphere ESXi Hypervisor software. You can also create the VM on Amazon AWS or on a server running the Kernel-based Virtual Machine (KVM) Hypervisor software.

For server requirements, see Server Hardware Recommendations.

To create a vEdge Cloud VM instance on the ESXi hypervisor:

1. Launch the vSphere Client and create a vEdge Cloud VM instance.

2. Add a vNIC for the tunnel interface.

3. Start the vEdge Cloud VM instance and connect to the console.

The details of each step are provided below.

If you are using the VMware vCenter Server to create the vEdge Cloud VM instance, follow the same procedure. Note, however, that the vCenter Server screens look different than the vSphere Client screens shown in the procedure.

Launch vSphere Client and Create a vEdge Cloud VM Instance

1. Launch the VMware vSphere Client application, and enter the IP address or name of the EXSi server, your username, and your password. Click Login to log in to the ESXi server.

   The system displays the ESXi screen.

2. Click File > Deploy OVF Template to deploy the virtual machine.

3. In the Deploy OVF Template screen, enter the location to install and download the OVF package. This package is the vedge.ova file that you downloaded from Cisco. Then click Next.

4. Click Next to verify OVF template details.

5. Enter a name for the deployed template and click Next. The figure below specifies a name for the vEdge instance.

6. Click Next to accept the default format for the virtual disks.

7. Click Next to accept your destination network name as the destination network for the deployed OVF template. In the figure below, CorpNet is the destination network.

8. In the Ready to Complete screen, click Finish.

   ​The system has successfully created the VM instance with the parameters you just defined and displays the vSphere Client screen with the Getting Started tab selected. By default, this includes four vNICs which can be used for the management, tunnel, or service interface.

Add a New vNIC

1. In the left navigation bar of the vSphere Client, select the vEdge Cloud VM instance you just created, and click Edit virtual machine settings.

2. In the vEdge Cloud – Virtual Machine Properties screen, click Add to add a new vNIC. Then click OK.

3. Click Ethernet Adapter for the type of device you wish to add. Then click Next.

4. In the Adapter Type drop-down, select VMXNET3 for the vNIC to add. Then click Next.

5. In the Ready to Complete screen, click Finish.

6. The vEdge Cloud – Virtual Machine Properties screen opens showing that the new vNIC is being added. Click OK to return to the vSphere Client screen.

Modify the MTU for a vSwitch

To allow the interface to carry jumbo frames (packets with an MTU of 2000 bytes), configure the MTU for each virtual switch (vSwitch):

1. Launch the ESXi Hypervisor and select the Configuration tab.

2. In the Hardware field, click Networking. The network adapters you added are displayed in the right pane.

   1. Click Properties for the vSwitch whose MTU you wish to modify.

3. In the vSwitch Properties screen, click Edit.

4. In the Advanced Properties MTU drop-down, change the vSwitch MTU to the desired value. The range is 2000 to 9000. Then click OK.

Start the vEdge Cloud VM Instance and Connect to the Console

1. In the left navigation bar of the vSphere Client, select the vEdge Cloud VM instance you just created, and click Power on the virtual machine. The vEdge Cloud virtual machine is powered on.

2. Select the Console tab to connect to the vEdge Cloud console.​

3. At the login prompt, log in with the default username, which is admin, and the default password, which is admin. To view the vEdge Cloud router default configuration, enter the following command:

    vEdge# show running-config 

Mapping vNICs to Interfaces

When you create a vEdge Cloud router VM instance on ESXi in the procedure in the previous section, you create two vNICs: vNIC 1, which is used for the management interface, and vNIC 2, which is used as a tunnel interface. From the perspective of the VM itself, these two vNICs map to the eth0 and eth1 interfaces, respectively. From the perspective of the Cisco Catalyst SD-WAN software for the vEdge Cloud router, these two vNICs map to the mgmt0 interface in VPN 512 and the ge0/0 interface in VPN 0, respectively. You cannot change these mappings.

You can configure up to five additional vNICs, numbered 3 through 7, on the VM host. You can map these vNICs to interfaces eth2 through eth7 as desired, and to Cisco Catalyst SD-WAN interfaces ge0/1 through ge0/7, as desired.

The table below summarizes the mapping between vNICs, VM host interfaces, and vEdge Cloud interfaces.

Note

The traffic destined to VRRP IP is not forwarded by ESXi, since VRRP MAC address is not learned by the Virtual Software Switch of ESXi associated with the vEdge Ethernet interface. This is due to the limitation of the VMWare ESXi, which does not allow multiple unicast MAC address configuration on vNIC. As a workaround, place the vNIC in promiscuous mode and perform MAC filtering in the software. To let Cisco vEdge software place interface in promiscuous mode, Virtual Software Switch port-group or switch configuration must be changed to allow the same. Be aware that ESXi VSS forwards all packets to all virtual machines that are connected to the port-group or switch. This can have an adverse performance impact on the ESXi Host other virtual machines. This might also have an adverse effect on the vEdge packet processing performance. Design your network carefully to avoid performance impact.

What's Next

See Install Signed Certificates on vEdge Cloud Routers.

To start a software vEdge Cloud router, you must create a virtual machine (VM) instance for it. This article describes how to create a VM instance on a server running the Kernel-based Virtual Machine (KVM) Hypervisor software. You can also create the VM on Amazon AWS or on a server running the vSphere ESXi Hypervisor software.

For server requirements, see Server Hardware Recommendations.

Create vEdge Cloud VM Instance on the KVM Hypervisor

To create a vEdge Cloud VM instance on the KVM hypervisor:

1. Launch the Virtual Machine Manager (virt-manager) client application. The system displays the Virtual Machine Manager screen.

2. Click New to deploy the virtual machine. The system opens the Create a new virtual machine screen.

3. Enter the name of the virtual machine. The figure below specifies a name for the vEdge Cloud instance.

   1. Select Import existing disk image.

   2. Click Forward.

4. In Provide the existing storage path field, click Browse to find the vEdge Cloud software image.

   1. In the OS Type field, select Linux.

   2. In the Version field, select the Linux version you are running.

   3. Click Forward.

5. Specify Memory and CPU based on your network topology and the number of sites. Click Forward.

6. Select Customize configuration before install. Then click Finish.

7. Select Disk 1 in the left navigation bar. Then:

   1. Click Advanced Options.

   2. In the Disk Bus field, select IDE.

   3. In the Storage Format field, select qcow2.

   4. Click Apply to create the VM instance with the parameters you just defined. By default, this includes one vNIC. This vNIC is used for the management interface.

   Note	Cisco Catalyst SD-WAN software supports VMXNET3 and Virtio vNICs. It is recommended, however, that you use the Virtio vNICs.

8. In the vEdge Cloud Virtual Machine screen, click Add Hardware to add a second vNIC for the tunnel interface.

9. In the Add New Virtual Hardware screen, click Network.

   1. In the Host Device field, select an appropriate host device.

   2. Click Finish.

   The newly created vNIC is listed in the left pane. This vNIC is used for the tunnel interface.

10. Create an ISO file to include a cloud-init configuration for the vEdge Cloud router.

    Note

    Starting from Cisco SD-WAN Release 20.7.1, the cloud-init configuration file should only contain the minimum configuration required for setting up control connections to Cisco SD-WAN Manager. Other configuration such as the VPN0 and clear-text passwords should be pushed through the Add-On CLI template on Cisco SD-WAN Manager.

11. In the Virtual Machine Manager screen, click Add Hardware to attach the ISO file you created.

12. In the Add New Virtual Hardware screen:

    1. Click Select managed or other existing storage.

    2. Click Browse and select the ISO file you created.

    3. In the Device type field, select IDE CDROM.

    4. Click Finish.

13. To allow the interface to carry jumbo frames (packets with an MTU of 2000 bytes), configure the MTU for each virtual network (vnet) and virtual bridge NIC-containing VNET (virbr-nic) interface to a value in the range of 2000 to 9000:

    1. From the VM shell, issue the following command to determine the MTU on the vnet and virbr-nic interfaces:

       user@vm:~$ ifconfig -a
       virbr1-nic Link encap:Ethernet HWaddr 52:54:00:14:4e:6f
                 BROADCAST MULTICAST  MTU:1500  Metric
                 RX packets:0 errors:0 dropped:0 ovreruns:0 frame:0
                 TX packets:0 errors:0 dropped:0 ovreruns:0 carrier:0
                 collisions:0 txqueuelen:500
                 RX bytes:0 (0.0 B)  TX bytes:0 (0.0B)
       ...
       vnet0     Link encap:Ethernet  HWaddr fe:50:56:00:10:1e
                 inet6 addr: fe80::fc50:56ff:fe00:11e/64 Scope:Link
                 UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
                 RX packets:167850 errors:0 dropped:0 overruns:0 frame:0
                 TX packets:663186 errors:0 dropped:0 overruns:0 carrier:0
                 collisions:0 txqueuelen:500
                 RX bytes:19257426 (19.2 MB)  TX bytes:42008544 (42.0 MB)
       ...

    2. Change the MTU of each vnet:

       user@vm:~$ sudo ifconfig vnet number mtu 2000

    3. Change the MTU of each virbr-nic:

       user@vm:~$ sudo ifconfig virbr-nic number mtu 2000

    4. Verify the MTU value:

       user@vm:~$ ifconfig -a

14. In the vEdge Cloud Virtual Machine page, click Begin Installation in the top upper-left corner of the screen.

15. The system creates the virtual machine instance and displays the vEdge Cloud console.

16. At the login prompt, log in with the default username, which is admin, and the default password, which is admin. To view the vEdge Cloud router default configuration, enter the following command:

    vEdge# show running-config

Note that the Cisco Catalyst SD-WAN software supports VMXNET3 and Virtio vNICs. It is recommended, however, that you use the Virtio vNICs.

Mapping vNICs to Interfaces

When you create a vEdge Cloud router VM instance on KVM in the procedure in the previous section, you create two vNICs: vNIC 1, which is used for the management interface, and vNIC 2, which is used as a tunnel interface. From the perspective of the VM itself, these two vNICs map to the eth0 and eth1 interfaces, respectively. From the perspective of the Cisco Catalyst SD-WAN software for the vEdge Cloud router, these two vNICs map to the mgmt0 interface in VPN 512 and the ge0/0 interface in VPN 0, respectively. You cannot change these mappings.

You can configure up to five additional vNICs, numbered 3 through 7, on the VM host. You can map these vNICs to interfaces eth2 through eth7 as desired, and to Cisco Catalyst SD-WAN interfaces ge0/1 through ge0/7, as desired.

The table below summarizes the mapping between vNICs, VM host interfaces, and vEdge Cloud interfaces.

What's Next

See Install Signed Certificates on Edge Cloud Routers.

Certificates are used to authenticate routers in the overlay network. Once authentication is complete, the routers can establish secure sessions with other devices in the overlay network.

By default, the WAN Edge Cloud Certificate Authorization is automated. This is the recommended setting.

If you use third-party certificate authorization, configure certificate authorization to be manual:

1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

2. Click Hardware WAN Edge Certificate Authorization. (If you are usingCisco Catalyst SD-WAN Manager Release 20.12.1 or ealier, click Edit.

3. In the Security, select Enterprise Certificate (signed by Enterprise CA).

4. Click Save.

When a vEdge Cloud router virtual machine (VM) instance starts, it has a factory-default configuration, which allows the router to boot. However, the router is unable to join the overlay network. For the router to be able to join the overlay network, you must install a signed certificate on the router. The signed certificates are generated based on the router's serial number, and they are used to authorize the router to participate in the overlay network.

Starting from Releases 17.1, the Cisco SD-WAN Manager can act as a Certificate Authority (CA), and in this role it can automatically generate and install signed certificates on vEdge Cloud routers. You can also use another CA and then install the signed certificate manually. For Releases 16.3 and earlier, you manually install signed Symantec certificates on vEdge Cloud routers.

To install signed certificates:

1. Retrieve the vEdge authorized serial number file. This file contains the serial numbers of all the vEdge routers that are allowed to join the overlay network.

2. Upload the vEdge authorized serial number file to Cisco SD-WAN Manager.

3. Install a signed certificate on each vEdge Cloud router.

Retrieve vEdge Authorized Serial Number File

1. Go to http://viptela.com/support/ and log in.

2. Click Downloads.

3. Click My Serial Number Files. The screen displays the serial number files. Starting from Releases 17.1, the filename extension is .viptela. For Releases 16.3 and earlier, the filename extension is .txt.

4. Click the most recent serial number file to download it.

Upload vEdge Authorized Serial Number File

1. From the Cisco SD-WAN Manager menu, select Configuration > Devices.

2. Click vEdge List, and select Upload vEdge List.

3. In the Upload vEdge window:

   1. Click Choose File, and select the vEdge authorized serial number file you downloaded from Cisco.

   2. To automatically validate the vEdge routers and send their serial numbers to the controllers, click and select the checkbox Validate the Uploaded vEdge List and Send to Controllers. If you do not select this option, you must individually validate each router in the Configuration > Certificates > vEdge List page.

4. Click Upload.

During the process of uploading the vEdge authorized serial number file, the Cisco SD-WAN Manager generates a token for each vEdge Cloud router listed in the file. This token is used as a one-time password for the router. The Cisco SD-WAN Manager sends the token to the Cisco SD-WAN Validator and the Cisco SD-WAN Controller.

After the vEdge authorized serial number file has been uploaded, a list of vEdge routers in the network is displayed in the vEdge Routers Table in the Configuration > Devices page, with details about each router, including the router's chassis number and its token.

Install Signed Certificates in Releases 17.1 and Later

Starting from Releases 17.1, to install a signed certificates on a vEdge Cloud router, you first generate and download a bootstrap configuration file for the router. This file contains all the information necessary to allow the Cisco SD-WAN Manager to generate a signed certificate for the vEdge Cloud router. You then copy the contents of this file into the configuration for the router's VM instance. For this method to work, the router and the Cisco SD-WAN Manager must both be running Release 17.1 or later. Finally, you download the signed certificate to the router. You can configure the Cisco SD-WAN Manager to do this automatically or manually.

The bootstrap configuration file contains the following information:

• UUID, which is used as the router's chassis number.

• Token, which is a randomly generated one-time password that the router uses to authenticate itself with the Cisco SD-WAN Validator and the Cisco SD-WAN Manager.

• IP address or DNS name of the Cisco SD-WAN Validator.

• Organization name.

  Starting from Cisco Catalyst SD-WAN Manager Release 20.12.1 you cannot include a comma in the Organization Name field of the bootstrap configuration file.

• If you have already created a device configuration template and attached it to the vEdge Cloud router, the bootstrap configuration file contains this configuration. For information about creating and attaching a configuration template, see Create Configuration Templates for a vEdge Router .

You can generate a bootstrap configuration file that contains information for an individual router or for multiple routers.

Starting from Releases 17.1, you can also have Symantec generate signed certificates that you install manually on each router, as described later in this article, but this method is not recommended.

Configure the Cisco Catalyst SD-WAN Validator and Organization Name

Before you can generate a bootstrap configuration file, you must configure the Cisco SD-WAN Validator DNS name or address and your organization name:

1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

2. Click Validator. ( click Edit.)

3. In the DNS/IP Address: Port field, enter the DNS name or IP address of the Cisco SD-WAN Validator.

4. Click Save.

5. Verify the orgnization name. This name must be identical to that configured on the Cisco SD-WAN Validator. (If your are using Cisco Catalyst SD-WAN Manager Release 20.12.1 or earlier, to verify Organization Name, click View.

   Starting from Cisco Catalyst SD-WAN Manager Release 20.12.1, the system Organization Name cannot contain a comma. Comma is not allowed during the device configuration.

6. Click Save.

Configure Automatic or Manual vEdge Cloud Authorization

Signed certificates must be installed on each vEdge cloud router so that the router is authorized to participate in the overlay network. You can use the Cisco SD-WAN Manager as the CA to generate and install the signed certificate, or you can use an enterprise CA to install the signed certificate.

It is recommended that you use the Cisco SD-WAN Manager as a CA. In this role, Cisco SD-WAN Manager automatically generates and installs a signed certificate on the vEdge Cloud router. Having Cisco SD-WAN Manager act as a CA is the default setting. You can view this setting in the WAN vEdge Cloud Certificate Authorization, on the Cisco SD-WAN Manager Administration > Settings page.

To use an enterprise CA for generating signed certificates for vEdge Cloud routers:

1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

2. Click WAN Edge Cloud Certificate Authorization and select Manual.

3. Click Save.

Generate a Bootstrap Configuration File

Note

In Cisco SD-WAN Release 20.5.1, the cloud-init bootstrap configuration that you generate for the Cisco vEdge Cloud router cannot be used for deploying Cisco vEdge Cloud router 20.5.1. However, you can use the bootstrap configuration for deploying Cisco vEdge Cloud router 20.4.1 and the earlier versions.

To generate a bootstrap configuration file for a vEdge Cloud router:

1. From the Cisco SD-WAN Manager menu, select Configuration > Devices.

2. To generate a bootstrap configuration file for one or multiple vEdge Cloud routers:

   1. Click WAN Edge List, select Export Bootstrap Configuration.

   2. In the Generate Bootstrap Configuration field, select the file format:

      • For a vEdge Cloud router on a KVM hypervisor or on an AWS server, select Cloud-Init to generate a token, Cisco SD-WAN Validator IP address, vEdge Cloud router UUID, and organization name.

        Starting from Cisco Catalyst SD-WAN Manager Release 20.12.1, the system Organization Name cannot contain a comma. Comma is not allowed during the device configuration.

      • For a vEdge Cloud router on a VMware hypervisor, select Encoded String to generate an encoded string.

   3. From the Available Devices column, select one or more routers.

   4. Click the arrow pointing to right to move the selected routers to Selected Devices column.

   5. Click Generate Generic Configuration. The bootstrap configuration is downloaded in a .zip file, which contains one .cfg file for each router.

3. To generate a bootstrap configuration file individually for each vEdge Cloud router:

   1. Click WAN Edge List, select the desired vEdge Cloud router.

   2. For the desired vEdge Cloud router, click ..., and select Generate Bootstrap Configuration.

   3. In the Generate Bootstrap Configuration window, select the file format:

      • For a vEdge Cloud router on a KVM hypervisor or on an AWS server, select Cloud-Init to generate a token, Cisco SD-WAN Validator IP address, vEdge Cloud router UUID, and organization name.

      • For a vEdge Cloud router on a VMware hypervisor, select Encoded String to generate an encoded string.

      Note

      Beginning with Cisco vManage Release 20.7.1, there is an option available when generating a bootstrap configuration file for a Cisco vEdge device, enabling you generate two different forms of the bootstrap configuration file. If you are generating a bootstrap configuration file for a Cisco vEdge device that is using Cisco Catalyst SD-WAN Release 20.4.x or earlier, then check the The version of this device is 20.4.x or earlier check box. If you are generating a bootstrap configuration for a Cisco vEdge device that is using Cisco SD-WAN Release 20.5.1 or later, then do not use the check box.

   4. Click Download to download the bootstrap configuration. The bootstrap configuration is downloaded in a .cfg file.

   Then use the contents of the bootstrap configuration file to configure the vEdge Cloud router instance in AWS, ESXi, or KVM. For example, to configure a router instance in AWS, paste the text of the Cloud-Init configuration into the User data field:

By default, the ge0/0 interface is the router's tunnel interface, and it is configured as a DHCP client. To use a different interface or to use a static IP address, and if you did not attach a device configuration template to the router, change the vEdge Cloud router's configuration from the CLI. See Configuring Network Interfaces.

Install the Certificate on the vEdge Cloud Router

If you are using automated vEdge Cloud certificate authorization, which is the default, after you configure the vEdge Cloud router instance, Cisco SD-WAN Manager automatically installs a certificate on the router and the router's token changes to its serial number. You can view the router's serial number in the Configuration > Devices page. After the router's control connections to the Cisco SD-WAN Manager come up, any templates attached to the router are automatically pushed to the router.

If you are using manual vEdge Cloud certificate authorization, after you configure the vEdge Cloud router instance, follow this procedure to install a certificate on the router:

1. Install the enterprise root certificate chain on the router:

   vEdge# request root-cert-chain install filename [vpn vpn-id] 

   Then, Cisco SD-WAN Manager generates a CSR.

2. Download the CSR:

   1. From the Cisco SD-WAN Manager menu, select Configuration > Certificates.

   2. For the selected vEdge Cloud router for which to sign a certificate, click ... and select View CSR.

   3. To download the CSR, click Download.

3. Send the certificate to a third-party signing authority, to have them sign it.

4. Import the certificate into the device:

   1. From the Cisco SD-WAN Manager menu, select Configuration > Certificates.

   2. Click Controllers, and select Install Certificate.

   3. In the Install Certificate page, paste the certificate into the Certificate Text field, or click Select a File to upload the certificate in a file.

   4. Click Install.

5. Issue the following REST API call, specifying the IP address of your Cisco SD-WAN Manager:

   https://vmanage-ip-address/dataservice/system/device/sync/rootcertchain

Create the vEdge Cloud Router Bootstrap Configuration from the CLI

It is recommended that you generate the vEdge Cloud router's bootstrap configuration using Cisco SD-WAN Manager. If, for some reason, you do not want to do this, you can create the bootstrap configuration using the CLI. With this process, you must still, however, use Cisco SD-WAN Manager. You collect some of this information for the bootstrap configuration from Cisco SD-WAN Manager, and after you have created the bootstrap configuration, you use Cisco SD-WAN Manager to install the signed certificate on the router.

Installing signed certificates by creating a bootstrap configuration from the CLI is a three-step process:

1. Edit the router's configuration file to add the DNS name or IP address of the Cisco SD-WAN Validator and your organization name.

2. Send the router's chassis and token numbers to Cisco SD-WAN Manager.

3. Have Cisco SD-WAN Manager authenticate the vEdge Cloud router and install the signed certificate on the router.

To edit the vEdge Cloud router's configuration file from the CLI:

1. Open a CLI session to the vEdge Cloud router via SSH. To do this in Cisco SD-WAN Manager, select Tools > SSH Terminal page, and select the desired router.

2. Log in as the user admin, using the default password, admin. The CLI prompt is displayed.

3. Enter configuration mode:

   vEdge# config
   vEdge(config)#

4. Configure the IP address of the Cisco SD-WAN Validator or a DNS name that points to the Cisco SD-WAN Validator. The Cisco SD-WAN Validator's IP address must be a public IP address:

   vEdge(config)# system vbond (dns-name | ip-address)

5. Configure the organization name:

   vEdge(config-system)# organization-name name

6. Commit the configuration:

   vEdge(config)# commit and-quit
   vEdge#

To send the vEdge Cloud router's chassis and token numbers to Cisco SD-WAN Manager:

1. Locate the vEdge Cloud router's token and chassis number:

   1. From the Cisco SD-WAN Manager menu, select Configuration > Devices.

   2. Click WAN Edge List, locate the vEdge Cloud router.

   3. Make a note of the values in the vEdge Cloud router's Serial No./Token and Chassis Number columns.

2. Send the router's bootstrap configuration information to Cisco SD-WAN Manager:

   vEdge# request vedge-cloud activate chassis-number chassis-number token token-number

Issue the show control local-properties command on the router to verify the Cisco SD-WAN Validator IP address, the organization name the chassis number, and the token. You can also verify whether the certificate is valid.

Finally, have Cisco SD-WAN Manager authenticate the vEdge Cloud router and install the signed certificate on the router.

If you are using automated vEdge Cloud certificate authorization, which is the default, the Cisco SD-WAN Manager uses the chassis and token numbers to authenticate the router. Then, Cisco SD-WAN Manager automatically installs a certificate on the router and the router's token changes to a serial number. You can display the router's serial number in the Configuration > Devices page. After the router's control connections to Cisco SD-WAN Manager come up, any templates attached to the router are automatically pushed to the router.

If you are using manual vEdge Cloud certificate authorization, after you configure the vEdge Cloud router instance, follow this procedure to install a certificate on the router:

1. Install the enterprise root certificate chain on the router:

   vEdge# request root-cert-chain install filename [vpn vpn-id] 

   After you install the root chain certificate on the router, and after Cisco SD-WAN Manager receives the chassis and token numbers, Cisco SD-WAN Manager generates a CSR.

2. Download the CSR:

   1. From the Cisco SD-WAN Manager menu, select Configuration > Certificates.

   2. For the selected vEdge Cloud router for which to sign a certificate, click ... and select View CSR.

   3. To download the CSR, click Download.

3. Send the certificate to a third-party signing authority, to have them sign it.

4. Import the certificate into the device:

   1. From the Cisco SD-WAN Manager menu, select Configuration > Certificates.

   2. Click Controllers and select Install Certificate.

   3. In the Install Certificate page, paste the certificate into the Certificate Text field, or click Select a File to upload the certificate in a file.

   4. Click Install.

5. Issue the following REST API call, specifying the IP address of your Cisco SD-WAN Manager:

   https://vmanage-ip-address/dataservice/system/device/sync/rootcertchain

Install Signed Certificates in Releases 16.3 and Earlier

For vEdge Cloud router virtual machine (VM) instances running Releases 16.3 and earlier, when the vEdge Cloud router VM starts, it has a factory-default configuration, but is unable to join the overlay network because no signed certificate is installed. You must install a signed Symantec certificate on the vEdge Cloud router so that it can participate in the overlay network.

To generate a certificate signing request (CSR) and install the signed certificate on the vEdge Cloud router:

1. Log in to the vEdge Cloud router as the user admin, using the default password, admin. If the vEdge Cloud router is provided through AWS, use your AWS key pair to log in. The CLI prompt is displayed.

2. Generate a CSR for the vEdge Cloud router:

   vEdge# request csr upload path

   path is the full path and filename where you want to upload the CSR. The path can be in a directory on the local device or on a remote device reachable through FTP, HTTP, SCP, or TFTP. If you are using SCP, you are prompted for the directory name and filename; no file path name is provided. When prompted, enter and then confirm your organization name. For example:

   vEdge# request csr upload home/admin/vm9.csr
   Uploading CSR via VPN 0
   Enter organization name            : Cisco
   Re-enter organization name         : Cisco
   Generating CSR for this vEdge device
   ........[DONE]
   Copying ... /home/admin/vm9.csr via VPN 0
   CSR upload successful

3. Log in to the Symantec Certificate Enrollment portal:

   https://certmanager.<wbr/>websecurity.symantec.com/<wbr/>mcelp/enroll/index?jur_hash=<wbr/>f422d7ceb508a24e32ea7de4f78d37<wbr/>f8

4. In the Select Certificate Type drop-down, select Standard Intranet SSL and click Go. The Certificate Enrollment page is displayed. Cisco Catalyst SD-WAN uses the information you provide on this form to confirm the identity of the certificate requestor and to approve your certificate request. To complete the Certificate Enrollment form:

   1. In the Your Contact Information section, specify the First Name, Last Name, and Email Address of the requestor.

   2. In the Server Platform and Certificate Signing section, select Apache from the Select Server Platform drop-down. In the Enter Certificate Signing Request (CSR) box, upload the generated CSR file, or copy and paste the contents of the CSR file. (For details about how to do this, log in to support.viptela.com. Click Certificate, and read the Symantec certificate instructions.)

   3. In the Certificate Options section, enter the validity period for the certificate.

   4. In the Challenge Phrase section, enter and then re-enter a challenge phrase. You use the challenge phrase to renew, and, if necessary, to revoke a certificate on the Symantec Customer Portal. It is recommended that you specify a different challenge phrase for each CSR.

   5. Accept the Subscriber Agreement. The system generates a confirmation message and sends an email to the requestor confirming the certificate request. It also sends an email to the Cisco to approve the CSR.

5. After Cisco approves the CSR, Symantec sends the signed certificate to the requestor. The signed certificate is also available through the Symantec Enrollment portal.

6. Install the certificate on the vEdge Cloud router:

   vEdge# request certificate install filename [vpn vpn-id] 

   The file can be in your home directory on the local device, or it can be on a remote device reachable through FTP, HTTP, SCP, or TFTP. If you are using SCP, you are prompted for the directory name and filename; no file path name is provided.

7. Verify that the certificate is installed and valid:

   vEdge# show certificate validity 

After you have installed the certificate on the vEdge Cloud router, the Cisco SD-WAN Validator is able to validate and authenticate the router, and the router is able to join the overlay network.

What's Next

See Send vEdge Serial Numbers to the Controller Devices.

Only authorized routers can join the overlay network. The controller devices Cisco SD-WAN Manager, Cisco SD-WAN Controllers and Cisco SD-WAN Validators learn which routers are authorized to join the overlay network from the router-authorized serial number file. This is a file that you receive from Cisco. The router authorized serial number file lists the serial numbers and corresponding chassis numbers for all authorized routers. Upload the file to one of the Cisco SD-WAN Manager in your network, and it then distributes the file to the controllers.

When you upload the router serial number file, you can place the routers in one of these states:

• Invalid: When you power on the routers, they are not authorized to join the overlay network.

• Staging: When you power on the routers, they are validated and authorized to join the overlay network, and can establish connections only to the control plane. Over the control plane, the routers receive their configuration from Cisco SD-WAN Manager. However, the routers are unable to establish data plane connections, so they cannot communicate with other routers in the network. The Staging state is useful when you are preparing routers at one location and then sending them to different sites for installation. Once the routers reach their final destination, you change their state from Staging to Valid, to allow the routers to establish data plane connections and to fully join the overlay network.

• Valid: When you power on the routers, they are validated and authorized to join the overlay network, and they are able to establish both control plane and data plane connections in the network. Over the control plane, the routers receive their configuration from Cisco SD-WAN Manager. Over the data plane, they are able to communicate with other routers. The Valid state is useful when the routers are being installed at their final destination.

Note

To successfully send a router serial number file to Cisco Catalyst SD-WAN Manager in Cisco vManage Release 20.10.1 and earlier, ensure that the file is installed in /home/admin or /home/vmanage-admin. Using credentials other than admin or vmanage-admin to send a router serial number file will result in an error.

Once you have set up and started the virtual machines (VMs) for the vEdge Cloud routers and set up and started the hardware vEdge routers in your overlay network, they come up with a factory-default configuration.

Note

Log In to a Device for the First Time: When you first deploy a Cisco Catalyst SD-WAN overlay network, log in to the Cisco SD-WAN Validator, Cisco SD-WAN Manager, and Cisco SD-WAN Controller to manually create the device's initial configuration. Routers are shipped with a factory default configuration. If you choose to modify this configuration manually, log in through the router's console port.

For the overlay network to be operational and for the vEdge routers to be able to participate in the overlay network, you must do the following:

• Configure a tunnel interface on at least one interface in VPN 0. This interface must be connected to a WAN transport network that is accessible to all Cisco vEdge devices. VPN 0 carries all control plane traffic between the Cisco vEdge devices in the overlay network.

• Ensure that the Overlay Management Protocol (OMP) is enabled. OMP is the protocol responsible for establishing and maintaining the Cisco Catalyst SD-WAN control plane. It is enabled by default, and you cannot disable it. If you edit the configuration from the CLI, do not remove the omp configuration command.

• Ensure that BFD is enabled. BFD is the protocol that the transport tunnels on vEdge routers use for transmitting data traffic through the overlay network. BFD is enabled by default, and cannot be disabled. If you edit the configuration from the CLI, do not remove the bfd color command.

• Configure the IP address of DNS name of your network's Cisco SD-WAN Validator.

• Configure the IP address of the router.

  Note

  The DNS cache timeout should be proportional to the number of Cisco SD-WAN Validator IP addresses that DNS has to resolve, otherwise the control connection for Cisco SD-WAN Manager may not occur during a link failure. This is because, when there are more than six IP addresses (this is the recommended number since the default DNS cache timeout is currently two minutes) to be checked, the DNS cache timer expires even as the highest preferred interface tries all Cisco SD-WAN Validator IP addresses, before failing over to a different color. For instance, it takes about 20 seconds to attempt to connect to one IP address. So, if there are eight IP addresses to be resolved, the DNS cache timeout should be 20*8=160 seconds or three minutes.

You should also assign a system IP address to each vEdge router. This address, which is similar to the router ID on non-Cisco vEdge devices, is a persistent address that identifies the router independently of any interface addresses. The system IP is a component of the device's TLOC address. Setting the system IP address for a device allows you to renumber interfaces as needed without affecting the reachability of the Cisco vEdge device. Control traffic over secure DTLS or TLS connections between Cisco SD-WAN Controllers and vEdge routers and between Cisco SD-WAN Controllers and Cisco SD-WAN Validators is sent over the system interface identified by the system IP address. In the transport VPN (VPN 0), the system IP address is used as the loopback address of the device. You cannot use the same address for another interface in VPN 0.

You can also configure other features and functions required for your network topology.

You configure vEdge routers by creating configuration templates on the Cisco SD-WAN Manager. For each configuration templates, you create one or more feature templates, which you then consolidate into a vEdge router device template. You then attach the device template to a vEdge router. When the vEdge router joins the overlay network, the Cisco SD-WAN Manager automatically pushes the configuration template to the router.

It is strongly recommended that you create the full configuration for vEdge routers by creating configuration templates on the Cisco SD-WAN Manager. When the Cisco SD-WAN Manager discovers a router in the overlay network, it pushes the appropriate configuration template to the device. The configuration parameters in the configuration template overwrite the initial configuration.

Create Configuration Templates for the vEdge Routers

To create vEdge configuration templates, first create feature templates:

1. From the Cisco SD-WAN Manager menu, choose Configuration > Templates.

2. Click Feature Templates.

   Note	In Cisco vManage Release 20.7.x and earlier releases, Feature Templates is called Feature.

3. Click Add Template.

4. In the left pane, select vEdge Cloud or a router model.

5. In the right pane, select the System feature template. Configure the following parameters:

   1. Template Name

   2. Description

   3. Site ID

   4. System IP

   5. Timezone

   6. Hostname

   7. Console baud rate (vEdge hardware routers only)

   8. GPS location

6. Click Save to save the System template.

7. In the right pane, select VPN-Interface-Ethernet feature template. Configure the following parameters:

   1. Template Name

   2. Description

   3. Shutdown No

   4. Interface name

   5. IPv4 address (static or DHCP)

   6. IPv6 address (static of DHCPv6), if desired (in Releases 16.3 and later)

   7. Tunnel interface (for VPN 0), color, encapsulation, and services to allow.

8. Click Save to save the VPN-Interface Ethernet template.

9. In the right pane, select other templates to configure any desired features. Save each template when you complete the configuration. For information about configuration cellular parameters for vEdge 100m and vEdge 100wm routers, see the next section in this article.

For information about configuration templates and parameters, see the Cisco SD-WAN Manager configuration help articles for your software release.

Next, create a device template that incorporates all the feature templates for the vEdge router:

1. From the Cisco SD-WAN Manager menu, choose Configuration > Templates.

2. Click Device Templates.

   Note	In Cisco vManage Release 20.7.x and earlier releases, Device Templates is called Device.

3. From the Create Template drop-down list, select From Feature Template.

4. From the Device Model drop-down, select the type of device for which you are creating the device template. Cisco SD-WAN Manager displays the feature templates for the device type you selected. Required templates are indicated with an asterisk (*).

5. Enter a name and description for the device template. These fields are mandatory. The template name cannot contain special characters.

6. In the Transport & Management VPN section, under VPN 0, from the drop-down list of available templates, select the desired feature template. The list of available templates shows the ones that you have previously created.

7. To include additional feature templates in the device template, in the remaining sections, select the feature templates in turn, and from the drop-down list of available templates, select the desired template. The list of available templates are the ones that you have previously created. Ensure that you select templates for all mandatory feature templates and for any desired optional feature templates.

8. Click Create to create the device template.

To attach a device template to a device:

1. From the Cisco SD-WAN Manager menu, choose Configuration > Templates.

2. Click Device Templates, and select a template.

   Note	In Cisco vManage Release 20.7.x and earlier releases, Device Templates is called Device.

3. For the selected template, click ... and select Attach Device.

4. In the Attach Device window, either search for a device or select a device from the Available Device(s) column.

5. Click the arrow pointing right to move the device to the Selected Device(s) column on the right.

6. Click Attach.

When Cisco SD-WAN Manager discovers that the vEdge router has joined the overlay network, it pushes the configuration template to the router.

Configuring Cellular Routers

For vEdge 100m and vEdge 100wm routers, you configure cellular interface parameters on the VPN-Interface-Cellular feature template. In this template, the default Profile ID is 0, which enables automatic profile selection. The automatic profile uses the Mobile Country Code/Mobile Network Code (MCC/MNC) values on the router's SIM card. Profile 0 enables the cellular router to automatically join the overlay network during the Cisco Catalyst SD-WAN ZTP automatic provisioning process .

If your MCC/MNC is not supported, the automatic profile selection process fails, and the ZTP process is unable to autodetect the router. In this case, you must configure a cellular profile as follows:

1. In the right pane, select the Cellular Profile feature template.

2. Set the Profile ID to a value from 1 through 15, and configure the desired cellular parameters.

3. Save the Cellular Profile feature template.

4. In the right pane, select the VPN-Interface-Cellular template.

5. Select the Profile ID you configured in Step 2, and for Shutdown, click Yes.

6. Save the VPN-Interface-Cellular feature template.

7. Include the Cellular Profile and VPN-Interface Cellular templates in a device template.

8. Attach the device template to the vEdge router to activate the MCC/MCN.

9. In the right pane, select the VPN-Interface-Cellular template.

10. For Shutdown click No, to enable the cellular interface.

11. Save the VPN-Interface-Cellular feature template.

12. Repush the device template to the vEdge router. This is the device template that you pushed in Step 8.

Configure the vEdge Routers from the CLI

Normally, you create vEdge router configurations using Cisco SD-WAN Manager configuration templates. However, in some situations, such as network test and proof-of-concept (POC) environments, you might want to configure vEdge routers manually, either to speed up the configuration process or because your test environment does not include Cisco SD-WAN Manager. In such situations, you can configure vEdge routers from the router's CLI.

Note

If you configure a vEdge router manually from the CLI and then the router later becomes managed by a Cisco SD-WAN Manager, when the Cisco SD-WAN Manager discovers the router, it pushes the router's configuration from the Cisco SD-WAN Manager server to the router, overwriting the existing configuration.

For vEdge Cloud routers, use SSH to open a CLI session to the router. For hardware vEdge routers, connect to the router via the management console.

Configure Minimum Parameters from the CLI

To create the initial configuration on a Cisco vEdge device from a CLI session:

1. Open a CLI session to the Cisco vEdge device via SSH or the console port.

2. Log in as the user admin, using the default password, admin. The CLI prompt is displayed.

3. Enter configuration mode:

   vEdge# config 
   vEdge(config)#

4. Configure the hostname:

   vEdge(config)# system host-name hostname

   Configuring the hostname is optional, but is recommended because this name in included as part of the prompt in the CLI and it is used on various Cisco SD-WAN Manager pages to refer to the device.

5. Configure the system IP address. Starting from Releases 16.3 and later, the IP address can be an IPv4 or an IPv6 address. In earlier releases, it must be an IPv4 address.

   vEdge(config-system)#system-ip ip-address

   Cisco SD-WAN Manager uses the system IP address to identify the device so that the NMS can download the full configuration to the device.

6. Configure the numeric identifier of the site where the device is located:

   vEdge(config-system)# site-id site-id

7. Configure the organization name:

   vEdge(config-system)# organization-name organization-name

8. Configure the IP address of the Cisco SD-WAN Validator or a DNS name that points to the Cisco SD-WAN Validator. The IP address of the Cisco SD-WAN Validator must be a public IP address, to allow all Cisco vEdge devices in the overlay network to reach the Cisco SD-WAN Validator:

   vEdge(config-system)# vbond (dns-name | ip-address)

9. Configure a time limit for confirming that a software upgrade is successful:

   vEdge(config-system)# upgrade-confirm minutes

   The time can be from 1 through 60 minutes. If you configure this time limit, when you upgrade the software on the device, the Cisco SD-WAN Manager (when it comes up) or you must confirm that a software upgrade is successful within the configured number of minutes. If the device does not received the confirmation within the configured time, it reverts to the previous software image.

10. Change the password for the user "admin":

    vEdge(config-system)# user admin password password

    The default password is "admin".

11. Configure an interface in VPN 0 to be used as a tunnel interface. VPN 0 is the WAN transport VPN, and the tunnel interface carries the control traffic among the devices in the overlay network. For vEdge Cloud routers, the interface name has the format eth number. For hardware vEdge routers, the interface name has the format ge slot / port. You must enable the interface and configure its IP address, either as a static address or as a dynamically assigned address received from a DHCP server. Starting from Releases 16.3 and later, the IP address can be an IPv4 or an IPv6 address, or you can configure both to enable dual-stack operation. In earlier releases, it must be an IPv4 address.

    ​vEdge(config)# vpn 0
    vEdge(config-vpn-0)# interface interface-name
    vEdge(config-interface)# (ip dhcp-client | ip address prefix/length) 
    vSmart(config-interface)# (ipv6 address ipv6-prefix/length | ipv6 dhcp-client [dhcp-distance number | dhcp-rapid-commit]) 
    vEdge(config-interface)# no shutdown 
    vEdge(config-interface)# tunnel-interface

    Note

    You must configure a tunnel interface on at least one interface in VPN 0 in order for the overlay network to come up and for the Cisco SD-WAN Manager to be able to participate in the overlay network. This interface must connect to a WAN transport network that is accessible by all Cisco vEdge devices. VPN 0 carries all control plane traffic among the Cisco vEdge devices in the overlay network.

12. Configure a color for the tunnel to identify the type of WAN transport. You can use the default color (default), but you can also configure a more appropriate color, such as mpls or metro-ethernet, depending on the actual WAN transport.

    vEdge(config-tunnel-interface)# color color

13. Configure a default route to the WAN transport network:

    vEdge(config-vpn-0)# ip route 0.0.0.0/0 next-hop

14. Commit the configuration:

    vEdge(config)# commit and-quit
    vEdge#

15. Verify that the configuration is correct and complete:

    vEdge# show running-config

After the overlay network is up and operational, create a vEdge configuration template on the Cisco SD-WAN Manager that contains the initial configuration parameters. Use the following Cisco SD-WAN Manager feature templates:

• System feature template to configure the hostname, system IP address, and Cisco SD-WAN Validator functionality.

• AAA feature template to configure a password for the "admin" user.

• VPN-Interface-Ethernet feature template to configure the interface in VPN 0.

In addition, it is recommended that you configure the following general system parameters:

• From the Cisco SD-WAN Manager menu, select Administration > Settings and configure Organization name.

• From the Cisco SD-WAN Manager menu, select Configuration > Templates. For the NTP and System feature configuration templates, configure Timezone, NTP servers, and device physical location.

  • For the Banner feature configuration template, configure Login banner.

  • For the Logging feature configuration template, configure Logging parameters.

  • For the AAA feature configuration template, configure AAA, and RADIUS and TACACS+ servers.

  • For the SNMP feature configuration template, configure SNMP.

Sample Initial CLI Configuration

Below is an example of a simple configuration on a vEdge router. Note that this configuration includes a number of settings from the factory-default configuration and shows a number of default configuration values.

vEdge# show running-config
system
 host-name         vEdge
 gps-location latitude 40.7127837
 gps-location longitude -74.00594130000002
 system-ip         172.16.251.20
 site-id           200
 max-controllers   1
 organization-name "Cisco"
 clock timezone America/Los_Angeles
 upgrade-confirm   15
 vbond 184.122.2.2
 aaa
  auth-order local radius tacacs
  usergroup basic
   task system read write
   task interface read write
  !
  usergroup netadmin
  !
  usergroup operator
   task system read
   task interface read
   task policy read
   task routing read
   task security read
  !
  user admin
   password encrypted-password
  !
 !
 logging
  disk
   enable
  !
 !
 ntp
  keys
   authentication 1 md5 $4$L3rwZmsIic8zj4BgLEFXKw== 
   authentication 2 md5 $4$LyLwZmsIif8BvrJgLEFXKw== 
   authentication 60124 md5 $4$LXbzZmcKj5Bd+/BgLEFXKw==
   trusted 1 2 60124
  !
  server 180.20.1.2
   key              1
   source-interface ge0/3
   vpn              1
   version          4
  exit
 !
 radius
  server 180.20.1.2
   vpn              1 
   source-interface ge0/3
   secret-key       $4$L3rwZmsIic8zj4BgLEFXKw==
  exit
 !
 tacacs
  server 180.20.1.2 
   vpn              1024
   source-interface ge0/3
   secret-key       $4$L3rwZmsIic8zj4BgLEFXKw==
  exit  
 !
!                                                                                                            
omp
 no shutdown
 gradeful-restart
 advertise bgp
 advertise connected
 advertise static
!
security
 ipsec
  authentication-type ah-sha1-hmac sha1-hman
 !
! 
snmp
 no shutdown
 view v2
  oid 1.3.6.1
 !
 community private
  view          v2
  authorization read-only
 !
 trap target vpn 0 10.0.1.1 16662
  group-name     Cisco
  community-name private
 !
 trap group test
  all
   level critical major minor
  exit
 exit
!
vpn 0
 interface ge0/0
  ip address 184.111.20.2/24
  tunnel-interface
   encapsulation ipsec
   color mpls restrict
   no allow-service bgp
   allow-service dhcp
   allow-service dns
   allow-service icmp
   no allow-service sshd
   no allow-service netconf
   no allow-service ntp
   no allow-service ospf
   no allow-service stune
  !
  no shutdown
  bandwidth-upstream   60
  bandwidth-downstream 60
 !
 interface ge0/1
   no shutdown
 !
 interface ge0/2
   no shutdown
 !
 ip route 0.0.0.0/0 184.111.20.1                                                                                        
! 
vpn 1
 router
   bgp 111000
    neighbor 172.16.1.20
     no shutdown
     remote-as 111000
     password  $4$LzLwZj1ApK4zj4BgLEFXKw==
    !
   !
   ospf
    timers spf 200 1000 10000
    area 0
     interface ge0/1
       authentication type message-direct
       authentication message-digest message-digest-key 1 md5 $4$LzLwZj1ApK4zj4BgLEFXKw==
      exit
     exit
   !
 !
!

By default, collecting streams of data from a network device is not enabled.

To collect data streams from a WAN Edge router in the overlay network, perform the following steps.

Collecting data streams also requires that VPN 512 be configured in your Cisco Catalyst SD-WAN network.

1. From the Cisco SD-WAN Manager menu, select Administration > Settings.

2. Click Data Stream. (If you are using Cisco Catalyst SD-WAN Manager Release 20.12.1 or earlier, click Edit.)

3. Enable Data Stream.

4. From Cisco vManage Release 20.4.1, choose one of the following IP Address Type options:

   • Transport: Click this option send the data stream to the transport IP address of the Cisco SD-WAN Manager node to which the device is connected.

   • Management: Click this option send the data stream to the management IP address of the Cisco SD-WAN Manager node to which the device is connected.

   • System: Click this option to send the data stream to the internally configured system IP address of theCisco SD-WAN Manager node to which the device is connected.

     In a Cisco SD-WAN Manager cluster deployment, we recommend that you choose System so that the data stream is collected from edge devices that are managed by all Cisco SD-WAN Manager instances in the cluster.

5. From Cisco vManage Release 20.4.1, perform one of these actions:

   • If you choose Transport as the IP address type, in the Hostname field, enter the public transports IP address that is used to connect to the router.

     You can determine this IP address by using an SSH client to access the router and entering the show interface CLI command.

   • If you choose Management as the IP address type, in the Hostname field, enter the IP address or name of the host to collect the data.

     We recommend that this host is one that is used for out-of-band management and that it is located in the management VPN.

   This Hostname option is dimmed when IP Address Type is System.

6. In the VPN field, enter the number of the VPN in which the host is located.

   We recommend that this VPN be the management VPN, which is typically VPN 512.

   This VPN option is dimmed when IP Address Type is System.

7. Click Save.

Cisco Catalyst SD-WAN provides an automatic provisioning software as a service (SaaS) called zero-touch provisioning (ZTP), which allows hardware vEdge routers to join the overlay network automatically. The ZTP process begins when you power on a hardware vEdge router for the first time.

For the ZTP process to work:

• The edge or gateway router at the site where the hardware vEdge router is located must be able to reach public DNS servers. We recommend that the router be configured to reach the Google public DNS servers.

• For Cisco vEdge devices, the edge or gateway router at the site must be able to reachztp.viptela.com.

• For Cisco IOS XE Catalyst SD-WAN devices, the edge or gateway router at the site must be able to reach ztp. local-domain.

• A network cable must be plugged into the interface that the hardware router uses for ZTP. These interfaces are:

  • For Cisco vEdge 1000 routers: ge0/0

  • For Cisco vEdge 2000 routers: ge2/0

  • For Cisco vEdge 100 series routers: ge0/4

  • For Cisco IOS XE Catalyst SD-WAN devices, there is no specific interface that is used for connection to the ZTP server. The router attempts to obtain a DHCP IP address on one interface at a time. It uses the first interface on which it obtains the DHCP IP address to resolve the domain name ztp. local-domain to the IP address of the ZTP server.

The ZTP process occurs in the following sequence:

1. The hardware router powers up.

2. The router attempts to contact a DHCP server, sending a DHCP discovery message.

   1. If a DHCP server is present in the network, the router receives a DHCP offer message that contains the IP address of its ZTP interface. Then, the ZTP process continues with Step 3.

   2. For Cisco vEdge devices, and for Cisco IOS XE Catalyst SD-WAN devices from Cisco IOS XE Catalyst SD-WAN Release 17.7.1a, if no DHCP server is present, the router does not receive a DHCP offer. In this situation, the router initiates an automatic IP address detection process (also referred to as auto-IP). This process examines the ARP packets on the subnetwork and, from these packets, it infers the IP address of the ZTP interface. Then, the ZTP process continues with Step 3.

      For Cisco IOS XE Catalyst SD-WAN devices before Cisco IOS XE Catalyst SD-WAN Release 17.7.1a, if no DHCP server is present, the ZTP process does not continue.

3. The router contacts a DNS server to resolve the hostname ztp.viptela.com (for Cisco vEdge devices) or ztp. local-domain (Cisco IOS XE Catalyst SD-WAN devices) and receives the IP address of the Cisco Catalyst SD-WAN ZTP server

4. The router connects to the ZTP server. The ZTP server verifies the vEdge router and sends the IP address of the Cisco SD-WAN Validator. This Cisco SD-WAN Validator has the same Organization name as the vEdge router.

5. The router establishes a transient connection to the Cisco SD-WAN Validator and sends its chassis ID and serial number. (At this point in the ZTP process, the router does not have a system IP address, so the connection is established with a null system IP address.) The Cisco SD-WAN Validator uses the chassis ID and serial number to verify the router. The Cisco SD-WAN Validator then sends the IP address of Cisco SD-WAN Manager to the router.

6. The router establishes a connection to and is verified by Cisco SD-WAN Manager. Cisco SD-WAN Manager sends the router its system IP address.

7. The router re-establishes a connection to the Cisco SD-WAN Validator using its system IP address.

8. The router re-establishes a connection to Cisco SD-WAN Manager using its system IP address.

   For Cisco vEdge devices, if necessary, Cisco SD-WAN Manager pushes the proper software image to the vEdge router. As part of the software image installation, the router reboots.

9. After the reboot, the router reestablishes a connection to the Cisco SD-WAN Validator, which again verifies the router.

10. The router establishes a connection to Cisco SD-WAN Manager, which pushes the full configuration to the router. (If the router has rebooted, it re-establishes a connection to Cisco SD-WAN Manager.)

11. The router joins the organization's overlay network.

Note

For the ZTP process to succeed, Cisco SD-WAN Manager must contain a device configuration template for the vEdge router. If the Cisco SD-WAN Manager instance has no template, the ZTP process fails. Ignore the device-model and ztp-status display in the configuration preview and intent configuration. This information is visible after you push the configuration on device side.

Using ZTP on Non-Wireless Routers

The default configuration that is shipped on non-wireless hardware vEdge routers includes the following commands that allow the ZTP process to occur automatically:

• system vbond ztp.viptela.com—Configures the initial Cisco SD-WAN Validator to be the Cisco Catalyst SD-WAN ZTP SaaS server.

• vpn 0 interface ip dhcp-client—Enables DHCP on one of the interfaces in VPN 0, which is the transport interface. Note that the actual interface in the default configuration varies by router model. This interface must be connected to the Internet, MPLS, metro Ethernet, or other WAN network.

Warning: For ZTP to work, do not modify or delete either of these configuration commands before you connect the vEdge router to a WAN.

Using ZTP on Wireless Routers

The vEdge 100m and vEdge 100wm are wireless routers. On these routers, ZTP is supported using both the cellular and the Ethernet interfaces.

Note	In Release 16.3, you cannot use the LTE USB dongle on a vEdge 1000 router for ZTP.

The vEdge 100m router supports software Releases 16.1 and later. If the vEdge 100m router is running Release 16.2.10 or later, we recommend, when performing ZTP, that Cisco SD-WAN Manager also be running Release 16.2.10 or later.

The vEdge 100wm router supports software Releases 16.3 and later.

The default configuration that is shipped on wireless hardware vEdge routers includes the following commands that allow the ZTP process to occur automatically on the cellular interface:

• system vbond ztp.viptela.com: Configure the initial Cisco SD-WAN Validator to be the Cisco Catalyst SD-WAN ZTP SaaS server.

• vpn 0 interface cellular0 ip dhcp-client : Enable DHCP on one of the cellular interface called cellular0 in VPN 0, which is the transport interface. This interface must be connected to the cellular network.

• vpn 0 interface cellular0 technology : Associate a radio access technology (RAT) with the cellular interface. In the default configuration, the RAT is set to lte. For ZTP to work, you must change this value to auto.

• vpn 0 interface cellular0 profile 0 : Enable automatic profile selection. For firmware-dependent mobile carriers, the automatic profile uses the firmware default values. For other carriers, the automatic profile uses the Mobile Country Code/Mobile Network Code (MCC/MNC) values on the SIM card. One exception is the vEdge 100m-NT: The automatic profile tries OCN MVNO APN before the firmware default, which is NTT Docomo. If the router finds a matching entry, it autocreates profile 16, which is used for the ZTP connection. To check which profile is being used for the active ZTP connection, look at the Active profile entry in the show cellular sessions command output.

  The profile 0 configuration command recognizes the MCCs and MCNs listed in the vEdge SKU Information table. If your MCC/MNC is supported, you do not need to configure them in the Cellular Profile feature template or with the profile command. If your MCC/MNC is not supported, you must configure them manually, using the Cellular-Profile configuration template or the profile CLI command.

If you need to use Cisco SD-WAN Manager configuration templates to create the portions of the default configuration that allow ZTP to occur automatically, use the VPN-Interface-Cellular feature template. In the template the Profile ID field is set to 0 and the tunnel interface is enabled. Starting from Releases 16.3.1 and later, the Technology field has been added, and the default value is "lte". To match the vEdge router's ZTP cellular0 configuration, change the value to "auto".

Click Advanced, to view the default cellular MTU configuration is 1428 bytes:

The following guidelines help to troubleshoot issues that can occur when using ZTP from a wireless router:

• For ZTP to work correctly, ensure that you are using the correct SIM with the correct modem model (SKU).

• If the default profile APN is not configured correctly, the ZTP process does not work correctly. If ZTP does not work, issue the show cellular status command to display the error. If an error occurs, configure the appropriate APN and retry the ZTP process.

• For SKUs that do not have default profile APN configurations, such as Generic (MC7304) and North America (MC7354) SKUs, if the automatic profile selection does not detect the APN on the SIM card, configure the profile, including an APN. If the router has a second circuit that has access to Cisco SD-WAN Manager, add the profile information, including the APN, to the feature configuration template and then push the device template to the cellular router. Otherwise, configure the profile on the cellular router from the CLI, including an APN.

• To check whether the router is unable to detect the SIM card, issue the show cellular status command. Check for the SIM Read error. To correct this problem, insert the SIM card correctly in the router.

• In Release 16.3.0, after you run ZTP on a cellular router, the cellular interface is in a no shutdown state Because of this, Cisco SD-WAN Manager is unable to push a device configuration template to the router. To correct this problem, from the CLI on the router, configure the cellular interface state to be in shutdown state.