Configuring Infra for Cisco Cloud Network Controller Sites

Refreshing Cloud Site Connectivity Information

Any infrastructure changes, such as CSR and Region addition or removal, require a Multi-Site fabric connectivity site refresh. This section describes how to pull up-to-date connectivity information directly from each site's APIC.

Procedure


Step 1

Log in to the Cisco Nexus Dashboard Orchestrator GUI.

Step 2

In the left navigation menu, select Infrastructure > Site Connectivity.

Step 3

In the top right of the main pane, click Configure.

Step 4

In the left pane, under Sites, select a specific site.

Step 5

In the main window, click the Refresh button to discover any new or changed CSRs and regions.

Step 6

Finally, click Yes to confirm and load the connectivity information.

This will discover any new or removed CSRs and regions.

Step 7

Click Deploy to propagate the cloud site changes to other sites that have connectivity to it.

After you refresh a cloud site's connectivity and CSRs or regions are added or removed, you need to deploy infra configuration so other sites that have underlay connectivity to that cloud site get updated configuration.


Configuring Infra: Cloud Site Settings

This section describes how to configure site-specific Infra settings for Cloud Network Controller sites.

Procedure


Step 1

Log in to the Cisco Nexus Dashboard Orchestrator GUI.

Step 2

In the left navigation menu, select Infrastructure > Site Connectivity.

Step 3

In the top right of the main pane, click Configure.

Step 4

In the left pane, under Sites, select a specific cloud site.

Step 5

Provide the general Inter-Site Connectivity information.

  1. In the right <Site> Settings pane, select the Inter-Site Connectivity tab.

  2. Enable the Multi-Site knob.

    This defines whether the overlay connectivity is established between this site and other sites.

    Note that the overlay configuration will not be pushed to sites which do not have the underlay intersite connectivity established as desrcibed in the next step.

  3. (Optional) Specify the BGP Password.

Step 6

Provide site-specific Inter-Site Connectivity information.

  1. In the right properties sidebar for the cloud site, click Add Site.

    The Add Site window opens.

  2. Under Connected to Site, click Select a Site and select the site (for example, Site2) to which you want to establish connectivity from the site you are configuring (for example, Site1) .

    Once you select the remote site, the Add Site window will update to reflect both directions of connectivity: Site1 > Site2 and Site2 > Site1.

  3. In the Site1 > Site2 area, from the Connection Type dropdown, choose the type of connection between the sites.

    The following options are available:

    • Public Internet—connectivity between the two sites is established via the Internet.

      This type is supported between any two cloud sites or between a cloud site and an on-premises site.

    • Private Connection—connectivity is established using a private connection between the two sites.

      This type is supported between a cloud site and an on-premises site.

    • Cloud Backbone—connectivity is established using cloud backbone.

      This type is supported between two cloud sites of the same type, such as Azure-to-Azure or AWS-to-AWS.

    If you have multiple types of sites (on-premises, AWS, and Azure), different pairs of site can use different connection type.

  4. Choose the Protocol that you want to use for connectivity between these two sites.

    If using BGP-EVPN connectivity, you can optionally enable IPSec and choose which version of the Internet Key Exchange (IKE) protocol to use: IKEv1 (Version 1) or IKEv2 (Version 1) depending on your configuration.

    • For Public Internet connectivity, IPsec is always enabled.

    • For Cloud Backbone connectivity, IPsec is always disabled.

    • For Private Connection, you can choose to enable or disable IPsec.

    If using BGP-IPv4 connectivity instead, you must provide an external VRF which will be used for route leaking configuration from the cloud site you are configuring.

    After Site1 > Site2 connectivity information is provided, the Site2 > Site1 area will reflect the connectivity information in the opposite direction.

  5. Click Save to save the inter-site connectivity configuration.

    When you save connectivity information from Site1 to Site2, the reverse connectivity is automatically created from Site2 to Site1, which you can see by selecting the other site and checking the Inter-site Connectivity information in the right sidebar.

  6. Repeat this step to add inter-site connectivity for other sites.

    When you establish underlay connectivity from Site1 to Site2, the reverse connectivity is done automatically for you.

    However, if you also want to establish inter-site connectivity from Site1 to Site3, you must repeat this step for that site as well.

Step 7

Provide External Connectivity information.

If you do not plan to configure connectivity to external sites or devices that are not managed by NDO, you can skip this step.

Detailed description of an external connectivity use case is available in the Configuring External Connectivity from Cloud CSRs Using Nexus Dashboard Orchestrator document.

  1. In the right <Site> Settings pane, select the External Connectivity tab.

  2. Click Add External Connection.

    The Add External Connectivity dialog will open.

  3. From the VRF dropdown, select the VRF you want to use for external connectivity.

    This is the VRF which will be used to leak the cloud routes. The Regions section will display the cloud regions that contain the CSRs to which this configuration be applied.

  4. From the Name dropdown in the External Devices section, select the external device.

    This is the external device you added in the General Settings > External Devices list during general infra configuration and must already be defined as described in Configuring Infra: General Settings.

  5. From the Tunnel IKE Version dropdown, pick the IKE version that will be used to establish the IPSec tunnel between the cloud site's CSRs and the external device.

  6. (Optional) From the Tunnel Subnet Pool dropdown, choose one of the named subnet pools.

    Named subnet pool are used to allocate IP addresses for IPSec tunnels between cloud site CSRs and external devices. If you do not provide any named subnet pools here, the external subnet pool will be used for IP allocation.

    Providing a dedicated subnet pool for external device connectivity is useful for cases where a specific subnet is already being used to allocate IP addresses to the external router and you want to continue to use those subnets for IPSec tunnels for NDO and cloud sites.

    If you want to provide a specific subnet pool for this connectivity, it must already be created as described in Configuring Infra: General Settings.

  7. (Optional) In the Pre-Shared Key field, provide the custom keys you want to use to establish the tunnel.

  8. If necessary, repeat the previous substeps for any additional external devices you want to add for the same external connection (same VRF).

  9. If necessary, repeat this step for any additional external connections (different VRFs).

    Note that there's a one-to-one relationship for tunnel endpoints between CSRs and external devices, so while you can create additional external connectivity using different VRFs, you cannot create additional connectivity to the same external devices.


What to do next

While you have configured all the required inter-site connectivity information, it has not been pushed to the sites yet. You need to deploy the configuration as described in Deploying Infra Configuration

Recovering from Cloud Network Controller Site Downtime

When Cloud Network Controller (formerly Cloud APIC) instance/VM goes down for any reason while still being managed by NDO, you may be unable to undeploy or delete any existing templates associated with that cloud site. In this case, attempting to forcefully unmanage the site in NDO can cause stale configuration and deployment errors even if the site recovers.

To recover from this:

Procedure


Step 1

Bring up the new Cloud Network Controller sites and re-register the cloud sites.

  1. Log in to ND.

  2. Open the admin console.

  3. Navigate to the Sites page.

  4. From the Actions menu next to the site you redeployed, choose Edit Site.

  5. Check the "Re-register site" checkbox.

  6. Provide the new site details.

    You need to provide the new public IP address of site and login credentials.

  7. Click Save to re-register the site.

Once the connectivity status of the site shows UP, the site IPs in NDO are also updated and the new sites will be in 'managed' state.

Step 2

Undeploy the previously deployed templates for each schema.

  1. Log in to NDO.

  2. Navigate to Application Management and select Schemas.

  3. Click on a schema with the deployed templates.

  4. From the Actions menu next to the Template Properties, choose Undeploy Template and wait until the template is successfully undeployed.

Step 3

Refresh the site's infra configuration to ensure that the new Catalyst 8000V switches are added in NDO.

  1. Navigate to Infrastructure and select Site Connectivity.

  2. Click Configure at the top right of the screen.

  3. Select the cloud site under the Sites panel and click Refresh .

  4. Click Deploy on the top right of the screen and wait until all sites are successfully deployed.

Step 4

Redeploy all templates associated with this Cloud Network Controller site.

  1. Navigate to Application Management and select Schemas.

  2. Click on a schema with the templates undeployed earlier.

  3. Click Deploy to Sites and wait until the template is deployed.