This document describes the features, issues, and deployment guidelines for Cisco Multi-Site Orchestrator software.

Cisco Multi-Site is an architecture that allows you to interconnect separate Cisco APIC, Cloud APIC, and DCNM domains (fabrics) each representing a different region. This helps ensure multitenant Layer 2 and Layer 3 network connectivity across sites and extends the policy domain end-to-end across the entire system.

Cisco Multi-Site Orchestrator is the intersite policy manager. It provides single-pane management that enables you to monitor the health of all the interconnected sites. It also allows you to centrally define the intersite configurations and policies that can then be pushed to the different Cisco APIC, Cloud APIC, or DCNM fabrics, which in term deploy them in those fabrics. This provides a high degree of control over when and where to deploy the configurations.

For more information, see the “Related Content” section of this document.

Note: The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

Date

Description

November 20, 2021

Additional open issue CSCvy97158.

August 16, 2021

Additional open issues CSCvy94170, CSCvy99012, CSCvy61486.

August 9, 2021

Additional open issues CSCvy98518, CSCvy63967, CSCvy95575.

December 23, 2020

Release 3.2(1f) became available

New Software Features

This release adds the following new features:

Feature

Description

Multi-Site Orchestrator and Nexus Dashboard Integration

Multi-Site Orchestrator can now be deployed as an application on the Nexus Dashboard platform.

For additional information, see Cisco Multi-Site Deployment Guide.

Multi-Site and DCNM Integration

Cisco Multi-Site now supports intersite connectivity between DCNM fabrics and allows you to configure VRFs and Network for those fabrics.

For additional information, see Cisco Multi-Site Configuration Guide for DCNM Fabrics.

New Hardware Features

There is no new hardware supported in this release.

The complete list of supported hardware is available in the Cisco Multi-Site  Hardware Requirements Guide.

Changes in Behavior

If you are upgrading to this release, you will see the following changes in behavior:

●      For all new deployments, you must install the Multi-Site Orchestrator application in Nexus Dashboard.

Release 3.2(1) supports physical Nexus Dashboard clusters only.

●      If you are upgrading your existing deployment to Release 3.2(1) or later, you must deploy a new Nexus Dashboard cluster and migrate your existing configuration.

The procedure is described in detail in Cisco Multi-Site Deployment Guide.

●      Downgrading to releases prior to Release 3.2(1) is not supported.

If you want to revert to an earlier release, you must deploy a brand-new cluster using that release and restore the older configuration backup.

●      Release 3.2(1) supports managing Cisco APIC and Cisco DCNM sites only.

On-boarding Cisco Cloud APIC sites will be supported in future Multi-Site Orchestrator releases.

●      Site management and on-boarding have moved to a centralized location in the Nexus Dashboard GUI.

When migrating to Release 3.2(1), you will need to on-board the sites using the Nexus Dashboard GUI before restoring existing configuration. The procedure is described in detail in Cisco Multi-Site Deployment Guide.

●      User management and authentication have moved to a centralized location in the Nexus Dashboard GUI.

Existing local users will be transferred to the Nexus Dashboard during configuration import.

For existing remote authentication users, you will need to add the remote authentication server to the Nexus Dashboard as described in the Nexus Dashboard User Guide.

Open Issues

This section lists the open issues. Click the bug ID to access the Bug Search Tool and see additional information about the bug. The "Exists In" column of the table specifies the 3.2(1) releases in which the bug exists. A bug might also exist in releases other than the 3.2(1) releases.

Bug ID

Description

Exists in

CSCvo84218

When service graphs or devices are created on Cloud APIC by using the API and custom names are specified for AbsTermNodeProv and AbsTermNodeCons, a brownfield import to the Multi-Site Orchestrator will fail.

3.2(1f) and later

CSCvo20029

Contract is not created between shadow EPG and on-premises EPG when shared service is configured between Tenants.

3.2(1f) and later

CSCvn98355

Inter-site shared service between VRF instances across different tenants will not work, unless the tenant is stretched explicitly to the cloud site with the correct provider credentials. That is, there will be no implicit tenant stretch by Multi-Site Orchestrator.

3.2(1f) and later

CSCvs99052

Deployment window may show more policies been modified than the actual config changed by the user in the Schema.

3.2(1f) and later

CSCvt06351

Deployment window may not show all the service graph related config values that have been modified.

3.2(1f) and later

CSCvt00663

Deployment window may not show all the cloud related config values that have been modified.

3.2(1f) and later

CSCvt41911

After brownfield import, the BD subnets are present in site local and not in the common template config

3.2(1f) and later

CSCvt44081

In shared services use case, if one VRF has preferred group enabled EPGs and another VRF has vzAny contracts, traffic drop is seen.

3.2(1f) and later

CSCvt47568

Let's say APIC has EPGs with some contract relationships. If this EPG and the relationships are imported into MSO and then the relationship was removed and deployed to APIC, MSO doesn't delete the contract relationship on the APIC.

3.2(1f) and later

CSCvt02480

The REST API call "/api/v1/execute/schema/5e43523f1100007b012b0fcd/template/Template_11?undeploy=all" can fail if the template being deployed has a large object count

3.2(1f) and later

CSCvt15312

Shared service traffic drops from external EPG to EPG in case of EPG provider and L3Out vzAny consumer

3.2(1f) and later

CSCvt11713

Intersite L3Out traffic is impacted because of missing import RT for VPN routes

3.2(1f) and later

CSCvu56069

Site deletion throws error:

"Error from APIC: https://20.185.81.79, error: Invalid Configuration CT_EXTNETWORK_REQUIRES_EXTSUBNETPOOL: At present, there must be at least one cloudtemplateExtSubnetPool in cloudtemplateInfraNetwork uni/tn-infra/infranetwork-default when cloudtemplateExtNetwork is present; current count = 0 FORCE DELETE SITE"

3.2(1f) and later

CSCvv00462

Unable to add the APIC site with different site ID if it was previously removed MSO.

3.2(1f) and later

CSCvv67993

MSO will not update or delete VRF vzAny configuration which was directly created on APIC even though the VRF is managed by MSO.

3.2(1f) and later

CSCvw31631

When deploying fabric connectivity between on-premises and cloud sites, you may get a validation error stating that l3extSubnet/cloudTemplateBgpEvpn is already attached.

3.2(1f) and later

CSCvv95445

If you are logged into Application Services Engine 1.1.3d UI and MSO UI in different browser tabs, the backup import functionality does not work. This is due to different authorization cookie used for SE and MSO API.

3.2(1f) and later

CSCvv75133

In a shared services scenario, stale shadow BD/EPG entries are not cleared on the APIC when Preferred Group and regular contract is removed.

3.2(1f) and later

CSCvw47022

Shadow of cloud VRF may be unexpectedly created or deleted on the on-premises site.

3.2(1f) and later

CSCvw10432

Two cloud sites (with Private IP for CSRs) with the same InfraVNETPool on both sites can be added to MSO without any infraVNETPool validation.

3.2(1f) and later

CSCvw57672

API POST/GET/PUT/DEL requests to MSO will be accepted, but system might return an internal_server_error with code 500 and message as "The token is expired since 2020-11-23T12:41:15Z?.

3.2(1f) and later

CSCvw77151

You may see an error during deployment of the Policy(Vrf/Network) saying "profile does not exists" or unexpected update on the Vrf/Network.

3.2(1f) and later

CSCvw83190

Config drift for BD or VRF after backup restore or upgrade.

3.2(1f) and later

CSCvw85584

User will not be able to deploy the template and error message mentioned in the bug will be shown.

3.2(1f) and later

CSCvw86814

When importing brownfield DCNM sites to be managed by MSO, the Networks may remain in the “pending” state.

3.2(1f) and later

CSCvy02792

Physical domain mapping unexpectedly was removed from multiple EPG

3.2(1f) and later

CSCvy98518

MSO removes L3Out-BD association from sites after deleting even an unrelated L3Out in other templates

3.2(1f) and later

CSCvy63967

Open a schema which has around 800 objects (in this case ~ 400 EPGs and ~ 400 BDs)

Try to create a new EPG, and type the EPG name - takes 10 seconds after typing for the EPG name to show in text box.

3.2(1f) and later

CSCvx88132

Some EPGs not shown in Provider list in DHCP Relay Policy creation UI

3.2(1f) and later

CSCvy95575

AWS site is not shown with correct cloud site type and other details on the Connectivity View of Sites.

3.2(1f) and later

CSCvy94170

MSO able to login with username only for LDAP users without any password.

3.2(1f) and later

CSCvy99012

After migration, deploying a template led to deletion of static ports.

3.2(1f) and later

CSCvy61486

Removing EPG objects created from MSO for one site can unexpectedly remove the application profile on the remote site.

3.2(1f) and later

CSCvy97158

Shadow EPG/BDs are not removed when the contract is removed.

3.2(1f) and later

Resolved Issues

This section lists the resolved issues. Click the bug ID to access the Bug Search tool and see additional information about the issue. The "Fixed In" column of the table specifies whether the bug was resolved in the base release or a patch release.

Bug ID

Description

Fixed in

CSCvt47581

fvImportExtRoutes flag is created for VRF even though site1 & site3 external EPGs have provider contract.

3.2(1f) and later

Known Issues

This section lists known behaviors. Click the Bug ID to access the Bug Search Tool and see additional information about the issue.

Bug ID

Description

CSCvo82001

Unable to download Multi-Site Orchestrator report and debug logs when database and server logs are selected

CSCvo32313

Unicast traffic flow between Remote Leaf Site1 and Remote Leaf in Site2 may be enabled by default. This feature is not officially supported in this release.

CSCvn38255

After downgrading from 2.1(1), preferred group traffic continues to work. You must disable the preferred group feature before downgrading to an earlier release.

CSCvn90706

No validation is available for shared services scenarios

CSCvo59133

The upstream server may time out when enabling audit log streaming

CSCvd59276

For Cisco Multi-Site , Fabric IDs Must be the Same for All Sites, or the Querier IP address Must be Higher on One Site.

The Cisco APIC fabric querier functions have a distributed architecture, where each leaf switch acts as a querier, and packets are flooded. A copy is also replicated to the fabric port. There is an Access Control List (ACL) configured on each TOR to drop this query packet coming from the fabric port. If the source MAC address is the fabric MAC address, unique per fabric, then the MAC address is derived from the fabric-id. The fabric ID is configured by users during initial bring up of a pod site.

In the Cisco Multi-Site  Stretched BD with Layer 2 Broadcast Extension use case, the query packets from each TOR get to the other sites and should be dropped. If the fabric-id is configured differently on the sites, it is not possible to drop them.

To avoid this, configure the fabric IDs the same on each site, or the querier IP address on one of the sites should be higher than on the other sites.

CSCvd61787

STP and "Flood in Encapsulation" Option are not Supported with Cisco Multi-Site.

In Cisco Multi-Site topologies, regardless of whether EPGs are stretched between sites or localized, STP packets do not reach remote sites. Similarly, the "Flood in Encapsulation" option is not supported across sites. In both cases, packets are encapsulated using an FD VNID (fab-encap) of the access VLAN on the ingress TOR. It is a known issue that there is no capability to translate these IDs on the remote sites.

CSCvi61260

If an infra L3Out that is being managed by Cisco Multi-Site is modified locally in a Cisco APIC, Cisco Multi-Site might delete the objects not managed by Cisco Multi-Site  in an L3Out.

CSCvq07769

"Phone Number" field is required in all releases prior to Release 2.2(1). Users with no phone number specified in Release 2.2(1) or later will not be able to log in to the GUI when Orchestrator is downgraded to an earlier release.

CSCvu71584

Routes are not programmed on CSR and the contract config is not pushed to the Cloud site.

Compatibility

This release supports the hardware listed in the Cisco Multi-Site Hardware Requirements Guide.

Multi-Site Orchestrator releases are not dependent on the fabric controller releases. The fabrics and the Multi-Site Orchestrator itself can be upgraded independently of each other and run in mixed operation mode. For more information, see the Interoperability Support section in the  “Infrastructure Management” chapter of the Cisco Multi-Site Deployment Guide.

Release 3.2(1) supports Multi-Site Orchestrator deployments in Cisco Nexus Dashboard only. The VMware ESX (.ova) and Cisco Application Services Engine (.aci) form factors have been deprecated.

Release 3.2(1) supports managing Cisco APIC and Cisco DCNM sites only. On-boarding Cisco Cloud APIC sites will be supported in future releases.

Scalability

For the verified scalability limits for ACI fabrics, see the Cisco ACI Verified Scalability Guides.

Related Content

For DCNM fabrics, see the Cisco Data Center Manager (DCNM) page for a complete list of all Multi-Site documentation for DCNM fabrics.

For ACI fabrics, see the Cisco Application Policy Infrastructure Controller (APIC) page for a complete list of all Multi-Site documentation for ACI fabrics. On that page, you can use the "Choose a topic" and "Choose a document type" fields to narrow down the displayed documentation list and find a desired document.

The documentation includes installation, upgrade, configuration, programming, and troubleshooting guides, technical references, release notes, and knowledge base (KB) articles, and videos. KB articles provide information about a specific use cases or topics. The following tables describe the core Multi-Site documentation.

Document

Description

Cisco Multi-Site Orchestrator Release Notes

This document. Provides release information for the Cisco Multi-Site Orchestrator product.

Cisco Multi-Site Hardware Requirements Guide

Provides the hardware requirements and compatibility.

Cisco Multi-Site Deployment Guide

Describes how to install Cisco Multi-Site Orchestrator and perform day-0 operations.

Cisco Multi-Site Configuration Guide for ACI Fabrics

Describes Cisco Multi-Site configuration options and procedures.

Cisco Multi-Site Configuration Guide for DCNM Fabrics

Describes Cisco Multi-Site configuration options and procedures.

Cisco Multi-Site Verified Scalability

Contains the maximum verified scalability limits for this release of Cisco Multi-Site Orchestrator.

Cisco ACI Verified Scalability

Contains the maximum verified scalability limits for Cisco ACI fabrics.

Cisco DCNM Verified Scalability

Contains the maximum verified scalability limits for Cisco DCNM fabrics.

Cisco ACI YouTube channel

Contains videos that demonstrate how to perform specific tasks in the Cisco Multi-Site.

Documentation Feedback

To provide technical feedback on this document, or to report an error or omission, send your comments to mailto:apic-docfeedback@cisco.com. We appreciate your feedback.

Legal Information

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

© 2020 Cisco Systems, Inc. All rights reserved.