Cisco Crosswork Network Controller 2.0 Release Notes

Product Overview

Cisco Crosswork Network Controller empowers customers to simplify and automate intent-based network service provisioning, monitoring and optimization in a multi-vendor network environment with a common GUI and API.

The solution combines intent-based network automation to deliver critical capabilities for service orchestration and fulfilment, network optimization, service path computation, device deployment and management, and anomaly detection and automatic remediation. Using telemetry gathering and automated responses, Cisco Crosswork Network Controller delivers network optimization capabilities that would be nearly impossible to replicate even with a highly skilled and dedicated staff operating the network.

The fully integrated solution combines core capabilities from multiple innovative, industry-leading products including Cisco Network Services Orchestrator (NSO), Cisco Segment Routing Path Computation Element (SR-PCE), Cisco Crosswork Data Gateway, and Cisco Crosswork's infrastructure and suite of applications. Its unified user interface allows real-time visualization of the network topology and services, as well as service and transport provisioning, via a single pane of glass.

Primary Use Cases:

Orchestrated service provisioning: Provisioning of L2VPN and L3VPN services with underlay transport policies in order to define, meet, and maintain SLAs, using the UI or APIs.
Real-time network and bandwidth optimization:Intent-based closed-loop automation, congestion mitigation and dynamic bandwidth management based on Segment Routing and RSVP-TE. Optimization of bandwidth resource utilization by setting utilization thresholds on links and calculating tactical alternate paths when thresholds are exceeded. Real-time telemetry is used to detect changes in network traffic and then changes in the network are automatically implemented to deliver on the operator's intent.
Local Congestion Management: Local, interface-level congestion mitigation recommendations for rerouting traffic around the congested interface using tactical TE policies.
Visualization of network and service topology and inventory: Visibility into device and service inventory and visualization of devices, links, and transport/VPN services and their status on logical or geographical maps.
Performance-based closed-loop automation:Automated discovery and remediation of problems in the network by allowing Key Performance Indicator (KPI) customization and monitoring and triggering of pre-defined remediation tasks when a KPI threshold is breached. Cisco Crosswork Health Insights and Change Automation (optional add-ons) must be installed for this use case.
Planning, scheduling, and automating network maintenance tasks: Scheduling an appropriate maintenance window for a maintenance task after evaluating the potential impact of the down-time on the newtork (using WAE Design). Automating the execution of maintenance tasks (such as throughput checks, software upgrades, SMU installs) using playbooks. Cisco Crosswork Health Insights and Change Automation (optional add-ons) must be installed for this use case.
Secure zero-touch onboarding and provisioning of devices: Automatic onboarding of new IOS-XR devices and provisioning of Day0 configuration, resulting in faster deployment of new hardware at a lower operating cost. Cisco Crosswork Zero Touch Provisioning (optional add-on) must be installed for this use case.

Solution Components

The Cisco Crosswork Network Controllersolution is made up of the following components:

Table 1.
Component	Version	Description
Cisco Crosswork Infrastructure	4.0	A resilient and scalable platform on which all of the Cisco Crosswork applications can be deployed. The infrastructure is based on a cluster architecture for extensibility, scalability, and high availability.
Cisco Crosswork Data Gateway (CDG)	2.0	A secure, common collection platform for gathering network data from multi-vendor devices that supports multiple data collection protocols including MDT, SNMP, CLI, standards-based gNMI (dial-in), and syslog.
Cisco Crosswork Active Topology	2.0	Provides a unified user interface for device and service inventory, topology visualization, and service provisioning.
Cisco Crosswork Optimization Engine	2.0	Provides closed-loop tracking of the network state and real-time network optimization in response to changes in network state, allowing operators to effectively maximize network capacity utilization, as well as increase service velocity.
Cisco Network Services Orchestrator	5.4.2 or 5.4.4.1	An orchestration platform that makes use of pluggable function packs to translate network-wide service intent into device-specific configuration. Cisco NSO provides flexible service orchestration and lifecycle management across physical network elements and cloud-based virtual network functions (VNFs), fulfilling the role of the Network Orchestrator (NFVO) within the ETSI architecture. It provides complete support for physical and virtual network elements, with a consistent operational model across both. It can orchestrate across multi-vendor environments and support multiple technology stacks, enabling extension of end-to-end automation to virtually any use case or device.
Cisco Segment Routing Path Computation Element (SR-PCE)	7.3.1	An IOS-XR multi-domain stateful PCE supporting both segment routing (SR) and Resource Reservation Protocol (RSVP). Cisco SR-PCE builds on the native Path Computation Engine (PCE) abilities within IOS-XR devices, and provides the ability to collect topology and segment routing IDs through BGP-LS, calculate paths that adhere to service SLAs, and program them into the source router as an ordered list of segments.
Cisco Crosswork Health Insights (optional add-on)	4.0	A network health application that performs real-time Key Performance Indicator (KPI) monitoring, alerting, and troubleshooting. It builds dynamic detection and analytics modules that allow operators to monitor and alert on network events based on user-defined logic.
Cisco Crosswork Change Automation (optional add-on)	4.0	Automates the process of deploying changes to the network.
Cisco Crosswork Zero-Touch Provisioning (optional add-on)	2.0	Automatic onboarding of new IOS-XR devices and provisioning of Day0 configuration, resulting in faster deployment of new hardware at a lower operating cost.

What's New in Cisco Crosswork Network Controller 2.0

The table below lists the primary new features and functionality introduced in Cisco Crosswork Network Controller 2.0.

Table 2. New Features and Functionality in Cisco Crosswork Network Controller 2.0
Feature	What's New?
Scale	To support large scale deployment, the applications that make up Cisco Crosswork Network Controller (Cisco Crosswork Optimization, Cisco Crosswork Active Topology, and other applications) are built with workload and endpoint load balancing using the Cisco Crosswork infrastructure's cluster architecture. Overall scale support has increased significantly to 10K devices, 100K IGP interfaces, 30K transport policies(SR, RSVP), and 200K VPN services (L2VPN, L3VPN).
High availability	Kubernetes-based cluster architecture for extensibility, scalability, and high availability. The cluster can include up to three “hybrid” nodes and three additional “worker” nodes. Cisco Crosswork applications are installed on top of the integrated cluster infrastructure. High availability is now built into Cisco Crosswork’s cluster architecture. Users get alarms when nodes in the cluster have issues, and can restart or re-instantiate nodes at will. Load balancing is automatic across the cluster. Crosswork applications now support high availability with multi-instance microservices.
Cisco Crosswork Data Gateway and Data Collection	Multi-vendor streaming telemetry data collection using gNMI. Syslog-based events collection from network devices using RFC5424 and RFC3164. Auto-enrollment of Cisco Crosswork Data Gateway with the Cisco Crosswork infrastructure. Manual enrollment is no longer required. High availability Data gateway sharing across applications. Secure communication with data destination See the Cisco CDG 2.0 release notes for additional new Cisco CDG features.
User Interface	Enhanced, unified, integrated user interface and topology that combines all components within a single pane of glass. The dashboard in the Home page provides an at-a-glance operational summary of the network being managed, including reachability and operational status of devices, as well as transport policies and VPN services. Additional dashlets might be shown in the dashboard depending on which Cisco Crosswork applications are installed.
Device Management	Integration of Cisco Crosswork Zero Touch Provisioning (ZTP) enables onboarding and provisioning new IOS-XR devices automatically, resulting in faster deployment of new hardware at a lower operating cost. Operators can quickly and easily bring up devices using a Cisco-certified software image and a day-zero software configuration. Managed devices are now listed in a table alongside the topology map. The table contains basic information about the devices. Devices can be selected in the table to zoom in on them on the map.
Optimization/Congestion Mitigation	Support for Local Congestion Mitigation (LCM). Instead of optimizing for bandwidth resources in the network by rerouting traffic in the entire network (end-to-end path optimization), LCM checks the capacity in and around the congested area at an interface level and reroutes traffic between the endpoints of the congested interface (local interface-level optimization). LCM has a “human in the loop” aspect where the control of making changes in the network is in the hands of the operator. Ability to download a plan file using the Cisco Crosswork Optimization Engine API. This is essentially a topology snapshot which captures and represents the topology state at a given point in time, including the IGP topology as well as interface level statistics (traffic load). This plan file can be used for analysis of various "what-if" scenarios. For example, using Cisco WAE Design, the plan file can be used for impact analysis in a maintenance window scenario. Multi-pair SR-PCE support. See the Cisco Crosswork Optimization Engine 2.0 release notes for additional new features.
Transport/VPN Provisioning	Support for provisioning of TE tunnels and RSVP-TE configuration on headend and/or tailend devices using a subset of the standard IETF Yang models that define the configuration model for TE tunnel configuration. Support for RSVP-TE tunnel association with L2VPN P2P (t-ldp) VPN services. Support for provisioning of L2VPN and L3VPN services using a subset of the draft standard IETF L2NM/L3NM YANG models that define the network configuration model for L2VPN and L3VPN. Addition of "dry-run" functionality that allows preview of the configuration before it is actually written to the devices so that changes can be made if necessary. When editing an existing policy/service, the "dry-run" preview also shows the difference between the current and the new configuration. Ability to use commit flags when provisioning, which enables more control within a brownfield environment. Ability to download predefined service intent templates, customize the configuration parameters, and re-import the template in order to provision a policy/service. This provides an alternative to filling in every field in the provisioning GUI.
Topology	Ability to save a useful map display and layout as a named custom view so that it can be retrieved easily without having to rearrange the map each time. Device grouping functionality for easier device management. Ability to organize devices in groups, to create a hierarchy of groups and to visualize groups of devices on the topology map.
Datalytics	Integration of Cisco Crosswork Health Insights and Change Automation enables closed-loop control based on performance telemetry.
Documentation	The new Cisco Crosswork Infrastructure 4.0 and Applications Installation Guide covers installation of the cluster and installation of Crosswork applications on top of the infrastructure. There is no longer an individual installation guide for Cisco Crosswork Network Controller. The new Cisco Crosswork Infrastructure 4.0 and Applications Administration Guide covers setup and maintenance of the Crosswork system. There is no longer a Getting Started Guide for Cisco Crosswork Network Controller. All the information that was originally in the Getting Started Guide is now available in the Administration Guide. The new Cisco Crosswork Network Controller 2.0 Solution Workflow Guide provides an overview of the solution and its supported use cases. It walks users step-by-step through various common usage scenarios to illustrate how users can work with the solution components to achieve the desired benefits. This guide replaces the previous "Manage Transport Services" and "Manage VPN Services" documents.

Compatibility Information

Table 3. Supported Device Software
Operating System	Version	SR-PCE	PCE-Init	PCC-Init	NSO + CFP CLI	NSO + CFP NETCONF	Crosswork Infrastructure 4.0	Crosswork Optimization Engine	Crosswork ZTP
IOS-XR	6.5.3				Yes		Yes	Yes
	6.6.3	Yes	Yes	Yes	Yes		Yes	Yes
	7.0.2		Yes	Yes	Yes		Yes	Yes
	7.1.2	Yes	Yes	Yes	Yes		Yes	Yes
	7.2.1		Yes		Yes		Yes	Yes
	7.3.1	Yes	Yes (Cisco ASR 9000 Series only)	Yes	Yes	Yes	Yes	Yes	Yes
IOS-XE	17.4.1			Yes	Yes		Yes	Yes

Important Notes

Take into consideration the following important information before starting to use Cisco Crosswork Network Controller 2.0:

VPN Service Provisioning:

The Cisco NSO sample function packs are provided as a starting point for VPN service and RSVP-TE provisioning functionality in Cisco Crosswork Network Controller. While the samples can be used “as is” in some limited network configurations, they are intended to demonstrate the extensible design of Cisco Crosswork Network Controller. Answers to common questions can be found here and Cisco Customer Experience representatives can provide answers to general questions about the samples. Support for customization of the samples for your specific use cases can be arranged through your Cisco account team.

Known Issues and Limitations

The table below shows known issues and limitations that should be taken into account before starting to work with Cisco Crosswork Network Controller 2.0.

Table 4. Known Issues and Limitations
Issue/Limitation	Context within Cisco Crosswork Network Controller
Custom templates cannot be created using the GUI, nor can their contents be visualized in the GUI. Custom templates created offline can be applied to service models via GUI and API. However, topology map overlays and service configuration views will not display custom template configuration.	Provisioning GUI.
The Optimization Engine GUI shows TE metric type instead of Latency metric type for SR policies created from the Optimization Engine GUI with Latency as the metric type.	SR policy provisioning from Optimization Engine GUI
Cisco Crosswork Data Gateway operational state may transition to error state when there is little or no traffic for an extended period of time. Operational state will be updated when the traffic returns to normal levels.	Admin > Data Gateway Management
The error, "Get Dense Table Operation" may be shown in the Collection Job UI for the SNMP collection type when a large number of devices (300+) are reloaded in an environment. SNMP collection can be resumed by rebooting the VM from the Troubleshooting menu in Cisco Crosswork Data Gateway.	Cisco Crosswork Data Gateway
Services can be provisioned to devices when devices are not mapped to Cisco Crosswork Network Controller or are operationally down, provided they are reachable and in sync with NSO.	Provisioning GUI
NSO actions such as check-sync, sync-from, re-deploy, reconcile, etc., are not available through the Cisco Crosswork Network Controller provisioning UI.	Provisioning GUI
After a Cisco NSO backup and restore operation, Cisco Crosswork Network Controller discovers all services from Cisco NSO. Any delta in services after the NSO backup operation will be lost once the backup is restored.	Cisco NSO
Cisco Crosswork Network Controller can discover services through transit nodes (SR policy, etc.) for non-Cisco vendor devices. These devices will be in Unmanaged state and services cannot be provisioned on these unmanaged devices.	Provisioning GUI
Multiple users performing CRUD operations simultaneously through the Provisioning GUI may encounter failures when one of the sessions is performing bulk operations (e.g., edit route-policy on 100+ devices). NSO configures relevant changes on the network devices and may not respond to subsequent requests in an adequate timeframe, leading to a timeout.	Provisioning GUI
A device that is also an SR-PCE provider might become unreachable when the device alone is deleted from the Device Management page. To avoid this, add SR-PCE as a provider with a /32 mask.	Device Management, SR-PCE Provider
Segment hops are not visible on the map following multiple add device, delete device, and re-add device operations. Workaround is to restart Optimization Engine from Administration > Crosswork Manager.	Device Management, Optimization Engine GUI

Cisco Crosswork Network Controller 2.0 Documentation

The following documents are provided for Cisco Crosswork Network Controller 2.0. For links to related documentation that you might find useful, see Additional Related Documentation.

Table 5. Cisco Crosswork Network Controller 2.0 Documentation
Document	What is Included
Cisco Crosswork Network Controller 2.0 Release Notes	This document
Cisco Crosswork Infrastructure 4.0 and Applications Installation Guide	Shared installation guide for all the Cisco Crosswork applications and their common infrastructure. Covers: System requirements Installation prerequisites Installation instructions Upgrade instructions
Cisco Crosswork Infrastructure 4.0 and Applications Administration Guide	Shared administration guide for all the Cisco Crosswork applications and their common infrastructure. Covers: Managing clusters and data gateway Data collection High availability Backup and restore Onboard and manage devices Zero touch provisioning Set up maps Managing users, access and security Maintain system health
Cisco Crosswork Network Controller 2.0 Solution Workflow Guide	Solution overview Supported use cases and their benefits. Procedures for achieving the desired outcome for real-life usage scenarios using the Cisco Crosswork Network Controller UI.
Open Source Used in Cisco Crosswork Network Controller 2.0	Lists of licenses and notices for open source software used in Cisco Crosswork Network Controller 2.0.
API Documentation	Advanced users can extend the Cisco Crosswork functionality using the APIs. API documentation is available on Cisco Devnet.

Additional Related Documentation

This section provides links to documentation for products related to Cisco Crosswork Network Controller:

Cisco Crosswork Optimization Engine 2.0:
- User Guide
- Release Notes
Cisco Crosswork Change Automation and Health Insights 4.0:
- User Guide
- Release Notes
Cisco Crosswork Data Gateway 2.0
- Release Notes
- Detailed information about Cisco Crosswork Data Gateway is available in the Cisco Crosswork Infrastructure 4.0 and Applications Installation Guide and the Cisco Crosswork Infrastructure 4.0 and Applications Administration Guide.
Cisco Network Services Orchestrator 5.4.2 or 5.4.4.1
- Documentation for Cisco NSO 5.4.2 can be downloaded here.
- Documentation for Cisco NSO 5.4.4.1 can be downloaded here.
- Additional information about Cisco NSO can be found here.
Function packs:

You can access documentation for all Cisco Crosswork products at https://www.cisco.com/c/en/us/support/cloud-systems-management/crosswork-network-automation/tsd-products-support-series-home.html

Open Bugs in Cisco Crosswork

If you encounter problems while working with Cisco Crosswork, please check this list of open bugs. Each bug ID in the list links to a more detailed description and workaround.

You can use the Cisco Bug Search Tool to search for a specific bug or to search for all bugs in a release.

Go to the Cisco Bug Search Tool.

Enter your registered Cisco.com username and password, and click Log In.

The Bug Search page opens.

Note

If you do not have a Cisco.com username and password, you can register here.

Use any of these options to search for bugs, and then press Enter (Return) to initiate the search:

— To search for a specific bug, enter the bug ID in the Search For field.

— To search for bugs based on specific criteria, enter search criteria, such as a problem description, a feature, or a product name, in the Search For field.

— To search for bugs based on products, enter or choose the product from the Product list.

— To search for bugs based on releases, in the Releases list choose whether to search for bugs affecting a specific release, bugs that were fixed in a specific release, or both. Then enter one or more release numbers in the Releases field.
When the search results are displayed, use the filter tools to narrow down the results. You can filter the bugs by status, severity, and so on.

Tip

To export the results to a spreadsheet, click Export Results to Excel.

Cisco Crosswork Infrastructure 4.0 Release Notes

Cisco Crosswork Infrastructure is a microservices-based platform that brings together streaming telemetry and model-driven application programming interfaces (APIs) to redefine service provider network operations. It employs a cluster architecture to be extensible, scalable, and highly available.

Note

Henceforth, Cisco Crosswork Infrastructure is referred to as "Cisco Crosswork" in this document.

For installation, configuration and administration procedures, see the Cisco Crosswork Infrastructure 4.0 and Applications Installation Guide and the Cisco Crosswork Infrastructure 4.0 and Applications Administration Guide.

Infrastructure Enhancements

Starting with the 4.0 release, only the infrastructure components will be installed on the target machines. Each Cisco Crosswork application will be available as a separate installable package.
Node-level High Availability (HA) is provided for the Cisco Crosswork applications.
Ability to scale the Cisco Crosswork applications as per your business requirements.
Added license compliance check for non-Crosswork active collection jobs.
Support provided for an Alert Framework.
New certificate management architecture to support secure communication between Cisco Crosswork Data Gateway and devices. Includes device TLS certificate management from the Cisco Crosswork UI.
Support for secure GNMI telemetry.
Added extensibility framework that supports:
- Installing the Cisco Crosswork infrastructure without any applications.
- Dynamic integration of Cisco Crosswork applications after day 0.
- Managing (install, activate, upgrade, and uninstall) a purchased application in Cisco Crosswork.
Cisco Crosswork deployment is supported on Cisco CSP 5K devices.
Support for applications to send standard syslog events from Cisco Crosswork to external syslog servers.
Support for display of system alarms and events for troubleshooting scenarios.

Infrastructure Known Issues and Limitations

UI

Sometimes, NETCONF reachability times out for IOS XE devices. To recover, try increasing the NETCONF reachability timer to a higher timeout value (for example, 120 seconds).
While retrieving device inventory via API from Cisco Crosswork, use page size of 200.
In rare cases, after the successful registration, the License Authorization Status in the Smart Licensing page is not changed and will continue to display as being in EVALUATION mode. As a consequence, the evaluation timer will be started and incorrect messages will be displayed to the user. As a workaround, please de-register and register the product again.
If you restart microservices for a Crosswork application, the microservice may appear removed upon restart, but the application will continue to show a healthy status.

Alerting

Alarms, faults, errors, or any status indications for Cisco Crosswork Data Gateway will not be reflected on the VM node or its operational state.
Alerting service can become unresponsive during stress testing. Alerts related to Crosswork applications may not be generated during this time. If this happens, Cisco Crosswork will recover the alerting by automatically restarting the service.
If the node containing the Cisco Crosswork orchestrator is restarted, it might take up to 10 minutes before the health of the cluster can be viewed.

Topology

L2 links are discovered utilizing either point-to-point Cisco Discovery Protocol (CDP) or Link Layer Discovery Protocol (LLDP).
PCE is required for L3 link topology mapping.
Enable traps on routers to receive L2 link down and up status changes quickly. Otherwise, it may take one SNMP poll cadence (default is 5 minutes) to see the L2 link status change.

High Availability

Cisco Crosswork will not allow you to power off two hybrid nodes at the same time. If a system loses a hybrid node due to any faults, it must be replaced as soon as possible.