Cisco Crosswork Infrastructure 4.1 and Applications Administration Guide

Overview of Cisco Crosswork Data Gateway

Cisco Crosswork Data Gateway is a secure, common collection platform for gathering network data from multi-vendor devices. It is an on-premise application deployed close to network devices that supports multiple data collection protocols including MDT, SNMP, CLI, gNMI, syslog and NETCONF. The number of Cisco Crosswork Data Gateways you need depends on the number of devices supported, the amount of data being processed, the frequency at which it is collected and your network architecture.

When Cisco Crosswork Data Gateway is deployed with Cisco Crosswork Platform (also referred to as Cisco Crosswork in this guide), Cisco Crosswork acts as the controller application.

Crosswork Data Gateway uses the following concepts:

Crosswork Data Gateway VM - Crosswork Data Gateway VM that you install.
Crosswork Data Gateway Profile -
Cisco Crosswork Data Gateway supports the following profiles for On-Premise deployment. For information on VM requirements for each profile, see Section: Crosswork Data Gateway Requirements in the Cisco Crosswork Infrastructure 4.1 and Applications Installation Guide.
- Standard - for use with all Crosswork applications, except Crosswork Health Insights, and Crosswork Service Health (Automated Assurance).
- Extended - for use with Crosswork Health Insights and Crosswork Service Health (Automated Assurance).
Crosswork Data Gateway Pool - A logical unit of one or more Crosswork Data Gateway VMs with an option to enable high availability. When a Crosswork Data Gateway VM goes down, Cisco Crossswork automatically replaces the VM with a spare VM from the pool to ensure that devices are managed and data collections have minimal disruption.
Crosswork Data Gateway- A Crosswork Data Gateway VM that is assigned a virtual IP address when it is added to a Crosswork Data Gateway pool. Operations such as attaching or detaching devices, creating collection jobs happen on the Crosswork Data Gateway.
Data Destination - Internal or external recipients of data collected by the Crosswork Data Gateway. By default, Cisco Crosswork is defined as a data destination. Other destinations (external users) can be defined using the Cisco Crosswork UI or APIs.
Collection Job - A task that Crosswork Data Gateway has to complete to collect data. Crosswork applications create collection jobs to check device reachability, collect telemetry data needed to determine network and service health. The Cisco Crosswork UI and API allow you to configure collection jobs for non-Crosswork applications.
Custom Software Packages - Files and device model definitions to extend device coverage and support data collection from currently unsupported devices.

Note

This chapter explains only the Cisco Crosswork Data Gateway features that can be accessed via Cisco Crosswork UI. For more information about Cisco Crosswork Data Gateway VM and how to manage it, see Appendix A: Configure Crosswork Data Gateway VM.

Crosswork Data Gateway UI Overview

To open the Cisco Crosswork Data Gateway management view, log in to Cisco Crosswork and choose Administration > Data Gateway Management from the left navigation bar.

The Data Gateway Management page has three tabs:

Data Gateways: Displays details of the virtual Cisco Crosswork Data Gateways in the network. You can attach or detach devices to the Data Gateway from this tab.
Pools: Manage Cisco Crosswork Data Gateway pools.
Virtual Machines: Manage physical Cisco Crosswork Data Gateway VMs.

The following table explains the various fields in the Data Gateway Management page.

Table 1. Cisco Crosswork Data Gateway UI

Field

Description

Operational State

Operational state of the Cisco Crosswork Data Gateway VM.

A Crosswork Data Gateway VM has following operational states:

Unknown:

The Crosswork Data Gateway VM's operational state is unknown as it has enrolled itself with Cisco Crosswork, but hasn't established a session yet.
Degraded:

The Cisco Crosswork Data Gateway VM is reachable but one or more of its components are in a state other than OK.
Not Ready: When Cisco Crosswork Data Gateway has enrolled with Cisco Crosswork but is not ready to receive collection jobs since it is not an Active Data Gateway with an associated south bound virtual IP address
Up: The Cisco Crosswork Data Gateway VM is operational and all individual components are "OK".
Error:

The Cisco Crosswork Data Gateway VM is unreachable or some of its components are in Error state.

Admin State

Administration state of the Cisco Crosswork Data Gateway VM.

Up: The VM is administratively up.
Maintenance: Operations between Cisco Crosswork and the Cisco Crosswork Data Gateway are suspended to perform upgrades or other maintenance activities (for example, uploading certificates).

Virtual Machine Name

Name of the Cisco Crosswork Data Gateway VM.

Clicking the info icon next to the name displays the enrollment details of each VM. This includes details such as, the

Pool name
VM name
VM Type indicating if the profile of the Crosswork Data Gateway is Standard or Extended.
Management IP (eth0) with related MAC address
eth1 IP (north bound/vNIC1) with related MAC address

eth2 (south bound/vNIC2) with only the MAC address

Note

The eth2 IP (south bound IP) is assigned to the Crosswork Data Gateway VM during pool creation. Hence, it will not be displayed as part of enrollment details for each VM.

IPv4 Mgmt.IP Address

Management IPv4 address of the Cisco Crosswork Data Gateway VM.

IPv6 Mgmt.IP Address

Management IPv6 address of the Cisco Crosswork Data Gateway VM.

Role

Shows the role of the Cisco Crosswork Data Gateway VM. It can be either:

Assigned: when Cisco Crosswork Data Gateway VM is assigned to a pool.
Unassigned: when Cisco Crosswork Data Gateway VM is not assigned to any pool.
Spare: when Cisco Crosswork Data Gateway VM is part of a pool but is in standby mode

Cisco Crosswork Data Gateway VMs that have the Role as Unassigned need to be assigned to a Crosswork Data Gateway pool before they can used.

Outage History

Outage history of the Cisco Crosswork Data Gateway VM over a period of 14 days.

State aggregation for a day is done in the order of precedence as Error , Degraded, Up, Unknown and Not Ready.

For example, if the Crosswork Data Gateway VM went Unknown to Degraded to Up, color is displayed as Degraded (orange) for that day as Degraded takes precedence over Up and Unknown.

If the Crosswork Data Gateway was in Error state at any time during that day, the tile is Red. If the Data Gateway was not in Error but in Degraded State anytime of the day, the tile is Orange. If the DG was not in Error or Degraded state and was only Up, then the tile is Green.

Pool Name

Name of the Crosswork Data Gateway pools to which the Crosswork Data Gateway VM has been assigned.

Data Gateway Name

Name of the Cisco Crosswork Data Gateway that is created automatically when you add a Crosswork Data Gateway VM to a pool.

High Availability Status

High availability status of a Crosswork Data Gateway could be either:

Protected: All VMs are UP and there is at least one standby available in the pool.
Not Protected: All standby VMs are DOWN.
Limited Protection: Some stnadby VMs are DOWN, but there is still at least one standby that is UP.
None Planned: No standby VMs were added to the pool during pool creation.

Average Availability

Value indicating the health of the Cisco Crosswork Data Gateway VM. This percentage is calculated as the total time (in milliseconds) a Crosswork Data Gateway was in UP state over the time between start time of first event and end time of last event .

Note

The end time of last event is the current time stamp, so the duration of last event is between its start time and current time stamp.

VM ID

VM ID of the Cisco Crosswork Data Gateway VM.

Attached Device Count

Number of devices attached to the Cisco Crosswork Data Gateway pool.

Unique Identifier

Unique identifier of the Cisco Crosswork Data Gateway VM.

Set Up Crosswork Data Gateway to Collect Data

Crosswork Data Gateway requires you to complete the following setup tasks first, before it can run collection jobs.

Note

This workflow assumes that you have already installed Cisco Crosswork Data Gateway as explained in Cisco Crosswork Infrastructure 4.1 and Applications Installation Guide.

It is sufficient to complete Step 1 to Step 3 in the following table to get Crosswork Data Gateway set up and running with Cisco Crosswork and other Crosswork applications. Step 4 to Step 6 are optional and required only in case you wish to extend the Crosswork Data Gateway's capability to collect and forward data by creating external data destinations and custom collection jobs.

Table 2. Tasks to Complete to Set Up Cisco Crosswork Data Gateway to Collect Data
Task	Follow the steps in...
1. Create Crosswork Data Gateway pools.	Create a Cisco Crosswork Data Gateway Pool
2. Attach devices to Crosswork Data Gateway.	Attach Devices to a Crosswork Data Gateway
3. Verify that the default collection jobs are created and running successfully.	Monitor Collection Jobs
4. (optional) Extend device coverage to collect data from currently unsupported devices or third-party devices.	Manage Custom Software Packages
5. (optional) Forward data to external data destinations.	Create and Manage External Data Destinations
6. (optional) Create custom collection jobs (outside of those built for you by Cisco Crosswork).	Manage Crosswork Data Gateway Collection Jobs

Crosswork Data Gateway High Availability with Pools

A Cisco Crosswork Data Gateway pool ensures that your devices are managed and collections occur with minimal disruption.

A pool can consist of one or more Cisco Crosswork Data Gateway VMs with an option to enable high availability.

If a Cisco Crosswork Data Gateway VM in the pool goes down, Cisco Crosswork automatically replaces that VM with a standby VM from the pool. Devices and any existing collection jobs are assigned automatically from the failed VM to the standby VM. Once the VM that went down becomes operational, it becomes a standby VM in the pool.

Figure 1. Crosswork Data Gateway High Availability

Note

If more than one Crosswork Data Gateway VM in a pool have same Southbound IP address, reboot the standby Crosswork Data Gateway, so that the standby Crosswork Data Gateway VM loses its southbound IP address once it comes up.

For example, CDG1 (Active) with southbound IP address IP1 goes down. Cisco Crosswork replaces CDG1 with CDG2(Standby) as new active and programs the same IP1 as southbound IP on CDG2. CDG1 later comes up and becomes the new standby in the pool, but retains the same IP1 as its southbound IP address. This results in both CDG1 and CDG2 having same IP1 as southbound IPs.

A Crosswork Data Gateway pool has following states:

Protected: All VMs are UP and there is at least one standby VM in the pool.
Not Protected: All the standby VMs are DOWN and there are none available to replace a VM that is in use.
Limited Protection: Some standby VMs are DOWN, but there is still at least one standby that is UP.
None Planned: No standby VMs were added to the pool during pool creation.

Create a Cisco Crosswork Data Gateway Pool

When you create a Cisco Crosswork Data Gateway pool, follow these guidelines:

You must create at least one pool and assign Crosswork Data Gateway VM(s) to it. This step is mandatory to set up the Crosswork Data Gateway for collection.
We recommend creating pools with Cisco Crosswork Data Gateways of similar profiles i.e., either all standard Crosswork Data Gateways or all extended Crosswork Data Gateways in a pool. Create heterogenous pools i.e., pools with different types of Crosswork Data Gateways only for device or job migration.

To create a Crosswork Data Gateway pool:

Before you begin

Before creating a Cisco Crosswork Data Gateway pool:

Ensure that you have installed all Crosswork Data Gateway VMs that you wish to add to the pool.
The Operational State of the Crosswork Data Gateway VMs is Not Ready.
Have network information such as subnet mask and gateway information ready.
Decide if you wish to enable high availability for the pool.

Procedure

Step 1

From the main menu, choose Administration > Data Gateway Management and click Pools tab.

Step 2

In the Pools tab, click Add icon button to create a pool.

Step 3

In the Pool Parameters pane, enter the values for the following parameters:

Field

Description

Pool Name

Name of the pool that suitably describes the network.

Subnet Mask

Subnet mask for each Cisco Crosswork Data Gateway to communicate with the devices.

Gateway

Gateway address for each Cisco Crosswork Data Gateway to communicate with the devices.

Note

This field is not applicable if a Cisco Crosswork Data Gateway VM has fewer than 3 vNICs.

Description

A description of the pool.

Step 4

In the Pool Resources pane, add the following details:

Add a Virtual IP address for every active data gateway needed: A virtual IP address for every active Cisco Crosswork Data Gateway VM.

Note

Enter either IPv4 or IPv6 addresses that are not in use on the network. Combination of IPv4 and IPv6 addresses is not allowed.

Add the number of standby data gateways desired for protection: Entering a value greater than 0 in this field enables high availability for the pool. When an active data gateway goes down, a 'standby' in the pool replaces it to ensure protection.

The number of Crosswork Data Gateway VMs you add to the pool should be equal to the total number of virtual IPs and standby Crosswork Data Gateway VMs. For example, if you have entered 3 virtual IPs and wish to have 2 standby VMs, add 5 Cisco Crosswork Data Gateway VMs to the pool.

Step 5

Add Cisco Crosswork Data Gateway VMs to the pool.

Select VMs from the Unassigned Virtual Machine(s) on the left and click right arrow to move the VMs to the Virtual Machine(s) Added to Pool.

Step 6

Click Save.

After you create a Cisco Crosswork Data Gateway pool, a virtual Crosswork Data Gateway gets created automatically and is visible under Data Gateways tab. Attach devices to this virtual Crosswork Data Gateway to run collection jobs.

Attach Devices to a Crosswork Data Gateway

Follow these guidelines when you attach devices to a Crosswork Data Gateway.

A device can be attached to only one Crosswork Data Gateway.
For optimal performance, we recommend attaching devices to a Crosswork Data Gateway in batches of 300 devices or fewer.
Before attaching devices to an exisiting Crosswork Data Gateway, we recommend that you check the health of the Crosswork Data Gateway. See Monitor Crosswork Data Gateway Health for more information.

Before you begin

Ensure that the Admin state and Operational state of the Crosswork Data Gateway to which you want to attach devices is Up.

Procedure

Step 1	From the main menu, navigate to Administration > Data Gateway Management > Data Gateways.
Step 2	For the Crosswork Data Gateway to which you want to attach devices, in Actions column, click and select Attach Devices. The Attach Devices window opens showing all the devices available for attaching.
Step 3	To attach all the devices, click Attach All Devices. Otherwise, select the devices you want to attach and click Attach Selected Devices.
Step 4	In Confirm - Attach Devices dialog, click Attach.

Verify that your changes are successful by checking the Attached Device Count column in the Data Gateways pane.

Monitor the Crosswork Data Gateway health to ensure that the Crosswork Data Gateway is functioning well with the newly attached devices. See Monitor Crosswork Data Gateway Health.

Manage Crosswork Data Gateway Post-Setup

This section explains various maintenance tasks within the Crosswork Data Gateway.

Monitor Crosswork Data Gateway Health
Crosswork Data Gateway High Availability with Pools
Manage Cisco Crosswork Data Gateway Device Assignments
Maintain Crosswork Data Gateway VMs

Monitor Crosswork Data Gateway Health

The Crosswork Data Gateway details page provides operational and health summary of a Crosswork Data Gateway. It also has details of the health of various containerized services running on the Crosswork Data Gateway. The overall health of Crosswork Data Gateway also depends on the health of each containerized service.

We recommend monitoring the health of the Crosswork Data Gateways in your network periodically to prevent overloading and take corrective actions well in time.

From the Cisco Crosswork Main Menu, navigate to Administration > Data Gateway Management > Data Gateways.
In the Data Gateways pane, click the Crosswork Data Gateway name. The Crosswork Data Gateway details page opens and displays the operational and health summary of the Data Gateway.

Monitor the Crosswork Data Gateway parameters in this page to ensure that Data Gateway is healthy and running as expected.

General Cisco Crosswork Data Gateway Details - Displays general details of the Crosswork Data Gateway including operational state, high availability state, attached device count, and assigned jobs. The Actions option lists the various troubleshooting options that are available from the UI.
History - Shows the outage history chart of the Cisco Crosswork Data Gateway over 14 days including timestamp, outage time, and clear time. Use the options in the top-right corner of the pane to zoom in, zoom out, pan, or download the SVG and PNG of the history chart of a specific time period within the graph.
Events - Displays a list of all Cisco Crosswork Data Gateway transition state changes over the last 14 days. It includes information such as the event details, including operational state changes, role changes, a message indicating the reason for the status change, timestamp, and duration.

Health - Shows the health information of the Cisco Crosswork Data Gateway. The timestamp in the top-right corner is the timestamp when the last health data was collected. If the Crosswork Data Gateway is in an Error state or if the data is stale for any reason, the timestamp label highlights that the data is old.

Important

Few important points to note:

If the CPU Utilization of a Crosswork Data Gateway exceeds 90%, we recommend that you move devices to another Crosswork Data Gateway that has a lower CPU Utilization percentage.
If the CPU Utilization of a Crosswork Data Gateway exceeds 80%, we recommend that you do not create more collection jobs until you have moved devices to another CDG or have added other VMs to the pool.

Service Status - Displays the health information of the individual container services running on the Crosswork Data Gateway and their resource consumption with an option to restart (Action> Restart) an individual service.

Cisco Crosswork reports CPU utilization of a service against maximum of 800% (8vCPUs) for Standard Profiles and 1600% (16vCPUs) for Extended Profiles.

Note

The list of container services differs between Standard Crosswork Data Gateway and Extended Crosswork Data Gateway. Extended Crosswork Data Gateway has more containers installed.

The resource consumption data that is displayed is from docker statistics. These values are higher than the actual resources consumed by the containerized service.

In case of issues, use the Actions > Troubleshooting option to troubleshoot the Crosswork Data Gateway. See Troubleshoot Crosswork Data Gateway for more information.

Manage a Crosswork Data Gateway Pool

Follow the steps to edit or delete a Cisco Crosswork Data Gateway pool. To create a pool, see Create a Cisco Crosswork Data Gateway Pool.

Before you begin

Before you delete a pool, ensure that you detach devices from the pool or move the devices to another pool. You can only delete Crosswork Data Gateway pools that do not have any devices attached.

Procedure

Step 1

From the main menu, choose Administration > Data Gateway Management and click Pools tab.

Step 2

Edit a Crosswork Data Gateway Pool:

Select the pool which you wish to edit from the list of pools that is displayed in this page,
Click button to open Edit High Availability (HA) Pool page.

When you edit a resource pool, you can only change the parameters in the Pool Resources pane. You cannot edit the parameters in the Pool Parameters pane. To make changes to the parameters in the Pool Parameters pane, create a new pool with the desired values and move the Cisco Crosswork Data Gateway VMs to that pool.

In the Pool Resources pane, you can:

Add a Virtual IP address for every active data gateway needed.
Change the number of standby Crosswork Data Gateway VMs.

Add and remove Crosswork Data Gateway VMs from the pool.

Note

A Crosswork Data Gateway VM can be taken out of the pool only if all devices have been unmapped from the Crosswork Data Gateway. After a Crosswork Data Gateway VM is removed from the pool, a standby VM in the same pool becomes its replacement automatically.

Click Save after you have completed making your changes.

Step 3

Delete a Crosswork Data Gateway Pool:

Select the pool you want to delete and click .
Click Delete in the Delete High Availability (HA) Pool window to delete the pool.

Manage Cisco Crosswork Data Gateway Device Assignments

Follow these guidelines when you move or detach devices from a Crosswork Data Gateway.

A device can be attached to only one Crosswork Data Gateway.
When moving devices to a Crosswork Data Gateway in different pool, ensure that the Gateway of the pool is same as the Gateway of the current pool. Moving devices to a Crosswork Data Gateway with mismatching Gateway will result in failed collections.
Detaching a device from Cisco Crosswork Data Gateway deletes all collection jobs corresponding to the device. If you do not want to lose the collection jobs submitted for the device you wish to detach, move the device to another Cisco Data Gateway instead.

Follow the steps below to move or detach devices from a Crosswork Data Gateway pool. To add devices to the pool, see Attach Devices to a Crosswork Data Gateway.

Procedure

Step 1

From the Cisco Crosswork Main Menu, navigate to Administration > Data Gateway Management > Data Gateways.

Step 2

Move Devices:

For the Crosswork Data Gateway from which you want to move devices, under Actions column, click and select Move Devices. The Move Attached Devices window opens showing all the devices available for moving.
From the To this Data Gateway dropdown, select the data gateway to which you want to move the devices.
To move all the devices, click Move All Devices. Otherwise, select the devices you want to move and click Move Selected Devices.
In Confirm - Move Devices window, click Move.

Step 3

Detach Devices:

For the Crosswork Data Gateway from which you want to detach devices, under Actions column, click and select Detach Devices. The Detach Devices window opens showing all attached devices.
To detach all the devices, click Detach All Devices. Otherwise, select the devices you want to detach and click Detach
In Confirm - Detach Devices window, click Detach

Verify that your changes are successful by checking the Attached Device Count under the Data Gateways pane. Click the i icon next to the attached device count to see the list of all devices attached to the selected Crosswork Data Gateway.

Maintain Crosswork Data Gateway VMs

This section explains the maintenance tasks of the Crosswork Data Gateway VM.

Change the Administration State of Cisco Crosswork Data Gateway VM
Delete Cisco Crosswork Data Gateway VM from Cisco Crosswork
Redeploy a Crosswork Data Gateway VM

Change the Administration State of Cisco Crosswork Data Gateway VM

To perform upgrades or other maintenance within the data center is may become necessary to suspend operations between Cisco Crosswork platform and the Cisco Crosswork Data Gateway. This can be done by placing the Cisco Crosswork Data Gateway into Maintenance mode. During downtime, admin can do modifications to Cisco Crosswork Data Gateway, such as updating the certificates, etc.

Note

If the maintenance activities are affecting the communication between Crosswork and Crosswork Data Gateway, the collection is interrupted and resumes when the communication is restored. Similarly if the maintenance activities are affecting the communication between Crosswork Data Gateway and external destinations (Kafka/gRPC), the collection is interuppted and resumes when the communication is restored.

Once changes are done, admin can change the administration state to Up. Once the Crosswork Data Gateway VM is up, Cisco Crosswork resumes sending jobs to it.

Follow the steps below to change the administration state of a Crosswork Data Gateway VM:

Procedure

Step 1

From the main menu, choose Administration > Data Gateway Management > Virtual Machines.

Step 2

For the Cisco Crosswork Data Gateway whose adminstrative state you want to change, click on Edit icon under Actions column.

Step 3

Select the adminstration state to which you want to switch to.

Delete Cisco Crosswork Data Gateway VM from Cisco Crosswork

Follow the steps below to delete a Cisco Crosswork Data Gateway VM from Cisco Crosswork:

Before you begin

It is recommended that you move the attached devices to another data gateway to not lose any jobs corresponding to these devices. If you detach the devices from Cisco Crosswork Data Gateway VM, then the corresponding jobs are deleted.

Procedure

Step 1	From the main menu, choose Administration > Data Gateway Management > Virtual Machines.
Step 2	For the Crosswork Data Gateway that you want to delete, click under Actions column and click Delete.
Step 3	The Cisco Crosswork Data Gateway VM must be in maintenance mode to be deleted. Click Switch & Continue when prompted to switch to Maintenance mode..
Step 4	Check the check box for "I understand the concern associated with deleting the Data Gateways." and click Remove CDG.

Redeploy a Crosswork Data Gateway VM

If a Crosswork Data Gateway VM has gone down and can no longer be used, then delete the old VM and install a new one. For details on how to install a new Crosswork Data Gateway VM, refer to Section: Install Cisco Crosswork Data Gateway in the Cisco Crosswork Infrastructure 4.1 and Applications Installation Guide.

Note

If the Crosswork Data Gateway VM was already enrolled with Cisco Crosswork and you have installed the VM again with the same name, change the Administration State of the Crosswork Data Gateway VM to Maintenance for auto-enrollment to go through.

If a Crosswork Data Gateway VM was already enrolled with Cisco Crosswork and Cisco Crosswork was installed again, re-enroll the existing Crosswork Data Gateway VM with Cisco Crosswork. See Re-enroll Crosswork Data Gateway.

Create and Manage External Data Destinations

Cisco Crosswork allows you to create external data destinations (Kafka or external gRPC) that can be used by collection jobs to deposit data.

It can be accessed by navigating to Administration > Data Gateway Global Settings > Data Destinations pane. You can add a new data destination, update the settings configured for an existing data destination, and delete a data destination.

The table in the Data Destinations pane lists the approved data destinations that can be used by the collection jobs to deposit their data.

Note

The Crosswork_Kafka and cd-astack-pipeline are internal data destinations and cannot be updated or deleted.

The UUID is the Unique identifier for the data destination. Cisco Crosswork automatically generates this ID when an external data destination is created. This is a required parameter for creating collection jobs.

To view details of a data destination, in the Data Destinations pane, click icon next to the data destination name whose details you want to see.

Licensing Requirements for External Collection Jobs

To be able to create collection jobs that can forward data to external data destinations, ensure that you meet the following licensing requirements:

From the main menu, go to Administration > Application Management > Smart License.
Select Crosswork Platform Services in the application field.
Ensure that the status is as follows:
- Registration Status - Registered
  
  Indicates that you have registered with Cisco Smart Software Manager (CSSM) and are authorized to use the reserved licensed features.
- License Authorization Status - Authorized (In Compliance).
  
  Indicates that you have not exceeded the device count in the external collection jobs.
- Under Smart Licensing Usage, CW_EXTERNAL_COLLECT has status as In Compliance.

If you do not register with Cisco Smart Software Manager (CSSM) after the Evaluation period has expired or you have exceeded the device count in external collection jobs (License Authorization Status is Out of Compliance), you will not be able to create external collection jobs. However, you can still view and delete any existing collection jobs.

Add/Edit a Data Destination

Follow the steps below to add a new data destination. You can then use this data destination for data collection. You can also add multiple data destinations.

Note

If you reinstall an already existing external Kafka data destination with the same IP address, then the collectors need to be restarted for changes to take place .
You can secure the communication channel between Cisco Crosswork and the specified data destination i.e., either Crosswork Kafka or external Kafka. Step 6 of the below procedure explain how to do that.

However, enabling security can impact performance.
- If your external data destination requires a TLS connection, keep the public certificate ready or if it requires client authentication, keep the client certificate and key files ready. The client key might be password-encrypted which will need to be configured as part of the data destination provisioning. Currently, Crosswork Data Gateway supports IP-based certificates only.
- Ensure that the certificates are PEM encoded and the key file is in PKCS#8 format when generating them with your Certificate Authority.
Create the Kafka topics prior to submitting the job to Cisco Crosswork. Depending on external Kafka and how topics are managed in that external Kafka, Cisco Crosswork logs may show the exception listed when and if the topic does not exist at the time of dispatching the collected data to that specific external Kafka / topic. This could be either due to the topic is not yet created or topic got deleted prior to the completion of the requested collection job and dispatching the collected data.
```
destinationContext: topicmdt4
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.
```

Before you begin

If you are using an external Kafka server for data collection, ensure the following:

You have configured the following properties on the external Kafka server:

Note

Refer your Kafka documentation for description and usage of these properties as this explanation is out of scope of this document.

num.io.threads = 8
num.network.threads = 3
message.max.bytes= 30000000

You have created Kafka topics that you want to be used for data collection.

Procedure

Step 1

From the main menu, choose Administration > Data Gateway Global Settings.

Step 2

From Data Destinations pane, click Add icon button. The Add Destination page opens.

If you want to edit an existing destination, click Edit icon button to open Edit Destination page and edit parameters.

Note

Updating a data destination causes the Cisco Crosswork Data Gateway using it to re-establish a session with that data destination. Data collection will be paused and resumes once the session is re-established.

Step 3

Enter or modify the values for the following parameters:

Field

Value

Destination Name

Enter a descriptive data destination name. The name can contain a maximum of 128 alphanumeric characters, plus underscores ("_") or hyphens ("-"). No other special characters are allowed.

If you have many data destinations, make the name as informative as possible to be able to distinguish later.

Server Type

From the drop down, select the server type of your data destination (Kafka/gRPC).

Encoding

From the drop down, select the encoding (json/gpbkv).

Compression Type

From the drop down, select the compression type:

Compression types supported for Kafka are snappy, gzip, lz4, zstd, and none)

Note

zstd compression type is supported only for Kafka 2.0 or higher.

Compression types supported for gRPC are snappy, gzip, and deflate.

Maximum Message Size (bytes) (Kafka-only)

Enter the maximum message size in bytes.

Default Value: 100000000 bytes/ 30 MB
Min: 1000000 bytes/1 MB
Max: 100000000 bytes/ 30 MB

Batch Size (bytes) (Kafka-only)

Enter the required batch size in bytes.

Default Value: 6400000 bytes/6.4 MB
Min: 16384 bytes/ 16.38 KB
Max: 6400000 bytes/6.4 MB

Linger (milliseconds) (Kafka-only)

Enter the required linger time in milliseconds.

Default Value: 5000 ms
Min: 0 ms
Max: 5000 ms

For telemetry based collection, it is recommended to use the destination settings of Batch size as 16384 bytes and linger as 500 ms, for optimal results.

Step 4

Select a TCP/IP stack from the Connection Details options. IPv4 and IPv6 are supported.

Step 5

Complete the Connection Details fields as described in the following table. The fields displayed will vary with the connectivity type you chose. The values you enter must match the values configured on the external Kafka or gRPC server.


Connectivity Type	Fields
IPv4	Enter the required IPv4 Address/ Subnet Mask, and Port. You can add multiple IPv4 addresses by clicking + Add Another IPv4 subnet mask ranges from 1 to 32 and port range from 1024 to 65535.
IPv6	Enter the required IPv6 Address/ Subnet Mask, and Port. You can add multiple IPv6 addresses by clicking + Add Another. IPv6 subnet mask ranges from 1 to 128 and port range from 1024 to 65535.

Step 6

(Optional) To connect securely to the data destination, enable the Enable Secure Communication option under Security Details.

Step 7

Click Save.

What to do next

If you have enabled the Enable Secure Communication option, navigate to the Certificate Management page in the Cisco Crosswork UI (Administration > Certificate Management) and add the relevant certificate for the newly added data destination. This step is mandatory to establish a secure communication to the device. See Manage Certificates for more information.

Note

If you do not add the certificate for the data destination after enabling the Enable Secure Communication option, Cisco Crosswork still connects to the destination in non-secure mode for any collection jobs.

Delete a Data Destination

Follow the steps to delete a data destination:

Before you begin

A data destination can only be deleted if it is not associated with any collection job. We recommend to check in the Collection Jobs view to see if any collection jobs are using the data destination.

Procedure

Step 1	From the main menu, choose Administration > Data Gateway Global Settings.
Step 2	Select the Data destination(s) you want to delete and click button.
Step 3	In Delete Data Destination(s) pop up, click Delete to confirm.

Manage Custom Software Packages

You can upload Custom Device Packages to Cisco Crosswork, for example, when required to extend device coverage and collection capabilities to third-party devices. System Device and MIB Packages are bundled in the Crosswork software and are automatically downloaded to the system instances. You cannot modify system device and MIB packages. modifiable.

You can upload three types of custom software packages to Cisco Crosswork:

CLI Device Package: To use CLI-based KPIs to monitor device health for third-party devices. All custom CLI device packages along with their corresponding YANG models should be included in file custom-cli-device-packages.tar.xz. Multiple files are not supported.

Custom MIB Packages: Custom MIBs and device packages can be specific to third-party devices or be used to filter the collected data or format it differently for Cisco devices. These packages can be edited. All custom SNMP MIB packages along with YANG models should be included in file custom-mib-packages.tar.xz. Multiple files are not supported.

Note

Cisco Crosswork Data Gateway enables SNMP polling on third party devices for standard MIBs already included in the system. Proprietary MIBs are required only if the collection request references MIB TABLE names or SCALAR names from a proprietary MIB. However, if the requests are OID-based, then MIBs are not required.

SNMP Device Package: Cisco Crosswork Data Gateway allows you to extend the SNMP coverage by uploading custom SNMP device packages with any additional MIB and YANG descriptions you require.

The Custom software pane can be accessed via Adminstration > Data Gateway Global Settings.

To download a custom software package, click on the Download icon button next to its name in the File Name column.

Add a Custom Software Package

The following is a list of guidelines about uploading software packages to Cisco Crosswork.

You can upload one or more xar file in a single device package tar.gz file.
Cisco Crosswork doesn't allow Custom MIB package files to overwrite the System MIB Package files. It results in a failed upload attempt.
Ensure that the custom software package TAR file has just the device package folders and none of the parent folder or hierarchy of folders as part of the TAR file. If not imported properly, Cisco Crosswork throws exceptions when executing the job with custom device package.
Cisco Crosswork does not validate the files being uploaded other than checking the file extension.

Follow these steps to upload a custom software package:

Before you begin

When uploading new MIBs as a part of Custom MIB Package, ensure that those new MIBs files can be uploaded within collectors along with existing System MIB files i.e., all dependencies in the files are resolved properly.

For information on how to validate custom MIBs and Yangs i.e., to check if they can be uploaded to Cisco Crosswork, see Use Custom MIBs and Yangs on Cisco DevNet.

Procedure

Step 1	From the main menu, choose Administration > Data Gateway Global Settings.
Step 2	In Custom Software pane, click . To update the existing Custom CLI Device Package, click the upload icon next to the File name in the table.
Step 3	In the Add Custom Software pop up, select the type of custom software package you want to import from the Type dropdown.
Step 4	Click in the blank field of File Name to open the file browser window and select the custom software package to import and click Open.
Step 5	Add a description of the custom software package in the Notes field. This is recommended if you have many packages, to be able to distinguish among them.
Step 6	Click Upload.

What to do next

Restart all impacted services to get the latest custom MIB package updates.

Delete a Custom Software Package

Deleting a custom software package causes deletion of all YANG and XAR files from Cisco Crosswork. This impacts all collection jobs using the custom software package.

Follow the steps to delete a custom software package:

Procedure

Step 1	From the main menu, choose Administration > Data Gateway Global Settings.
Step 2	From the Custom Software pane, select the custom package you want to delete and click .
Step 3	In the Delete Custom Software window that appears, click Delete to confirm.

Manage Crosswork Data Gateway Collection Jobs

A collection job describes what task a Cisco Crosswork Data Gateway is expected to perform. Applications request data collection via collection jobs. Cisco Crosswork then assigns these collection jobs to a Cisco Crosswork Data Gateway to serve the request.

Crosswork Data Gateway supports multiple data collection protocols including CLI, MDT, SNMP, gNMI (dial-in), syslog, and NETCONF. Crosswork Data Gateway can collect any type of data as long as it can be forwarded over one of the supported protocols.

There are two types of data collection requests in Cisco Crosswork:

Data collection request to forward data for internal processes within Cisco Crosswork. Cisco Crosswork creates system jobs for this purpose. You cannot create or edit system jobs.
Data collection request to forward data to external data destinations.

You can forward collected data to an external data destination and Cisco Crosswork Health Insights in a single collection request by adding the external data destination when creating a KPI profile. For more information, see Section: Create a New KPI Profile in the Cisco Crosswork Change Automation and Health Insights 4.1 User Guide.

Note

Cisco Crosswork Data Gateway drops incoming traffic if there is no corresponding (listening) collection job request for the same. It also drops data, syslog events, and SNMP traps received from an unsolicited device (i.e., not attached to Crosswork Data Gateway).
Polled data cannot be requested from the device until Cisco Crosswork Data Gateway is ready to process and transmit the data.

Types of Collection Jobs

You can create the following list of collection jobs from the Cisco Crosswork UI (CLI/SNMP only) or using APIs to request data.

CLI Collection Job
SNMP Collection Job
MDT Collection Job
Syslog Collection Job
gNMI Collection Job
NETCONF Collection Job

For each collection job that you create, Cisco Crosswork Data Gateway executes the collection request and forwards the collected data to the preferred data destination.

This chapter describes how to create collection jobs from the Cisco Crosswork UI. To create collection jobs using APIs, see Crosswork Data Gateway APIs on Cisco Devnet.

The initial status for all the collection jobs in the Cisco Crosswork UI is Unknown. Upon receiving a collection job, Cisco Crosswork Data Gateway performs basic validations on it. If the collection job is valid, its status changes to Successful, else it changes to Failed.

The value of Cadence is in seconds. This value can be set between 10 seconds and 2764800 seconds ( i.e. at most 32 days) max, depending on how frequently configured sensor data should be collected.

Note

We recommend a cadence of 60 seconds.

When collection from a device is skipped due to previous execution still in progress, Cisco Crosswork Data Gateway raises a warning log. No alert is generated for this scenario.

CLI Collection Job

Cisco Crosswork Data Gateway supports CLI-based data collection from the network devices. Following commands are supported for this type of collection job:

show and the short version sh
traceroute
dir

Devices should not have any banner configuration for CLI collection to work properly. Please refer to device documentation on how to turn this off.

You can create a CLI collection job from the Cisco Crosswork UI or using APIs. See Create a Collection Job from Cisco Crosswork UI or Cisco DevNet for more information.

Following is a sample payload of CLI collection job for a Kafka external destination.

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "CLI_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "658adb03-cc61-448d-972f-4fcec32cbfe8"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "cli_sensor": {
            "command": "show platform"
          }
        },
        "cadence_in_millisec": "tel:60000"
      }
    ],
    "sensor_output_configs": [
     {
        "sensor_data": {
          "cli_sensor": {
            "command": "show platform"
          }
        },
        "destination": {
          "destination_id": "1e71f2fb-ea65-4242-8efa-e33cec71b369",
          "context_id": "topic1"
        }
      }
    ]
  }
}

SNMP Collection Job

Cisco Crosswork Data Gateway supports SNMP-based data collection based on the OIDs supported on the devices.

The SNMP collector makes a poll request to Cisco Crosswork to get its configuration profile (a list of MIB objects to collect and a list of devices to fetch from). It determines the corresponding OIDs by looking up the pre-packaged list of MIB modules or the custom list of MIB modules.

Note

Cisco Crosswork Data Gateway enables SNMP polling on third party devices for standard MIBs already included in the system. Proprietary MIBs are required only if the collection request references MIB TABLE names or SCALAR names from a proprietary MIB. However, if the requests are OID-based, then MIBs are not required.

After the OIDs are resolved, they are provided as input to the SNMP collectors.

The device packages can be imported into the Crosswork Data Gateway VM as described in Section Add a Custom Software Package.

Supported SNMP versions for data polling and traps are:

Polling Data
- SNMP V1
- SNMP V2
- SNMP V3 ( no auth nopriv, auth no priv, authpriv)
- Supported auth protocols – SHA-1,MD5
- Supported priv protocols – DES, 3DES, AES128, AES192, AES256, CiscoAES192, CiscoAES256
Traps
- SNMP V1
- SNMP V2
- SNMP V3 ( no auth nopriv)

Sample Configurations on Device:

Version

Command

To...

V1

snmp-server group <group_name> v1

snmp-server user <user_name> <group_name> v1

Define the SNMP version, user/user group details.

snmp-server host <host_ip> traps <community_string> udp-port 1062

For example,

snmp-server host a.b.c.d traps test udp-port 1062

Define the destination to which trap data must be forwarded.

snmp-server traps snmp linkup

snmp-server traps snmp linkdown

Enable traps to notify link status.

V2c

snmp-server group <group_name> v2c

snmp-server user <user_name> <group_name> v2c

Define the SNMP version, user/user group details.

snmp-server host <host_ip> traps SNMP version <community_string> udp-port 1062

snmp-server host a.b.c.d traps version 2c v2test udp-port 1062

Define the destination to which trap data must be forwarded.

Note

The IP address mentioned here must be the virtual IP address of the Crosswork Data Gateway.

snmp-server traps snmp linkup

snmp-server traps snmp linkdown

Enable traps to notify link status.

V3

snmp-server group <group_name> v3 auth notify <user_name> read <user_name> write <user_name>

snmp-server view <user_name> 1.3 included

Define the SNMP version, user/user group details.

snmp-server user <user_name> <group_name> v3 auth md5 <password> priv aes 128 <password>

snmp-server host <host_IP> traps version 3 priv <user_name> udp-port 1062

Define the destination to which trap data must be forwarded.

Note

The IP address mentioned here must be the virtual IP address of the Crosswork Data Gateway.

snmp-server traps snmp linkup

snmp-server traps snmp linkdown

Enable traps to notify link status.

The SNMP Collector supports the following operations:

SCALAR

Note

If a single collection requests for multiple scalar OIDs, you can pack multiple SNMP GET requests in a single getbulkrequestquery to the device.

TABLE
MIB_WALK
TRAP
DEVICE_PACKAGE

These operations are defined in the sensor config (see payload sample below).

Note

There is an optional deviceParams attribute snmpRequestTimeoutMillis (not shown in the sample payloads) that should be used if the device response time is more than 1500 milliseconds. It’s not recommended to use snmpRequestTimeoutMillis unless you are absolutely certain that your device response time is very high.

The value for snmpRequestTimeoutMillis should be specified in milliseconds:

The default and minimum value is 1500 milliseconds. However, there is no limitation on the maximum value of this attribute.

Following is an SNMP collection job sample:

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "SNMP_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "c70fc034-0cbd-443f-ad3d-a30d4319f937",
            "8627c130-9127-4ed7-ace5-93d3b4321d5e",
            "c0067069-c8f6-4183-9e67-1f2e9bf56f58"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.1.3.0",
              "snmp_operation": "SCALAR"
            }
          }
        },
        "cadence_in_millisec": "60000"
      },
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.31.1.1",
              "snmp_operation": "TABLE"
            }
          }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "sensor_output_configs": [
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.1.3.0",
              "snmp_operation": "SCALAR"
            }
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic1_461cb8aa-a16a-44b8-b79f-c3daf3ea925f"
        }
      },
      {
        "sensor_data": {
          "snmp_sensor": {
            "snmp_mib": {
              "oid": "1.3.6.1.2.1.31.1.1",
              "snmp_operation": "TABLE"
            }
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic2_e7ed6300-fc8c-47ee-8445-70e543057f8a"
        }
      }
    ]
  }
}

SNMP Traps Collection Job

SNMP Traps Collection jobs can be created only via API. Trap listeners listen on a port and dispatch data to recipients (based on their topic of interest).

Crosswork Data Gateway listens on UDP port 1062 for Traps.

Note

Before submitting SNMP Trap collection jobs, SNMP TRAPS need to configured properly on the device to be sent to virtual IP address of the Crosswork Data Gateway.

SNMP Trap Collection Job Workflow

On receiving a SNMP trap, Cisco Crosswork Data Gateway :

Checks if any collection job is created for the device.
Checks the trap version and community string.
For SNMP v3, also validates for user auth and priv protocol and credentials.

Crosswork Data Gateway filters the traps based on the trap OID mentioned in the sensor path and sends only those requested.

If the collection job is invalid, there is missing configuration on the device, or no trap is received, the status of the job remains "Unknown". For list of supported Traps and MIBs, see List of Pre-loaded Traps and MIBs for SNMP Collection.

Crosswork Data Gateway supports three types of non-yang/OID based traps:

Table 4. List of Supported Non-Yang/OID based Traps
sensor path	purpose
*	To get all the traps pushed from the device without any filter.
MIB level traps	OID of one MIB notifications (Ex: 1.3.6.1.2.1.138.0 to get all the isis-mib level traps)
Specific trap	OID of the specific trap (Ex: 1.3.6.1.6.3.1.1.5.4 to get the linkUp trap)

Crosswork Data Gateway supports the following YANG paths:

Table 5. List of Supported Yang paths
sensor path	purpose
snmp-trap-raw-oper:traps/data	To get all the traps pushed from the device without any filter.
IF-MIB:notifications	To get all the IF-MIB notifications (ex: linkUp, linkDown, etc.)
ISIS-MIB:notifications	To get all the ISIS-MIB notifications.
SNMPv2-MIB:notifications	To get all the snmpV2 Mib notifications.

Following is an SNMP-Trap collection job sample:

{
  "collection_job": {
    "application_context": {
      "context_id": "collection-job1",
      "application_id": "APP1"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "TRAP_COLLECTOR"
    },
    "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "a9b8f43d-130b-4866-a26a-4d0f9e07562a",
            "8c4431a0-f21d-452d-95a8-84323a19e0d6",
            "eaab2647-2351-40ae-bf94-6e4a3d79af3a"
          ]
        }
      }
    },
    "sensor_input_configs": [
      {
        "sensor_data": {
          "trap_sensor": {
            "path": "1.3.6.1.6.3.1.1.4"
          }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "sensor_output_configs": [
      {
        "sensor_data": {
          "trap_sensor": {
            "path": "1.3.6.1.6.3.1.1.4"
          }
        },
        "destination": {
          "destination_id": "4c2ab662-2670-4b3c-b7d3-b94acba98c56",
          "context_id": "topic1_696600ae-80ee-4a02-96cb-3a01a2415324"
        }
      }
    ]
  }
}

Enabling Traps forwarding to external applications

We recommended selectively enabling only those traps that are needed by Crosswork on the device .

To identify the type of trap from the data received on the destination, look for oid (OBJECT_IDENTIFIER, for example, 1.3.6.1.6.3.1.1.4.1.0 ) and strValue associated to the oid in the OidRecords (application can match the OID of interest to determine the kind of trap).

Below are some sample values and a sample payload to forward traps to external applications:

Link up

1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.4
Link Down

1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.3
Syslog

1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.4.1.9.9.41.2.0.1
Cold Start

1.3.6.1.6.3.1.1.4.1.0 = 1.3.6.1.6.3.1.1.5.1

{
  "nodeIdStr": "BF5-XRV9K1.tr3.es",
  "nodeIdUuid": "C9tZ5lJoSJKf5OZ67+U5JQ==",
  "collectionId": "133",
  "collectionStartTime": "1580931985267",
  "msgTimestamp": "1580931985267",
  "dataGpbkv": [
    {
      "timestamp": "1580931985267",
      "name": "trapsensor.path",
      "snmpTrap": {
        "version": "V2c",
        "pduType": "TRAP",
        "v2v3Data": {
          "agentAddress": "172.70.39.227",
          "oidRecords": [
            {
              "oid": "1.3.6.1.2.1.1.3.0",
              "strValue": "7 days, 2:15:17.02"
            },
            {
              "oid": "1.3.6.1.6.3.1.1.4.1.0",  // This oid is the Object Identifier.
              "strValue": "1.3.6.1.6.3.1.1.5.3" // This is the value that determines the kind of trap.
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.1.8",
              "strValue": "8"
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.2.8",
              "strValue": "GigabitEthernet0/0/0/2"
            },
            {
              "oid": "1.3.6.1.2.1.2.2.1.3.8",
              "strValue": "6"
            },
            {
              "oid": "1.3.6.1.4.1.9.9.276.1.1.2.1.3.8",
              "strValue": "down"
            }
          ]
        }
      }
    }
  ],
  "collectionEndTime": "1580931985267",
  "collectorUuid": "YmNjZjEzMTktZjFlOS00NTE5LWI4OTgtY2Y1ZmQxZDFjNWExOlRSQVBfQ09MTEVDVE9S",
  "status": {
    "status": "SUCCESS"
  },
  "modelData": {},
  "sensorData": {
    "trapSensor": {
      "path": "1.3.6.1.6.3.1.1.5.4"
    }
  },
  "applicationContexts": [
    {
      "applicationId": "APP1",
      "contextId": "collection-job-snmp-traps"
    }
  ]
}

MDT Collection Job

Crosswork Data Gateway supports data collection from network devices using Model-driven Telemetry (MDT) to consume telemetry streams directly from devices (for IOS-XR based platforms only).

Crosswork Data Gateway supports data collection for the following transport mode:

MDT TCP Dial-out Mode

Cisco Crosswork leverages NSO to push the required MDT configuration to the devices and will send the corresponding collection job configuration to the Crosswork Data Gateway.

Note

If there is some change (update) in existing MDT jobs between backup and restore operations, Cisco Crosswork does not replay the jobs for config update on the devices as this involves NSO. You have to restore configs on NSO/devices. Cisco Crosswork only restores the jobs in database.
Before using any YANG modules, check if they are supported. See Section: List of Pre-loaded YANG Modules for MDT Collection

Following is a sample of MDT collection payload:

{
	"collection_job": {
		"job_device_set": {
			"device_set": {
				"device_group": "mdt"
			}
		},
		"sensor_output_configs": [{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
					}
				},
				"destination": {
					"context_id": "cw.mdt_sensor.cisco-ios-xr-infra-statsd-oper.gpb",
					"destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
				}
			},
			{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/data-rate"
					}
				},
				"destination": {
					"context_id": "cw.mdt_sensor.cisco-ios-xr-infra-statsd-oper.gpb",
					"destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
				}
			}
		],
		"sensor_input_configs": [{
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/data-rate"
					}
				},
				"cadence_in_millisec": "70000"
			}, {
				"sensor_data": {
					"mdt_sensor": {
						"path": "Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters"
					}
				},
				"cadence_in_millisec": "70000"
			}
		],
		"application_context": {
			"context_id": "c4",
			"application_id": "a4-mdt"
		},
		"collection_mode": {
			"lifetime_type": "APPLICATION_MANAGED",
			"collector_type": "MDT_COLLECTOR"
		}
	}
}

Syslog Collection Job

Crosswork Data Gateway supports Syslog-based events collection from devices. Following Syslog formats are supported:

RFC5424 syslog format
RFC3164 syslog format

Note

In order to gather syslog data from devices in the network, you must configure the devices to send syslog data to the Crosswork Data Gateway. Refer to the platform-specific documentation.

For sample device configuration, see Configure Syslog (Non-Secure) on Device. Cisco Crosswork also allows you to setup secure syslog communication to the device. See sample device configuration at Configure Secure Syslog on Device.

Syslog Data Collection

Syslog data can be filtered by specifying either PRI-based SyslogSensor or Filters-based SyslogSensor. Only those syslog events that match the filters mentioned in the payload are sent to the specified destination.

Following is a sample Syslog collection payload with PRI-based SyslogSensor filter.

{
  "collection_job": {
      "job_device_set": {
      "device_set": {
        "devices": {
          "device_ids": [
            "c6f25a33-92e6-468a-ba0d-15490f1ce787"
          ]
        }
      }
    },
    "sensor_output_configs": [
      {
        "sensor_data": {
          "syslog_sensor": {
            "pris": {
                "facilities": [0, 1, 3, 23,4],
                "severities": [0, 4, 5, 6, 7]
            }
        }
        },
        "destination": {
          "context_id": "syslogtopic",
          "destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
        }
      }
    ],
    "sensor_input_configs": [
      {
        "sensor_data": {
          "syslog_sensor": {
            "pris": {
                "facilities": [0,1, 3, 23,4],
                "severities": [0,4, 5, 6, 7]
            }
        }
        },
        "cadence_in_millisec": "60000"
      }
    ],
    "application_context": {
      "context_id": "demomilesstone2syslog",
      "application_id": "SyslogDemo2"
    },
    "collection_mode": {
      "lifetime_type": "APPLICATION_MANAGED",
      "collector_type": "SYSLOG_COLLECTOR"
    }
  }
}

The Filters-based SyslogSensor is based on Regex , PRI and severity-facility. You can specify and combine multiple filters (maximum 3 filters) using AND or OR. By default, an AND condition is applied if there is no logical operator specified. Following is a sample Syslog collection payload with Filters-based SyslogSensor filter.

{
      "collection_job": {
      "job_device_set": {
      "device_set": {
      "devices": {
      "device_ids": [
      "ce33ad3c-d6d0-42b7-b24b-67dfa77c6ee8"
                ]
              }
           }
       },
      "sensor_output_configs": [{
        "sensor_data": {
          "syslog_sensor": {
            "filters": {
              "filter": [{
                "syslog_filter": {
                  "severity_facility": {
                      "severity": {
                        "op": "LESSER_THAN",
                         "value": 7
                                  },
                          "facility": {
                          "op": "EQUALS",
                          "value": 23
                           }
                       }
                    }
                },
                {
                  "syslog_filter": {
                   "pri_filter": {
                    "value": {
                      "op": "GREATER_THAN",
                        "value": 180
                                    }
                                  }
                                }
                          },
                         {
                           "syslog_filter": {
                            "regex_filter": {
                            "pattern": "SSHD\\[\\d+\\]"
                                            }
                                          }
                                        }
                                        ],
                              "operator": "AND"
                                     }
                                   }
                              },
             "destination": {
             "context_id": "3filtersand",
              "destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
                            }
                          }],
          "sensor_input_configs": [{
          "sensor_data": {
           "syslog_sensor": {
                 "filters": {
                    "filter": [{
                        "syslog_filter": {
                            "severity_facility": {
                                "severity": {
                                    "op": "LESSER_THAN",
                                        "value": 7
                                              },
                                    "facility": {
                                     "op": "EQUALS",
                                      "value": 23
                                                }
                                              }
                                             }
                                            },
                                      {
                                    "syslog_filter": {
                                    "pri_filter": {
                                      "value": {
                                      "op": "GREATER_THAN",
                                      "value": 180 
                                              }
                                            }
                                          }
                                        },
                                      {
                                  "syslog_filter": {
                                  "regex_filter": {
                                  "pattern": "SSHD\\[\\d+\\]"
                                      }
                                    }
                                  }
                                ],
                          "operator": "AND"
                              }
                            }
                          },
              "cadence_in_millisec": "60000"
                      }],
          "application_context": {
          "context_id": "AND_syslog.3Filters_oneofeach",
          "application_id": "testing.postman.syslog.3Filters_oneofeach_AND"
              },
         "collection_mode": {
         "lifetime_type": "APPLICATION_MANAGED",
         "collector_type": "SYSLOG_COLLECTOR"
          }
       }
}

Syslog Collection Job Output

When you onboard a device from Cisco Crosswork UI (Device Management > Network Devices > Device Details), the value you choose in the Syslog Format field configures the format in which syslog events received from the device should be parsed by the Syslog Collector. You can choose either UNKNOWN, RFC5424 or RFC3164.

Following is the sample output for each of the options:

UNKNOWN - Syslog Collection Job output contains syslog events as received from device.

Note

If the device is configured to generate syslog events in RFC5424/RFC3164 format but no format is specified in the Syslog Format field, by default this is considered as UNKNOWN.

Sample output:

node_id_str: "xrv9k-VM8"
node_id_uuid: ":i\300\216>\366BM\262\270@\337\225\2723&"
collection_id: 1056
collection_start_time: 1616711596200
msg_timestamp: 1616711596201
data_gpbkv {
  timestamp: 1616711596201
  name: "syslogsensor.path"
  fields {
    name: "RAW"
    string_value: "<6>1 Mar 25 15:34:41.321 PDT - SSHD_ 69570 - - 98949: RP/0/RP0/CPU0:SSHD_[69570]: %SECURITY-SSHD-6-INFO_SUCCESS : Successfully authenticated user \'admin\' from \'40.40.40.116\' on \'vty0\'(cipher \'aes128-ctr\', mac \'hmac-sha1\') \n"
  }
  fields {
    name: "DEVICE_IP"
    string_value: "40.40.40.30"
  }
}
collection_end_time: 1616711596200
collector_uuid: "17328736-b726-4fe3-b922-231a4a30a54f:SYSLOG_COLLECTOR"
status {
  status: SUCCESS
}
model_data {
}
sensor_data {
  syslog_sensor {
    pris {
      facilities: 0
      facilities: 3
      facilities: 4
      facilities: 23
      severities: 0
      severities: 5
      severities: 6
      severities: 7
    }
  }
}
application_contexts {
  application_id: "SyslogApp-xr-8-job1"
  context_id: "xr-8-job1"
}
version: "1"

RFC5424 - If the device is configured to generate syslog events in RFC5424 format and the RFC5424 format is selected in the Syslog Format field, the Syslog Job Collection output contains syslog events as received from device (RAW) and the RFC5424 best-effort parsed syslog events from the device.

Note

The syslog collector will parse the syslog event(best effort parsing) as per the following Java RegEx pattern:

Sample output:

....
....
 
 
collection_start_time: 1596307542398
msg_timestamp: 1596307542405
data_gpbkv {
  timestamp: 1596307542405
  name: "syslogsensor.path"
  fields {
    name: "RAW"
    string_value: "<13>1 2020 Aug  1 12:03:32.461 UTC:  iosxr254node config 65910 - - 2782: RP/0/RSP0/CPU0:2020 Aug  1 12:03:32.461 UTC: config[65910]: %MGBL-SYS-5-CONFIG_I : Configured from console by admin on vty0 (10.24.88.215) \n"
  }
  fields {
    name: "RFC5424"
    string_value: "pri=13,  severity=5,  facility=1,  version=1,  date=2020-08-01T12:03:32.461,  remoteAddress=/172.28.122.254,  host=\'iosxr254node\',  message=\'2782: RP/0/RSP0/CPU0:2020 Aug  1 12:03:32.461 UTC: config[65910]: %MGBL-SYS-5-CONFIG_I : Configured from console by admin on vty0 (10.24.88.215) \', messageId=null, processName=config, structuredDataList=null"
  }
  fields {
    name: "DEVICE_IP"
    string_value: "172.28.122.254"
  }
}
collection_end_time: 1596307542404
collector_uuid: "ac961b09-8f67-4c93-a99a-31eef50f7fa9:SYSLOG_COLLECTOR"
status {
  status: SUCCESS
}
...
...

RFC3164 - If the device is configured to generate syslog events in RFC3164 format and the RFC3164 format is selected in Syslog Format field, the Syslog Job Collection output contains both RAW (as received from device) syslog events and the RFC3164 best-effort parsed syslog events from the device.

Note

The syslog collector will parse the syslog event(best effort parsing) as per the following Java RegEx pattern:

Sample output:

....
.....
collection_id: 20
collection_start_time: 1596306752737
msg_timestamp: 1596306752743
data_gpbkv {
  timestamp: 1596306752743
  name: "syslogsensor.path"
  fields {
    name: "RAW"
    string_value: "<14>2020 Aug  1 11:50:22.799 UTC:  iosxr254node 2756: RP/0/RSP0/CPU0:2020 Aug  1 11:50:22.799 UTC: config[65910]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user \'admin\'. Use \'show configuration commit changes 1000000580\' to view the changes. \n"
  }
  fields {
    name: "RFC3164"
    string_value: "pri=14,  severity=6,  facility=1,  version=null,  date=2020-08-01T11:50:22.799,  remoteAddress=/172.28.122.254,  host=\'iosxr254node\',  message=\'RP/0/RSP0/CPU0:2020 Aug  1 11:50:22.799 UTC: config[65910]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user \'admin\'. Use \'show configuration commit changes 1000000580\' to view the changes. \', tag=2756"
  }
  fields {
    name: "DEVICE_IP"
    string_value: "172.28.122.254"
  }
}
collection_end_time: 1596306752742
collector_uuid: "ac961b09-8f67-4c93-a99a-31eef50f7fa9:SYSLOG_COLLECTOR"
status {
  status: SUCCESS
}
....
....

If the Syslog Collector is unable to parse the syslog events according to the format specified in the Syslog Format field, then the Syslog Collection Job output contains syslog events as received from device (RAW).

Configure Syslog (Non-Secure) on Device

This section lists sample configuration to configure syslog in the RFC3164 or RFC5424 format on the device.

Configure RFC3164 Syslog format

Note

The configuration highlighted in the code below is required to avoid formatting issues in the parsed output.

For Cisco IOS XR devices:

logging <CDG IP> port 9514 OR logging <CDG IP> vrf <vrfname> port 9514 
logging trap [severity]
logging facility [facility value]
logging suppress duplicates
service timestamps log datetime msec show-timezone year
logging hostnameprefix <some host related prefix e.g.iosxrhost2>

For Cisco IOS XE Devices:

no logging message-counter syslog 
logging trap <serverity>
logging facility <facility>
logging host <CDG IP> transport tcp port 9898 session-id string <sessionidstring> --> To use TCP channel
OR
logging host <CDG IP> transport udp port 9514 session-id string <sessionidstring> ---> To use UDP channel
OR
logging host <CDG IP> vrf Mgmt-intf transport udp port 9514 session-id string <sessionidstring> --> To use UDP via vrf 
service timestamps log datetime msec year show-timezone

Configure RFC5424 Syslog format

For Cisco IOS XR devices:

logging <CDG IP> port 9514 OR logging <server 1> vrf <vrfname> port 9514 
logging trap [severity]
logging facility [facility value]
logging suppress duplicates
service timestamps log datetime msec show-timezone year
logging hostnameprefix <some host related prefix e.g.iosxrhost2>
logging format rfc5424

For Cisco IOS XE Devices:

no logging message-counter syslog
logging trap <serverity>
logging facility <facility>
logging host <CDG IP> transport tcp port 9898 session-id string <sessionidstring> --> To use TCP channel
OR
logging host <CDG IP> transport udp port 9514 session-id string <sessionidstring> ---> To use UDP channel
OR
logging host <CDG IP> vrf Mgmt-intf transport udp port 9514 session-id string <sessionidstring> --> To use UDP via vrf 
service timestamps log datetime msec year show-timezone
logging trap syslog-format 5424 --> if applicable

Configure Secure Syslog on Device

Follow these steps to establish a secure syslog communication to the device.

Download the Cisco Crosswork trust chain from the Certificate Management UI page in Cisco Crosswork.
Configure device with the Cisco Crosswork trust chain.

Download Syslog Certificates

In the Cisco Crosswork UI, go to Administration > Certificate Management.
Click i in the 'crosswork-device-syslog' row.
Click Export All to download the certificates.

The following files are downloaded to your system.

Configure Cisco Crosswork Trustpoint on Device

Sample XR Device Configuration to enable TLS

RP/0/RSP0/CPU0:ASR9k(config)#crypto ca trustpoint syslog-root
RP/0/RSP0/CPU0:ASR9k(config-trustp)#enrollment terminal
RP/0/RSP0/CPU0:ASR9k(config-trustp)#crl optional
RP/0/RSP0/CPU0:ASR9k(config-trustp)#commit
RP/0/RSP0/CPU0:ASR9k(config-trustp)#end
RP/0/RSP0/CPU0:ASR9k#
RP/0/RSP0/CPU0:ASR9k#crypto ca authenticate syslog-root
Fri Jan 22 11:07:41.880 GMT
  
  
Enter the base 64 encoded certificate.
End with a blank line or the word "quit" on a line by itself
  
-----BEGIN CERTIFICATE-----
MIIGKzCCBBOgAwIBAgIRAKfyU89yjmrXVDRKBWuSGPgwDQYJKoZIhvcNAQELBQAw
bDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMREwDwYDVQQHEwhTYW4gSm9zZTEa
................................................................
................................................................
jPQ/UrO8N3sC1gGJX7CIIh5cE+KIJ51ep8i1eKSJ5wHWRTmv342MnG2StgOTtaFF
vrkWHD02o6jRuYXDWEUptDOg8oEritZb+SNPXWUc/2mbYog6ks6EeMC69VjkZPo=
-----END CERTIFICATE-----
  
Read 1583 bytes as CA certificate
  Serial Number  : A7:F2:53:CF:72:8E:6A:D7:54:34:4A:05:6B:92:18:F8
  Subject:
                CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Issued By      :
                CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Validity Start : 02:37:09 UTC Sat Jan 16 2021
  Validity End   : 02:37:09 UTC Thu Jan 15 2026
  SHA1 Fingerprint:
                209B3815271C22ADF78CB906F6A32DD9D97BBDBA
  
Fingerprint: 2FF85849EBAAB9B059ACB9F5363D5C9CDo you accept this certificate? [yes/no]: yes
RP/0/RSP0/CPU0:ASR9k#config
RP/0/RSP0/CPU0:ASR9k(config)#crypto ca trustpoint syslog-inter
RP/0/RSP0/CPU0:ASR9k(config-trustp)#enrollment terminal
RP/0/RSP0/CPU0:ASR9k(config-trustp)#crl optional
RP/0/RSP0/CPU0:ASR9k(config-trustp)#commit
RP/0/RSP0/CPU0:ASR9k#crypto ca authenticate syslog-inter
Fri Jan 22 11:10:30.090 GMT
  
  
Enter the base 64 encoded certificate.
End with a blank line or the word "quit" on a line by itself
  
-----BEGIN CERTIFICATE-----
MIIGFDCCA/ygAwIBAgIRAkhqHQXcJzQzeQK6U2wn8PIwDQYJKoZIhvcNAQELBQAw
bDELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMREwDwYDVQQHEwhTYW4gSm9zZTEa
................................................................
................................................................
5lBk617z6cxFER5c+/PmJFhcreisTxXg1aJbFdnB5C8f+0uUIdLghykQ/zaZGuBn
AAB70c9r9OeKGJWzvv1e2U8HH1pdQ/nd
-----END CERTIFICATE-----
  
Read 1560 bytes as CA certificate
  Serial Number  : 02:48:6A:1D:05:DC:27:34:33:79:02:BA:53:6C:27:F0:F2
  Subject:
                CN=device-syslog,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Issued By      :
                CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Validity Start : 02:37:11 UTC Sat Jan 16 2021
  Validity End   : 02:37:11 UTC Mon Jan 16 2023
  SHA1 Fingerprint:
                B06F2BFDE95413A8D08A01EE3511BC3D42F01E59
  
CA Certificate validated using issuer certificate.
RP/0/RSP0/CPU0:ASR9k#show crypto ca certificates
Fri Jan 22 15:45:17.196 GMT
 
 
Trustpoint       : syslog-root
==================================================
CA certificate
  Serial Number  : A7:F2:53:CF:72:8E:6A:D7:54:34:4A:05:6B:92:18:F8
  Subject:
        CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Issued By      :
        CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Validity Start : 02:37:09 UTC Sat Jan 16 2021
  Validity End   : 02:37:09 UTC Thu Jan 15 2026
  SHA1 Fingerprint:
         209B3815271C22ADF78CB906F6A32DD9D97BBDBA
 
 
Trustpoint       : syslog-inter
==================================================
CA certificate
  Serial Number  : 02:48:6A:1D:05:DC:27:34:33:79:02:BA:53:6C:27:F0:F2
  Subject:
        CN=device-syslog,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Issued By      :
        CN=Crosswork Device Root CA,O=CISCO SYSTEMS INC,L=San Jose,ST=CA,C=US
  Validity Start : 02:37:11 UTC Sat Jan 16 2021
  Validity End   : 02:37:11 UTC Mon Jan 16 2023
  SHA1 Fingerprint:
         B06F2BFDE95413A8D08A01EE3511BC3D42F01E59
RP/0/RSP0/CPU0:ASR9k(config)#logging tls-server syslog-tb131
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#tls-hostname 10.13.0.159
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#trustpoint syslog-inter
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#severity debugging
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#vrf default
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#commit
RP/0/RSP0/CPU0:ASR9k(config-logging-tls-peer)#exit
RP/0/RSP0/CPU0:ASR9k(config)#exit
RP/0/RSP0/CPU0:ASR9k#exit
RP/0/RSP0/CPU0:ASR9k#show running-config logging
Fri Jan 22 11:17:19.385 GMT
logging tls-server syslog-tb131
vrf default
severity debugging
trustpoint syslog-inter
tls-hostname <CDG Southbound IP>
!
logging trap debugging
logging format rfc5424
logging facility user
logging hostnameprefix ASR9k
logging suppress duplicates
  
RP/0/RSP0/CPU0:ASR9k#

Sample XE Device Configuration to enable TLS

csr8kv(config)#crypto pki trustpoint syslog-root
csr8kv(ca-trustpoint)#enrollment terminal
csr8kv(ca-trustpoint)#revocation-check none
csr8kv(ca-trustpoint)#chain-validation stop
csr8kv(ca-trustpoint)#end
csr8kv(config)#crypto pki authenticate syslog-root
 
Enter the base 64 encoded CA certificate.
End with a blank line or the word "quit" on a line by itself
 
-----BEGIN CERTIFICATE-----
MIIFPjCCAyYCCQCO6pK5AOGYdjANBgkqhkiG9w0BAQsFADBhMQswCQYDVQQGEwJV
UzELMAkGA1UECAwCQ0ExETAPBgNVBAcMCE1pbHBpdGFzMQ4wDAYDVQQKDAVDaXNj
................................................................
................................................................
JbimOpXAncoBLo14DXOJLvMVRjn1EULE9AXXCNfnrnBx7jL4CV+qHgEtF6oqclFW
JEA=
-----END CERTIFICATE-----
 
Certificate has the following attributes:
       Fingerprint MD5: D88D6D8F E53750D4 B36EB498 0A435DA1
      Fingerprint SHA1: 649DE822 1C222C1F 5101BEB8 B29CDF12 5CEE463B
 
% Do you accept this certificate? [yes/no]: yes
Trustpoint CA certificate accepted.
% Certificate successfully imported
 
 
csr8kv(config)#crypto pki trustpoint syslog-intermediate
csr8kv(ca-trustpoint)#enrollment terminal
csr8kv(ca-trustpoint)#revocation-check none
csr8kv(ca-trustpoint)#chain-validation continue syslog-root
csr8kv(ca-trustpoint)#end
csr8kv(config)#crypto pki authenticate syslog-intermediate
 
Enter the base 64 encoded CA certificate.
End with a blank line or the word "quit" on a line by itself
 
-----BEGIN CERTIFICATE-----
MIIFfTCCA2WgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwXDELMAkGA1UEBhMCVVMx
EzARBgNVBAgMCkNhbGlmb3JuaWExDjAMBgNVBAoMBUNpc2NvMQ4wDAYDVQQLDAVT
................................................................
................................................................
Nmz6NQynD7bxdQa9Xq9kyPuY3ZVKXkf312IRH0MEy2yFX/tAen9JqOeZ1g8canmw
TxsWA5TLzy1RmxqQh88f0CM=
-----END CERTIFICATE-----
Trustpoint 'syslog-intermediate' is a subordinate CA.
but certificate is not a CA certificate.
Manual verification required
Certificate has the following attributes:
       Fingerprint MD5: FE27BDBE 9265208A 681670AC F59A2BF1
      Fingerprint SHA1: 03F513BD 4BEB689F A4F4E001 57EC210E 88C7BD19
 
csr8kv(config)#logging host <CDG Southbound IP> transport tls port 6514
csr8kv(config)#logging trap informational syslog-format rfc5424
csr8kv(config)#logging facility user
csr8kv(config)#service timestamps log datetime msec year show-timezone

csr8kv(config)#logging tls-profile tlsv12

gNMI Collection Job

Cisco Crosswork supports gRPC Network Management Interface (gNMI) based telemetry data collection via Cisco Crosswork Data Gateway. It supports only gNMI Dial-In (gRPC Dial-In) streaming telemetry data based on subscription and relaying subsequent subscription response (notifications) to the requested destinations.

Note

gNMI collection is supported as long as the models are supported by the target device platform. gNMI must be configured on devices before you can submit gNMI collection jobs. Check platform-specific documentation.

To configure gNMI on the device, see Sample Device Configuration - gNMI.

In gNMI, both secure and insecure mode can co-exist on the device. Cisco Crosswork gives preference to secure mode over non-secure mode based on the information passed in the inventory.

If a device reloads, gNMI collector ensures that the existing subscriptions are re-subscribed to the device.

gNMI specification does not have a way to mark end of message. Hence, Destination and Dispatch cadence is not supported in gNMI collector.

Cisco Crosswork Data Gateway supports the following types of subscribe options for gNMI:

Table 6. gNMI Subscription Options
Type	Subtype	Description
Once		Collects and sends the current snapshot of the system configuration only once for all specified paths
Stream	SAMPLE	Cadence-based collection.
	ON_CHANGE	First response includes the state of all the elements for the subscribed path, followed by subsequent updates to the changes leaf values.
	TARGET_DEFINED	Router/Device chooses the mode of subscription on a per-leaf basis based on the subscribed path (i.e. one of SAMPLE or ON_CHANGE)

Crosswork Data Gateway supports the ability to subscribe to multiple subscription paths in a single subscription list to the device. For example, you can specify a combination of ON_CHANGE and subscription mode ONCE collection jobs. ON_CHANGE mode collects data only on change of any particular element for the specified path, while subscription mode ONCE collects and sends current system data only once for the specified path.

Note

Crosswork Data Gateway relies on the device to declare the support of one or more modes.

gNMI sensor path with default values does not appear in the payload. This is a known protobuf behavior.

For boolean the default value is false. For enum, it is gnmi.proto specified.

Example 1:

message GNMIDeviceSetting {
bool suppress_redundant = 1;
bool allow_aggregation = 4;
bool updates_only = 6;
}

Example 2:

enum SubscriptionMode {
TARGET_DEFINED = 0; //default value will not be printed
ON_CHANGE = 1;
SAMPLE = 2;
}

Following is a sample gNMI collection payload. In this sample you see two collections for the device group "milpitas". The first collects interface statistics, every 10 seconds using the "mode" = "SAMPLE". The second job captures any changes to the interface state (up/down). If this is detected it is simply sent "mode= "STREAM" to the collector.

{
    "collection_job": {
        "job_device_set": {
            "device_set": {
                "device_group": "milpitas"
            }
        },
        "sensor_output_configs": [{
            "sensor_data": {
                "gnmi_standard_sensor": {
                    "Subscribe_request": {
                        "subscribe": {
                            "subscription": [{
                                "path": {
                                    "origin": "openconfig-interfaces",
                                    "elem": [{
                                        "name": "interfaces/interface/state/ifindex"
                                    }]
                                },
                                "mode": "SAMPLE",
                                "sample_interval": 10000000000
                            }, {
                                "path": {
                                    "origin": "openconfig-interfaces",
                                    "elem": [{
                                        "name": "interfaces/interfaces/state/counters/out-octets"
                                    }]
                                },
                                "mode": "ON_CHANGE",
                                "sample_interval": 10000000000
                            }],
                            "mode": "STREAM",
                            "encoding": "JSON"
                        }
                    }
                }
            },
            "destination": {
                "context_id": "hukarz",
                "destination_id": "c2a8fba8-8363-3d22-b0c2-a9e449693fae"
            }
        }],
        "sensor_input_configs": [{
            "sensor_data": {
                "gnmi_standard_sensor": {
                    "Subscribe_request": {
                        "subscribe": {
                            "subscription": [{
                                "path": {
                                    "origin": "openconfig-interfaces",
                                    "elem": [{
                                        "name": "interfaces/interface/state/ifindex"
                                    }]
                                },
                                "mode": "SAMPLE",
                                "sample_interval": 10000000000
                            }, {
                                "path": {
                                    "origin": "openconfig-interfaces",
                                    "elem": [{
                                        "name": "interfaces/interfaces/state/counters/out-octets"
                                    }]
                                },
                                "mode": "ON_CHANGE",
                                "sample_interval": 10000000000
                            }],
                            "mode": "STREAM",
                            "encoding": "JSON"
                        }
                    }
                }
            },
            "cadence_in_millisec": "60000"
        }],
        "application_context": {
            "context_id": "testing.group.gnmi.subscription.onchange",
            "application_id": "testing.postman.gnmi.standard.persistent"
        },
        "collection_mode": {
            "lifetime_type": "APPLICATION_MANAGED",
            "collector_type": "GNMI_COLLECTOR"
        }
    }
}

Enable Secure gNMI communication between Device and Crosswork Data Gateway

Cisco Crosswork can only use one rootCA certificate (self-signed or signed by a trusted root CA) which means all device certificates must be signed by same CA.

If you have certificates signed by a different a trusted root CA, you can skip the first step and start from Step 2 to import the rootCA certificate in Cisco Crosswork.

Follow these steps to enable secure gNMI between Cisco Crosswork and the devices:

Generate the certificates. See Generate Device Certificates.
.
Upload the certificates to the Crosswork Certificate Management UI in Cisco Crosswork. See Configure gNMI Certificate.
Update device configuration with secure gNMI port details from Cisco Crosswork UI. See Update Protocol on Device from Cisco Crosswork
Enable gNMI on the device. See Sample Device Configuration - gNMI
Configure the certificates and device key on the device. Import Certificates on Device.

Generate Device Certificates

This section explains how to create certificates with OpenSSL.

Steps to generate certificates have been validated with Open SSL and Microsoft. For the purpose of these instructions, we have explained the steps to generate device certificates with Open SSL.

Note

To generate device certificates with a utility other than Open SSL or Microsoft, please work with the Cisco Support Team.

Create the rootCA
```
# openssl genrsa -out rootCA.key
# openssl req -subj /C=/ST=/L=/O=/CN=CrossworkCA -x509 -new -nodes -key rootCA.key -sha256 -out rootCA.pem -days 1024
```
In the above command, the days attribute determines the how long the certificate is valid. The minimum value is 30 days which means you will need to update the certificates every 30 days. We recommend setting the value to 365 days.

Create device key and certificate


# openssl genrsa -out device.key
# openssl req -subj /C=/ST=/L=/O=/CN=Crosswork -new -key device.key -out device.crs
# openssl x509 -req -extfile <(printf "subjectAltName=IP.0: 10.58.56.18") -in device.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -sha256 -out device3.crt -days 1024

If you have multiple devices, instead of creating multiple device certificates, you can specify multiple device IP addresses separated by a comma in the subjectAltName.

# openssl x509 -req -extfile <(printf "subjectAltName=IP.0: 10.58.56.18, IP.1: 10.58.56.19, IP.2: 10.58.56.20 ..... ") -in device.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -sha256 -out device.crt -days 1024

Configure gNMI Certificate

Crosswork Data Gateway acts as the gNMI client while the device acts as gNMI server. Crosswork Data Gateway validates the device using a trust chain. It is expected that you have a global trust chain for all the devices. If you have multiple trust chains, add all the device trust chains (single or multiple vendors) in a single .pem file and upload this .pem file to the Crosswork Certificate Management UI.

Note

You can upload only one gNMI Certificate to Crosswork.

To configure the gNMI Certificate:

Procedure

Step 1

From the Cisco Crosswork UI, go to Administration > Certificate Management.

Step 2

Click the + icon to add certificate.

Step 3

In Add Certificate window, enter the following details:

Device Certificate Name - Enter a name for the certificate.
Certificate Role - Select Device gNMI Communication from the drop-down list.
Device Trust Chain - Browse your local file system to the location of the rootCA file and upload it.

Note

If gNMI certificate is already configured and you wish to onboard a device with a different trust chain, update the existing .pem file to include details of the new CA. Select the existing gNMI certificate from the list, click the Edit icon and upload the new .pem file.

Step 4

Click Save.

The gNMI Certificate is listed in the configured certificates once it has been added successfully.

Update Protocol on Device from Cisco Crosswork

After you have configured the gNMI certificate in the Cisco Crosswork, update the device with secure protocol details either from the Cisco Crosswork UI( Device Management > Network Devices) or by specifying the protocol details as GNMI_SECURE Port in the .csv file.

The following image shows the updated secure Protocol details for a device.

Sample Device Configuration - gNMI

Cisco IOS XR devices

Enable gRPC over an HTTP/2 connection.
```
Router#configure
Router(config)#grpc
Router(config-grpc)#port <port-number>
```
The port number ranges from 57344 to 57999. If a port number is unavailable, an error is displayed.
Set the session parameters.
```
Router(config)#grpc{ address-family | dscp | max-request-per-user | max-request-total | max-streams | 
max-streams-per-user | no-tls | service-layer | tls-cipher | tls-mutual | tls-trustpoint | vrf }
```
where:
- address-family: set the address family identifier type
- dscp: set QoS marking DSCP on transmitted gRPC
- max-request-per-user: set the maximum concurrent requests per user
- max-request-total: set the maximum concurrent requests in total
- max-streams: set the maximum number of concurrent gRPC requests. The maximum subscription limit is 128 requests. The default is 32 requests
- max-streams-per-user: set the maximum concurrent gRPC requests for each user. The maximum subscription limit is 128 requests. The default is 32 requests
- no-tls: disable transport layer security (TLS). The TLS is enabled by default.
- service-layer: enable the grpc service layer configuration
- tls-cipher: enable the gRPC TLS cipher suites
- tls-mutual: set the mutual authentication
- tls-trustpoint: configure trustpoint
- server-vrf: enable server vrf

Enable TPA (Traffic Protection for Third-Party Applications).

tpa
vrf default
  address-family ipv4
   default-route mgmt
   update-source dataports MgmtEth0/RP0/CPU0/0

Cisco IOS XE Devices

The following example shows how to enable the gNMI server in insecure mode:

Device# configure terminal
Device(config)# gnmi-yang
Device(config)# gnmi-yang server
Device(config)# gnmi-yang port 50000 <The default port is 50052.>
Device(config)# end
Device

The following example shows how to enable the gNMI server in secure mode:

Certs and trustpoint are only required for secure gNMI servers.

Device# configure terminal
Device(config)# gnmi-yang server
Device(config)# gnmi-yang secure-server
Device(config)# gnmi-yang secure-trustpoint trustpoint1
Device(config)# gnmi-yang secure-client-auth
Device(config)# gnmi-yang secure-port 50001 <The default port is 50051.>
Device(config)# end
Device

Device certificates

Certs and trustpoint are only required for secure gNMI servers.

Creating Certs with OpenSSL on Linux

The following example shows how to create Certs with OpenSSL on a Linux machine:

# Setting up a CA
openssl genrsa -out rootCA.key 2048
openssl req -subj /C=/ST=/L=/O=/CN=rootCA -x509 -new -nodes -key rootCA.key -sha256 -out rootCA.pem
 
# Setting up device cert and key
openssl genrsa -out device.key 2048
openssl req -subj /C=/ST=/L=/O=/CN=<hostnameFQDN> -new -key device.key -out device.csr
openssl x509 -req -in device.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out device.crt -sha256
# Encrpyt device key - needed for input to IOS
openssl rsa -des3 -in device.key -out device.des3.key -passout pass:<password - remember this for later>
 
# Setting up client cert and key
openssl genrsa -out client.key 2048
openssl req -subj /C=/ST=/L=/O=/CN=gnmi_client -new -key client.key -out client.csr
openssl x509 -req -in client.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out client.crt -sha256

Installing Certs on a Cisco IOS XR Device

To install certs on Cisco IOS XR, replace files in the following path:

Login into XR machine.
Type run command on terminal prompt.
RP/0/RP0/CPU0:xrvr-7.2.1#run
Navigate to the following directory:
cd /misc/config/grpc
Replace the content of the following files:
- replace contents of ems.pem with device.crt
- replace contents of ems.key with device.key
- replace contents of ca.cert with rootCA.pem

Installing Certs on a Cisco IOS XE Device

The following example shows how to install certs on a Cisco IOS XE device:

# Send:
Device# configure terminal
Device(config)# crypto pki import trustpoint1 pem terminal password password1 
 
# Receive:
% Enter PEM-formatted CA certificate.
% End with a blank line or "quit" on a line by itself.
 
# Send:
# Contents of rootCA.pem, followed by newline + 'quit' + newline:
-----BEGIN CERTIFICATE-----
<snip>
-----END CERTIFICATE-----
quit
 
# Receive:
% Enter PEM-formatted encrypted private General Purpose key.
% End with "quit" on a line by itself.
 
# Send:
# Contents of device.des3.key, followed by newline + 'quit' + newline:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,D954FF9E43F1BA20
<snip>
-----END RSA PRIVATE KEY-----
quit
 
# Receive:
% Enter PEM-formatted General Purpose certificate.
% End with a blank line or "quit" on a line by itself.
 
# Send:
# Contents of device.crt, followed by newline + 'quit' + newline:
-----BEGIN CERTIFICATE-----
<snip>
-----END CERTIFICATE-----
quit
 
# Receive:
% PEM files import succeeded.
Device(config)#
 
# Send:
Device(config)# crypto pki trustpoint trustpoint1
Device(ca-trustpoint)# revocation-check none
Device(ca-trustpoint)# end
Device#

Import Certificates on Device

Install Certificates on a Cisco IOS XR Device

To install certificates on a Cisco IOS XR device,

Copy rootCA.pem, device.key and device.crt to the device under /tmp folder.
Login into the IOS XR device.
Use the run command to enter the VM shell.
RP/0/RP0/CPU0:xrvr-7.2.1#run
Navigate to the following directory:
cd /misc/config/grpc

Create or replace the content of the following files:

Note

If TLS was previously enabled on your device, the following files will already be present in which case replace the content of these files as explained below. If this is the first time, you are enabling TLS on the device, copy the files from the /tmp folder to this folder.

ems.pem with device.crt
ems.key with device.key
ca.cert with rootCA.pem

Restart TLS on the device for changes to take effect. This can be done disabling TLS with "no-tls" command and re-enabling it with "no no-tls" config command on the device.

Installing Certs on a Cisco IOS XE Device

The following example shows how to install certs on a Cisco IOS XE device:

# Send:
Device# configure terminal
Device(config)# crypto pki import trustpoint1 pem terminal password password1 
 
# Receive:
% Enter PEM-formatted CA certificate.
% End with a blank line or "quit" on a line by itself.
 
# Send:
# Contents of rootCA.pem, followed by newline + 'quit' + newline:
-----BEGIN CERTIFICATE-----
<snip>
-----END CERTIFICATE-----
quit
 
# Receive:
% Enter PEM-formatted encrypted private General Purpose key.
% End with "quit" on a line by itself.
 
# Send:
# Contents of device.des3.key, followed by newline + 'quit' + newline:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,D954FF9E43F1BA20
<snip>
-----END RSA PRIVATE KEY-----
quit
 
# Receive:
% Enter PEM-formatted General Purpose certificate.
% End with a blank line or "quit" on a line by itself.
 
# Send:
# Contents of device.crt, followed by newline + 'quit' + newline:
-----BEGIN CERTIFICATE-----
<snip>
-----END CERTIFICATE-----
quit
 
# Receive:
% PEM files import succeeded.
Device(config)#
 
# Send:
Device(config)# crypto pki trustpoint trustpoint1
Device(ca-trustpoint)# revocation-check none
Device(ca-trustpoint)# end
Device#

NETCONF Collection Job

Crosswork Data Gateway supports Network Configuration Protocol (NETCONF) based data collection from network devices.

For NETCONF collection, Crosswork Data Gateway leverages the following device packages that are loaded for the CLI Collection job.

System device packages – system device packages that are downloaded once the Crosswork Data Gateway boots up.
Custom device packages – custom device packages uploaded from UI or API.

Configure NETCONF on devices before you submit NETCONF collection jobs. Refer to the platform-specific documentation.

NETCONF collector supports two types of data collection:

Pull-based collection

Supports cadence-based collection and on-demand collection.

Note

NETCONF command-based collection is not supported.

Event-based collection

Supports NETCONF event notifications as mentioned in https://tools.ietf.org/html/rfc5277 document. On-demand collection is not supported for this type of collection and the cadence specified for these collection jobs is ignored.

NETCONF Collection Job Workflow

NETCONF collection job is submitted to the collection service (Helios/Magellan) specifying the cadence or number of collections requested or with the event notification RPC.
The collection service (Helios/Magellan) sends collection job to Crosswork Data Gateway’s NETCONF collector.
Depending on type of collection, that is event-based or pull-based collection, NETCONF collector initiates collection from the device.
The collected data is forwarded to specified data destinations (gRPC/Kafka).

Sample payload:

{
  "createUpdateJob": {
    "jobId": {
      "deviceId": "6fa90381-95f3-4a95-ac32-37754e002225",
      "sensorPath": {
        "netconfSensor": {
          "devicePackage": {
            "devicePackageName": "optical_inventory_svo_mne",
            "functionName": "getRawNodeInfo"
          }
        }
      },
      "collectionType": "PERSISTENT_COLLECTION_TYPE"
    },
    "collectionType": "PERSISTENT_COLLECTION_TYPE",
    "deviceId": "6fa90381-95f3-4a95-ac32-37754e002225",
    "sensorConfig": {
      "sensorPath": {
        "netconfSensor": {
          "devicePackage": {
            "devicePackageName": "optical_inventory_svo_mne",
            "functionName": "getRawNodeInfo"
          }
        }
      },
      "cadenceInMillisec": "60000"
    },
    "destinationSensorConfigs": [
      {
        "jobDestinationId": {
          "destinationId": "6dbc2a4c-e827-438f-9bab-bbeb508c06e2",
          "destinationContextId": "NativeNetconfTopic"
        },
        "destinationId": "6dbc2a4c-e827-438f-9bab-bbeb508c06e2",
        "destinationContextId": "NativeNetconfTopic",
        "sensorConfigHandler": {
          "action": "NORMAL"
        },
        "applicationContext": [
          {
            "applicationId": "EPNM-APP",
            "contextId": "Native-Netconf"
          }
        ]
      }
    ]
  }
}

Create a Collection Job from Cisco Crosswork UI

Follow the steps to create a collection job:

Note

Collection jobs created through the Cisco Crosswork UI page can only be published once.

Before you begin

Ensure that a data destination is created (and active) to deposit the collected data. Also, have details of the sensor path and MIB that you plan to collect data from.

Procedure

Step 1

From the main menu, go to Administration > Collection Jobs.

Step 2

In the Collection Jobs pane on the left hand side, click Add icon button.

Step 3

In the Job details page, enter values for the following fields:

Application ID: A unique identifier for the application.
Context: A unique identifier to identify your application subscription across all collection jobs.
Collector Type: Select the type of collection - CLI or SNMP.

Click Next.

Step 4

Select the devices from which the data is to be collected. You can either select based on device tag or manually. Click Next.

Step 5

(Applicable only for CLI collection) Enter the following sensor details:

Select data destination from Select Data Destination drop-down.
Select sensor type from Sensor Types pane on the left.

If you selected CLI PATH, Click Add icon button and enter the following paramters in the Add CLI Path dialog box:

Collection Cadence: Push or poll cadence in seconds.
Command: CLI command

Topic: Topic associated with the output destination.

Note

Topic can be any string if using an external gRPC server.

If you selected Device Package, click Add icon button and enter values for the following parameters in the Add Device Package Sensor dialog box:

Collection cadence: Push or poll cadence in seconds.
Device Package Name: Custom XDE device package ID used while creating device package.
Function name: Function name within custom XDE device package.
Topic: Topic associated with the output destination.

Enter Key and String value for the paramters.

Click Save.

Step 6

(Applicable only for SNMP collection) Enter the following sensor details:

Select data destination from Select Data Destination drop-down.
Select sensor type from Sensor Types pane on the left.

If you selected SNMP MIB, Click Add icon button and enter the following parameters in the Add SNMP MIB dialog box:

Collection Cadence: Push or poll cadence in seconds.
OID
Operation: Select the operation from the list.
Topic: Topic associated with the output destination.

If you selected Device Package, click Add icon button and enter values for the following parameters in the Add Device Package Sensor dialog box:

Collection Cadence: Push or poll cadence in seconds.
Device Package Name: Custom device package ID used while creating device package.
Function name: Function name within custom device package.
Topic: Topic associated with the output destination.

Enter Key and String value for the paramters.

Click Save.

Step 7

Click Create Collection Job.

Note

When a collection job is submitted for an external Kafka destination i.e., unsecure Kafka, the dispatch job to Kafka fails to connect. The error seen in collector logs is org.apache.kafka.common.errors.TimeoutException: Topic cli-job-kafka-unsecure not present in metadata after 60000 ms. In Kafka logs, the error seen is SSL authentication error "[2021-01-08 22:17:03,049] INFO [SocketServer brokerId=0] Failed authentication with /80.80.80.108 (SSL handshake failed) (org.apache.kafka.common.network.Selector).

This happens because port is blocked on external Kafka VM. You can use the following command to check if port is listening on Kafka docker/server port:

netstat -tulpn

Fix the problem on the Kafka server and restart the Kafka server process.

Monitor Collection Jobs

You can monitor the status of the collection jobs currently active on all the Crosswork Data Gateway instances enrolled with Cisco Crosswork from the Collection Jobs page.

In the Cisco Crosswork UI, from the left navigation bar, choose Administration > Collection Jobs.

The Collection Jobs pane shows the list of all active collection jobs along with their Status, App ID, and Context ID. The Job Details pane shows the details of all collection tasks associated with a particular job in the Collection Jobs pane. The overall status of the Collection job in the Collection Jobs pane is the aggregate status of all the collection tasks in the Jobs Details pane.

When you select a job in the Collection Jobs pane, the following details are displayed in the Job Details pane:

Application name and context associated with the collection job.

Status of the collection job.

Note

The status of a collection task associated with a device after it is attached to a Crosswork Data Gateway, is Unknown.
A job could have staus as Unknown for one of the following reasons:
- Crosswork Data Gateway has not yet reported its status.
- Loss of connection between Crosswork Data Gateway and Cisco Crosswork.
- Crosswork Data Gateway has received the collection job, but actual collection is still pending. For example, traps are not being sent to Crosswork Data Gateway southbound interface, or device is not sending telemetry updates.
After the collection job is processed, the status changes to 'Successful' if the processing was successful or else it changes to 'Failed'.
If a collection job is in degraded state, one of the reasons might be that the static routes to the device have been erased from Crosswork Data Gateway.
Cisco Crosswork Health Insights - KPI jobs must be enabled only on devices mapped to an extended Crosswork Data Gateway VM. Enabling KPI jobs on devices that are mapped to a standard Crosswork Data Gateway VM reports the collection job status as Degraded and the collection task status as Failed in the Jobs Details pane.

Job configuration of the collection job that you pass in the REST API request. Click icon next to Config Details to view the job configuration. Cisco Crosswork lets you view configuration in two modes:
- View Mode
- Text Mode
Collection type
Time and date of last modification of the collection job.
Collections (x): x refers to requested input collections that span device by sensor paths. The corresponding (y) Issues is the count of input collections that are in UNKNOWN or FAILED state.
Distributions (x): x refers to requested output collections that span device by sensor paths. The corresponding (y) Issues is the count of output collections that are in UNKNOWN or FAILED state.

Cisco Crosswork also displays the following details for collections and distributions:


Field	Description
Collection/Distribution Status	Status of the collection/distribution. It is reported on a on change basis from Crosswork Data Gateway. Click next to the collection/distribution status for details.
Hostname	Device hostname with which the collection job is associated.
Device Id	Unique identifier of the device from which data is being collected.
Sensor Data	Sensor path Click to see collection/distribution summary. From the sensor data summary pop up you can copy the sensor data by clicking Copy to Clipboard. Click to see collection/distribution metrics summary. The metrics are reported on cadence-basis i.e., once every 10 minutes by default. It shows the following metrics for a collection: last_collection_time_msec total_collection_message_count last_device_latency_msec last_collection_cadence_msec It shows the following metrics for a collection: total_output_message_count last_destination_latency_msec last_output_cadence_msec last_output_time_msec total_output_bytes_count
Destination	Data destination for the job.
Last Status Change Reported Time	Time and date on which last status change was reported for that device sensor pair from Crosswork Data Gateway

Note

Create Failed error means out of N devices, some devices failed to setup. However, the collection would happen on the devices that were successfully setup. You can identify the device(s) causing this error by using Control Status API.
If job creation failed on a particular device because of NSO errors, after fixing NSO errors , you have to manually change the administration state of the device first to "Down" and then "Up". However, doing so resets the collection on the device.

Note

Create/Delete failed errors are shown in a different screen pop up. Click next to the job status to see details of the error.

You may also try recreating the job using PUT collection job API with the same payload.

Collection Status for Event-based collection jobs

When data collection is successful, status of the Collection job changes from Unknown to Success in the Collection Jobs pane.
When a device is detached from the Crosswork Data Gateway, all corresponding collection jobs are deleted and collection job status is displayed as Success in the Collection Jobs pane. There are no devices or collection tasks displayed in the Job Details pane.
When a device is attached to a Crosswork Data Gateway, Crosswork Data Gateway receives a new collection job with the status set to Unknown that changes to Success after receiving events from the device.
If the device configuration is updated incorrectly on a device that is already attached to a Crosswork Data Gateway and after the Crosswork Data Gateway has received the job and events, there is no change in status of the collection task in the Jobs Details pane.
If the device inventory is updated with incorrect device IP, the collection task status in the Jobs Details pane is Unknown as expected.

Delete a Collection Job

System and Crosswork Change Automation and Health Insights collection jobs should not be deleted as it will cause issues. Only external collection jobs should be deleted from the Collection Jobs page.

Follow the steps to delete a collection job:

Procedure

Step 1	Go to Administration > Collection Jobs.
Step 2	In the Collection Jobs pane on the left hand side, select the collection job that you want to delete.
Step 3	Click .
Step 4	Click Delete when prompted for confirmation.

Troubleshoot Crosswork Data Gateway

You can troubleshoot the Crosswork Data Gateway from the UI or from the Interactive Console of the Crosswork Data Gateway VM.

This section explains the various troubleshooting options that are available from the Cisco Crosswork UI.

For details on troubleshooting options available from the Interactive Console of the Crosswork Data Gateway VM, see Troubleshooting Crosswork Data Gateway VM.

Check Connectivity to the Destination

To check connectivity to a destination from the Cisco Data Gateway, use the Ping and Traceroute options from Troubleshooting Menu.

Note

Ping traffic should be enabled on the network to ping the destination successfully.

Go to Administration > Data Gateway Management > Data Gateways.
Click the Cisco Crosswork Data Gateway name from which you want to check the connectivity.
In the Crosswork Data Gateway details page, on the top right corner, click Actions and choose: Ping or Traceroute.
- Ping - Enter details in the Number of Packets, and Destination Address fields and click Ping.
- Traceroute - Enter the Destination Address, and click Traceroute.
If the destination is reachable, Cisco Crosswork displays details of the Ping or Traceroute test in the same window.

Download Service Metrics

Use this procedure to download the metrics for all collection jobs for a Crosswork Data Gateway from the Cisco Crosswork UI.

Procedure

Step 1

Go to Administration > Data Gateway Management > Data Gateways.

Step 2

Click the Crosswork Data Gateway name for which you want to download the service metrics.

Step 3

In the Crosswork Data Gateway details page, on the top right corner, click Actions > Download Service Metrics.

Step 4

Enter a passphrase.

Note

Ensure that you make a note of this passphrase. This passphrase will be used later to decrypt the file.

Step 5

Click Download Service Metrics. The file is downloaded to the default download folder on your system in an encrypted format.

Step 6

After the download is complete, run the following command to decrypt it:

Note

In order to decrpyt the file, you must use openssl version 1.1.1i. Use the command openssl version to check the openssl version on your system.

openssl enc -d -AES-256-CBC -pbkdf2 -md sha512 -iter 100000 -in <service metrics file> -out <decrypted filename> -pass pass:<encrypt string>

Download showtech Logs

Follow the steps to download showtech logs from Cisco Crosswork UI:

Note

Showtech logs cannot be collected from the UI if the Crosswork Data Gateway is in an ERROR state. In the DEGRADED state of the Cisco Crosswork Data Gateway, if the OAM-Manager service is running and not degraded, you will be able to collect logs.

Procedure

Step 1

Go to Administration > Data Gateway Management > Data Gateways.

Step 2

Click the Crosswork Data Gateway name for which you want to download showtech.

Step 3

In the Crosswork Data Gateway details page, on the top right corner, click Actions and click Download Showtech.

Step 4

Enter a passphrase.

Note

Ensure that you make a note of this passphrase. You will need to enter this passphrase later to decrypt the showtech file.

Step 5

Click Download Showtech. The showtech file downloads in an encrypted format.

Note

Depending on how long the system was in use, it may take several minutes to download the showtech file.

Step 6

After the download is complete run the following command to decrypt it:

Note

In order to decrpyt the file, you must use OpenSSL version 1.1.1i. Use the command openssl version to check the openssl version on your system.

To decrypt the file on a MAC, you must install OpenSSL 1.1.1+. This is because LibreSSL's openssl command does not support all the switches supported by OpenSSL's openssl command.

openssl enc -d -AES-256-CBC -pbkdf2 -md sha512 -iter 100000 -in <showtech file> -out <decrypted filename> -pass pass:<encrypt string>

Reboot Cisco Crosswork Data Gateway VM

Follow the steps to reboot a Crosswork Data Gateway from Cisco Crosswork UI:

Note

Rebooting the Crosswork Data Gateway pauses its functionality until it is up again.

Procedure

Step 1	Go to Administration > Data Gateway Management > Data Gateways.
Step 2	Click the Cisco Crosswork Data Gateway name that you want to reboot.
Step 3	In the Crosswork Data Gateway details page, on the top right corner, click Actions, and click Reboot.
Step 4	Click Reboot Gateway.

Once the reboot is complete, check the operational status of the Cisco Crosswork Data Gateway in the Administration > Data Gateway Management > Virtual Machines window.

Change Log Level of Crosswork Data Gateway Components

Cisco Crosswork UI offers the option to change the log level of a Crosswork Data Gateway's components, for example collectors (cli-collector) and infra services (oam-manager). Log level changes apply only to the Crosswork Data Gateway on which you are making the change.

Note

Changing the log level for offload services is not supported.

Procedure

Step 1

Go to Administration > Data Gateway Management > Data Gateways.

Step 2

Click the Crosswork Data Gateway name on which you wish to change the log level for the collectors of Crosswork Infrastructure services.

Step 3

In the Crosswork Data Gateway details page, on the top right corner, click Actions > Change Log Level.

The Change Log Level window appears, indicating the current log level of each container service.

Step 4

Select the check box of the container service for which you wish to change the log level.

Step 5

From the Change Log Level drop-down list at the top of the table, select a log level from Debug, Trace, Warning, Info and Error.

Note

To reset the log level of all logs to the default log level (Info), click Reset to Default.

Step 6

Click Save to save the log level change.

After you click Save, a UI message appears indicating that the log level of the component was changed successfully.

Bias-Free Language

Results

Chapter: Cisco Crosswork Data Gateway

Cisco Crosswork Data Gateway

Overview of Cisco Crosswork Data Gateway

Crosswork Data Gateway UI Overview

Set Up Crosswork Data Gateway to Collect Data

Crosswork Data Gateway High Availability with Pools

Create a Cisco Crosswork Data Gateway Pool

Before you begin

Procedure

Attach Devices to a Crosswork Data Gateway

Before you begin

Procedure

Manage Crosswork Data Gateway Post-Setup

Monitor Crosswork Data Gateway Health

Manage a Crosswork Data Gateway Pool

Before you begin

Procedure

Manage Cisco Crosswork Data Gateway Device Assignments

Procedure

Maintain Crosswork Data Gateway VMs

Change the Administration State of Cisco Crosswork Data Gateway VM

Procedure

Delete Cisco Crosswork Data Gateway VM from Cisco Crosswork

Before you begin

Procedure

Redeploy a Crosswork Data Gateway VM

Create and Manage External Data Destinations

Licensing Requirements for External Collection Jobs

Add/Edit a Data Destination

Before you begin

Procedure

What to do next

Delete a Data Destination

Before you begin

Procedure

Manage Custom Software Packages

Add a Custom Software Package

Before you begin

Procedure

What to do next

Delete a Custom Software Package

Procedure

Manage Crosswork Data Gateway Collection Jobs

Types of Collection Jobs

CLI Collection Job

SNMP Collection Job

SNMP Traps Collection Job

Enabling Traps forwarding to external applications

MDT Collection Job

Syslog Collection Job

Syslog Data Collection

Syslog Collection Job Output

Configure Syslog (Non-Secure) on Device

Configure Secure Syslog on Device

Download Syslog Certificates

Configure Cisco Crosswork Trustpoint on Device

gNMI Collection Job

Enable Secure gNMI communication between Device and Crosswork Data Gateway

Generate Device Certificates

Configure gNMI Certificate

Procedure

Update Protocol on Device from Cisco Crosswork

Sample Device Configuration - gNMI

Import Certificates on Device

NETCONF Collection Job

Create a Collection Job from Cisco Crosswork UI

Before you begin

Procedure

Monitor Collection Jobs

Collection Status for Event-based collection jobs

Delete a Collection Job

Procedure

Troubleshoot Crosswork Data Gateway

Check Connectivity to the Destination

Download Service Metrics

Procedure

Download showtech Logs