Cisco Crosswork Situation Manager 7.3.x Administrator Guide

Powered by Moogsoft AIOps 7.3

 

 

Contents

Administrator Guide  4

Apply a Cisco Crosswork Situation Manager License  4

Reporting and Dashboards  4

Insights  4

Grafana Dashboards  5

Manage User Access  11

Manage Roles  11

Manage Users  14

Manage Teams  15

Configure Data Flow   16

Data Processing Flow   17

Workflow Engine  19

Action States  57

Transitions  58

Auto Close  58

Configure Clustering Algorithms  61

Configure Cookbooks and Recipes  61

Configure Tempus  75

Set Up Vertex Entropy  76

Import/Export Algorithm Configurations  78

Configure Merge Groups  79

Troubleshooting and Diagnostic Tools  83

Implement Generic Server Tools  83

Implement Situation Server Tools  84

Implement Alert Server Tools  87

Implement Client Tools  89

Configure Hotkeys  94

Configure ChatOps Shortcuts  95

Additional System Configuration  96

Create Shared Alert and Situation Filters  96

Configure the Landing Page  97

Customize User Experience and Workflow   97

Create Link Definitions  100

Configure Alert and Situation Columns  101

Configure and Retrain Probable Root Cause  103

Statistics Collection  103

Configure Labs Features  104

Schedule Maintenance Downtime  104

View Maintenance Windows  104

Create a Maintenance Window   105

Situation Visualization  105

Similarity diagrams  106

Alert list 107

Self Monitoring  107

Package States  107

Controls  108

Self Monitoring Tabs  108

Processing Metrics  108

Using Processing Metrics  109

Event Processing  110

Web Services  111

Event Ingestion  113

Message Bus  113

Configuration  115

Obtaining Documentation and Submitting a Service Request 116

Cisco Trademark  118

Cisco Copyright 118

 

 

 

NOTE: Available paragraph styles are listed in the Quick Styles Gallery in the Styles group on the Home tab. Alternatively, they can be accessed via the Styles window (press Alt + Ctrl + Shift + S).

Administrator Guide

The Administrator Guide contains topics about the various administrator functions you can perform in Cisco Crosswork Situation Manager.

Apply a Cisco Crosswork Situation Manager License

When you purchase Cisco Crosswork Situation Manager, Cisco provides you with a license key. After you install Cisco Crosswork Situation Manager, you can maintain your license key in the System Settings UI.

Navigate to Settings > System > Licensing to apply a new license or to view details of your current license. You can set up a system notification to alert you within 1 to 120 days of a pending license expiration.

To add a new license key, copy and paste the license text into the Add New License Key text box and click Update License. A green panel confirms "Your new license key has been applied" when you enter a valid license.

Reporting and Dashboards

You can take advantage of Cisco Crosswork Situation Manager Insights to analyze trends in operational performance. You can use the default dashboard or you can use Grafana to create a custom dashboard.

Insights

Insights is built on the Stats API that exposes time-series data so you can report on:

·         Active Situations, the teams they're assigned to, and the services they impact.

·         Reoccurring Situations that could indicate deeper systemic issues.

·         Key performance indicators like Mean Time To Resolution.

You can use the reporting tool of your choice to take advantage of the /document/preview/105323#UUID6284d1ff884d9ef2f768f29f3e17ac89. Otherwise, check out the Cisco Crosswork Situation Manager app for Grafana to view and modify the default dashboard. See Grafana Dashboards for more information.

Insights exposes a variety of statistics and metrics to help you understand your operations. For example, consider the valuable data in the default dashboard:

You can see how your teams are managing active Situations:

·         View the number of open Situations system-wide and see how many open Situations are unassigned, unacknowledged or that have been reassigned.

·         Identify if the number of open reoccurring Situations which can highlight areas of impact that need increased attention or resource allocation.

You can monitor the distribution of your Situations over time, to see which teams handle the most Situations and which services are most impacted by Situations. Key Performance Indicator metrics reveal how quickly Cisco Crosswork Situation Manager detects Situations and how quickly teams address open Situations over time:

·         Mean Time To Detect: the mean time to detect a Situation from the first event time.

·         Mean Time To Acknowledge: the mean time to acknowledge a Situation from the first event time.

·         Mean Time To Resolution: the mean time to resolve a Situation from the first event time.

For details on all the available Insights, see the Stats API.

Grafana Dashboards

You can view Cisco Crosswork Situation Manager statistics and reports in dashboards using Grafana.

To set this up, you need to install Grafana, install the Moogsoft AIOps plugin and install the Grafana integration in Cisco Crosswork Situation Manager.

Before You Begin

Before you install the Moogsoft AIOps plugin, ensure you have met the following requirements:

·         You have installed Grafana or you have a hosted instance of Grafana.

·         Enable HTTPS if you are using an on-premises instance of Grafana. See the Grafana docs for how to edit the protocol, cert_file, and cert_key properties in the Grafana configuration .ini file.

·         The port for Cisco Crosswork Situation Manager is open and accessible from Grafana.

Install the Moogsoft AIOps App

To install the Moogsoft AIOps app in Grafana, follow these steps:

·         Install the app.

·         Find it under Apps in your Grafana plugins and enter the following settings:

·         Enable the app. A "Test Success" message appears if successful.

After you have set up the app, you can configure your dashboards.

Default Dashboards

You can configure the statistics that Cisco Crosswork Situation Manager collects depending on which dashboards you want to use. See Configure Labs Features for more details. You can also create a custom dashboard.

Global Situation Overview

The Global Situation Overview dashboard provides a broad overview of your Situation statistics, teams insights, and mean times to acknowledge, detect and resolve.

The dashboard's panels display the number of open Situations, unassigned Situations, reassigned Situations and recurring Situations. Other panels include the top 10 teams by open Situations, the top 10 services by open Situations, the number of Situations by status, the number of Situations by severity and a graph view of MTTA, MTTD and MTTR.

To edit the dashboard, click the header of any panel and edit the statistic endpoint or add a query. Alternatively, click Add Row at the bottom of the screen.

Team Situation Overview

The Team Situation Overview dashboard displays a broad overview of your team's Situation statistics, team insights and mean time to acknowledge and resolve.

The dashboard's panels display the number of reassigned Situations per team, the number of reoccuring Situations per team, number of Situations impacting each service per team, the number of Situations by status per team and the MTTA/MTTR for the team.

To edit the dashboard, click the header of any panel and edit the statistic endpoint or add a query. Alternatively, click Add Row at the bottom of the screen.

Team Workload Breakdown

The Team Workload dashboard provides an overview of how an individual team is performing in Cisco Crosswork Situation Manager.

The dashboard's panels pull statistical data about the MTTA/MTTR per team, the status of the team's Situations, and the number of comments made by the team.

To edit the dashboard, click the header of any panel and edit the statistic endpoint or add a query. Alternatively, click Add Row at the bottom of the screen.

Noise Reduction

The Noise Reduction dashboard displays an overview of the noise reduction performance of Cisco Crosswork Situation Manager.

This dashboard shows statistics including the number of accumulated events reduced into alerts and Situations over a period of time, the percentage reduction of events to alerts, the percentage reduction of alerts to Situations and the overall reduction.

To edit the dashboard, click the header of any panel and edit the statistic endpoint or add a query. Alternatively, click Add Row at the bottom of the screen.

Individual Stats Overview

The Individual Stats Overview dashboard allows you to view and compare statistics for multiple Cisco Crosswork Situation Manager users.

The dashboard includes Situation metrics, MTTA and MTTR, user activity, Situation stats by user, user performance overview and user activity overview.

By default, Cisco Crosswork Situation Manager collects statistical data for this dashboard for all users with the Operator role. You can add the 'collect_insights' permission to other roles if you want to include other users in the dashboard. See Role Permissions for more information.

Individual User Deep Dive

The Individual User Deep Dive dashboard provides an in-depth summary of different performance metrics for a single user.

The dashboard includes a user Situation metric overview, a user activity overview, a user performance overview, Situation stats by users, user activity overview and user performance overview. You can view a breakdown of specific statistics such as the number of Situations the user has acknowledged, assigned, closed, reassigned and how many they have open. You can also see which ChatOps tools the user has executed, the number of comments they have made, the number of invitations they have received, the number of alerts they have marked with PRC and the individual's MTTA and MTTR.

By default, Moogsoft AIOps collects statistical data for this dashboard for all users with the Operator role. You can add the 'collect_insights' permission to other roles if you want to include other users in the dashboard. See Role Permissions for more information.

Create a Custom Dashboard

You can add and configure custom dashboards to display different Cisco Crosswork Situation Manager statistics in Grafana. For more information see the Grafana documentation. To create a dashboard, follow these steps:

·         Log in to your Grafana instance.

·         Click + and Dashboard.

·         Configure the dashboard to meet your requirements. For more information on the available API endpoints and statistics see the Stats API.

Once created, statistics from Cisco Crosswork Situation Manager appear in the dashboard.

Manage User Access

As an Administrator, you control user access to functions in the Cisco Crosswork Situation Manager UI. This ensures that authorized users can perform required functions and prevents unauthorised users from accessing the system and sensitive operations within it.

Functions you can restrict include the ability to assign alerts, assign Situation owners, mark alerts with Probable Root Cause (PRC) feedback, and access Integrations.

Follow these steps to manage user access:

·         Create roles for specific job functions.

·         Assign the permissions to perform operations to roles.

·         Enable user authentication via SAML or LDAP, or manually create users.

·         Specify a role for authenticating users, or manually assign roles to users.

·         Set up teams (optional).

Manage Roles

Roles group the permissions that users need to perform a set of tasks within Cisco Crosswork Situation Manager. You can create roles for specific job functions and assign them the permissions to perform certain operations. For example, you could create a role named Situation Manager and assign it the permissions to perform certain operations, such as managing alerts and Situations.

You must assign at least one role to every user. You can do this manually when you create the user or you can map roles to users if you use SAML.

Cisco Crosswork Situation Manager contains a set of predefined roles with a predefined set of permissions. See Role Permissions for details.

Create, Edit and Delete Roles

To view a list of roles, navigate to Settings > Roles. You can use the search box on the left to filter the list.

Select a role to view and edit its configuration. You can edit the following:

·         Selected permissions: See Role Permissions for a description of each permission.

·         Session timeout: You can use the system timeout of 60 minutes or define a custom timeout period.

·         Landing page: You can choose to inherit the landing page from the system configuration or select an alternative page.

You can also perform the following operations:

·         Click + to create a new role.

·         Click a role name and then - to delete a role.

You cannot delete a role that is assigned to users. Remove the role from all users first.

Role Permissions

Cisco Crosswork Situation Manager contains the following predefined roles: Super User, Administrator, Manager, Operator, Customer, and Grazer. The REST LAM Sender role is designed for use with REST LAM integrations. The role restricts UI functionality, so you should not assign it to UI users. The default permissions selected for these roles are as follows:

Manage Users

As with most software systems, you use user credentials to provide secure access to Cisco Crosswork Situation Manager for your personnel. You can use the System Settings UI to manage the various attributes that define users and the actions they are allowed to perform inside Cisco Crosswork Situation Manager.

As an alternative to managing users in the UI, you can configure the system to allow Single Sign-On (SSO) via the Security Assertion Markup Language (SAML) protocol. If you have a large number of users, enabling SSO saves you from setting them up individually. It also improves security by requiring users to remember a single complex password instead of multiple credentials for multiple systems.

Within the SAML configuration you can specify a role, primary group and team to assign to users when they authenticate for the first time. See Configure Single Sign-On with SAML for more information. You can also authenticate users with Lightweight Directory Access Protocol (LDAP). See Configure Single Sign-On with LDAP for more information.Configure Single Sign-On with SAMLConfigure Single Sign-On with LDAP

Manually Create and Edit Users

To view a list of users, navigate to Settings > Users. Use the search box on the left to filter the list. You can click the person icon to toggle the display of inactive users. Click + to create a new user or select a user to view and edit their attributes.

You cannot delete users. The system retains a history of all user activity, including collaboration posts and ownership of alerts and Situations. You can set obsolete users to inactive in the Personal tab. Cisco Crosswork Situation Manager includes the following predefined users:

·         Administrator: Super user role. For information on creating and editing roles, see Manage Roles.

·         Graze API user. Grazer role. This user is intended for system integration purposes, it is not a UI user.

·         System Owner: Super user role.

·         Moog: An anonymous system account used for unassigned alerts and Situations.

We recommend that you change the default password for each predefined user. Once you have set up your own users or enabled user authentication you may wish to deactivate the predefined users.

Edit User Details

Navigate to the Personal tab to view or edit the following user details:

·         Username with 32 characters maximum. Mandatory.

·         Full name.

·         Password. If you do not enter a password the user is created as an LDAP user.

·         Primary group. Mandatory.

·         Department.

·         Time zone if different to the system time zone.

·         Active status. Inactive users cannot log into the UI. New users are active by default.

·         Timeout. You can use the role timeout of 60 minutes or define a custom timeout period between 60 and 720 minutes (12 hours).

You can view or edit the user's email address or telephone number on the Contact tab.

Manage a User's Roles

Roles group the permissions users need to perform a set of tasks within Cisco Crosswork Situation Manager. See Manage Roles for information on creating and editing roles. You must assign at least one role to each user.

Assign a User to Teams

You can optionally group users into teams, to ensure that users working together view the Situations that are relevant to them. You can configure Cisco Crosswork Situation Manager to assign Situations to a particular team if they impact selected services or meet other criteria. See Manage Teams for further information.

Manage Teams

You can use the optional Teams feature in Cisco Crosswork Situation Manager to allow users working together to view the Situations that are relevant to them. You can configure the system to automatically create teams based on certain Situation data, or you can manually create teams.

Create, Edit and Delete Teams

To view a list of teams, navigate to Settings > Teams. Use the search box on the left to filter the list. You can click the people icon to toggle the display of inactive teams.

·         Click + to create a new team.

·         Click a team name and then the copy icon to duplicate a team.

To create a team, you must assign the team a name with a maximum of 64 characters. You can optionally provide a description of the team.

By default, teams are active. Deselect the Active checkbox to deactivate a team. Inactive teams don't appear in team rooms and cannot have Situations assigned to them.

Navigate to the Users tab to select the users that belong to this team. You can define an alternative landing page for the team members on the Settings tab.

To delete a team, select it and click -. When you delete a team Cisco Crosswork Situation Manager removes it from any alerts, Situations, services and notifications that it was assigned to or associated with. The system retains a record of all team activity. When you view a Situation assigned to a deleted team, the team name is shown with an "inactive" label.

If a deleted team is recreated by SAML/LDAP, it is assigned a status of inactive.

Create Automatic Teams

You can configure Cisco Crosswork Situation Manager to create teams based on current Situation data.

Note:

Creating automatic teams sets all existing teams to inactive.

You can create teams based on any of the following Situation fields:

·         Description

·         Services Impacted

·         Process Impacted

·         Queue

After Cisco Crosswork Situation Manager creates a team, you can view and edit the team settings and membership.

Configure Team Permissions

You can filter data visible to team members according to services, Situations or alerts:

·         Create a Service Filter in General to select services for which the team views affected Situations. Adding more than 200 services may affect system performance and stability.

·         Create a Situation Filter in General to select the Situations that you can assign to members of the team. If you only want team members to be able to see these Situations, ensure they do not have the 'all_data' permission. See Role Permissions for details.

·         Create an Alert Filter in Settings to select the alerts outside the team's Situation Filter that you want the team members to see.

Permissions are additive. Therefore, if you set several filters, Situations and alerts that meet any one of them are included. For example, you could set up a team's permissions so that the team views critical Situations affecting a database service, a messaging service, and a web service.

The all_data permission in a user role overrides the filters in team settings. Users with the 'all_data' permission can view all Situations and alerts.

If you want users to see only the alerts and Situations that are assigned to their team, configure roles and users as follows:

·         Create a role without the all_data permission.

·         Assign the role to the users.

·         Add the users to the team.

See Manage Roles and Manage Users for more information.

Configure Data Flow

As an administrator, you can configure the the data flow for events, alerts, and Situations to take advantage of the data processing features within Cisco Crosswork Situation Manager.

Data Processing Flow describes how your data enters the system as an event and progresses through the system as alert and, later, Situation data in Cisco Crosswork Situation Manager.

The Workflow Engine lets you apply custom logic to event, alert, and Situation processing within the Cisco Crosswork Situation Manager UI. You can check conditions against object data, modify object data, and control object routing to meet your data processing requirements.

You can also use the Auto Close feature to define criteria for automatically closing alerts and Situations. This housekeeping feature closes alerts and Situations that no longer require action in a timely manner.

If you upgraded from a previous version, you may have data processing configurations that use the Alert Rules Engine. The Alert Rules Engine lets you define criteria to process alerts according to different Transitions to move these alerts to different Action States. Before you start an implementation with the Alert Rules Engine, see if the Workflow Engine meets your needs.

Data Processing Flow

Before you configure or customize data processing in Cisco Crosswork Situation Manager, take some time to learn the components that comprise the basic flow for processing event, alert, and Situation data.

Except for the Link Access Modules (LAMs), the rest of the data processing components are individual moolets that run as part of the Moogfarmd. For more information, see Configure Data Processing.

A) LAMs / Data Ingestion

The LAMs or Integrations ingest raw event data from your monitoring sources and map them into Cisco Crosswork Situation Manager events.

See Introduction to Integrations for more information.

B) Event Workflow Engine

The Event Workflow Engine listens for events on the message bus and processes them based upon any active workflows.

See Workflow Engine for an overview of how the Workflow Engine UI works. See Workflow Engine Moolets for information on the Moolet.

C) Alert Builder

The Alert Builder deduplicates events into alerts and calculates the entropy value for alerts.

See Alert Builder for more information.

D) Enricher

The Enricher is an optional moolet that you can use to enrich alert data from external data sources such as a CMDB. See Enrichment for information about the enrichment process.

See Enricher Moolet for information on the Moolet.

E) Enricher Workflow Engine

The Enricher Workflow Engine listens for alerts on the message bus and processes them based upon any active workflows.

See Workflow Engine for an overview of how the Workflow Engine UI works. See Workflow Engine Moolets for information on the Moolet.

F) Maintenance Window Manager

The Maintenance Window Manager prevents alerts from creating Situations during known maintenance downtimes.

To learn how to create a maintenance window, see Schedule Maintenance Downtime. See Maintenance Window Manager for information on the Moolet.

G) Alert Workflow Engine

The Alert Workflow Engine listens for alerts on the message bus after they have passed through the Maintenance Window Manager. It processes alerts based upon any active workflows you have created.

See Workflow Engine for an overview of how the Workflow Engine UI works. See Workflow Engine Moolets for information on the Moolet.

H) Alert Rules Engine

If you upgraded from a previous version, you may have data processing configurations that use the Alert Rules Engine. The Alert Rules Engine lets you define criteria to process alerts according to different Transitions to move these alerts to different Action States. Before you start an implementation with the Alert Rules Engine, see if the Workflow Engine meets your needs.

See Alert Rules Engine for more information.

I) Clustering Algorithms

The clustering algorithms (Sigalisers) in Cisco Crosswork Situation Manager group related alerts into Situations.

See Clustering Algorithm Guide for an overview of the algorithms. To configure a clustering algorithm, see Configure Clustering Algorithms.

J) Situation Manager

The Situation Manager listens for Situation creation, update, and closure actions and lets you automate processes like data enrichment, assignment, or notification to a ticketing system.

The Situation Manager Labeler is part of the Situation Manager. See Situation Manager for more information.

K) Teams Manager

The /document/preview/11763#UUIDebda8c6cc2a65d6a10babbf49a500877 listens for new Situation creation, update, and closure actions. It handles the team assignments you create in the Settings UI. See Manage Teams.

See Teams Manager Moolet for information on the Moolet.

L) Situation Workflow Engine

The Situation Workflow Engine listens for Situations on the message bus after they have passed through the Situation Manager. It processes Situations based upon any active workflows you have created.

See Workflow Engine for an overview of how the Workflow Engine UI works. See Workflow Engine Moolets for information on the Moolet.

Workflow Engine

Implementers and administrators can use a workflow engine to add custom logic for event, alert, and Situation processing in Cisco Crosswork Situation Manager. You can check conditions against object data, modify object data, and control object routing. This allows you to fine-tune your Data Processing Flow. For example you can set up a workflow engine to process and normalize data from a LAM or Integration before it forwards the data to the Alert Builder.

Some scenarios where you can implement the Workflow Engine include:

·         Using the Alert Workflow Engine to escalate alerts. For example to respond to a disk space alert.

·         Using the Event Workflow Engine to prevent them from processing.

·         Enrich alerts with external data such as the services supported by a host or its location.

·         Using the Heartbeat Workflow Engine to detect the absence of events like a missing keep alive event from a predictable source.

·         Controlling stateful workloads. For example, you can configure a workflow to hold a "link down" event until Cisco Crosswork Situation Manager receives a corresponding "link up" event within a time limit.

·         Using the Situation Workflow Engine to integrate with external systems for ticketing, notification, automation, and reporting.

Workflow Engine Types

Each workflow engine is a Moolet that operates within the context of Moogfarmd. See Data Processing Flow for more information. Cisco Crosswork Situation Manager includes the following workflow engines by default:

·         Event Workflows process events data after data after ingestion from a LAM and before the Alert Builder.

·         Enrichment Workflows process alerts after the Alert Builder but before the Maintenance Window Manager.

·         Alert Workflows process alerts after the Maintenance Window Manager and before they pass to clustering algorithms. For example, you can use an alert workflow when you want to route a specific type of alert to a specific clustering algorithm.

·         Situation Workflows process Situations after the Teams Manager. For example, you can use a Situation workflow when you want to integrate with a ticketing system.

Because workflow engines are Moolets, you can use a Moobot to extend the engine functionality. You can also add new workflow engines. For more information, see Workflow Engine Moolets.

Workflows

Each workflow engine is a container for a set of workflows. In general a workflow comprises the following:

·         A workflow definition that defines the workflow purpose, the objects it affects, and some basic functionality.

·         Workflow actions that let you apply functions to object data and apply some routing rules in case the function returns false. There are functions to let you check conditions of object data within the workflow, modify the object data, and route the object within the workflow or to another Moolet.

Workflows execute in numerical order from first to last. You can rearrange the order of the workflows to control the order of execution. Similarly, within a workflow, actions execute in numerical order.

Each action within a workflow lets you control the flow in the case that its function returns false.

Cisco Crosswork Situation Manager ships with the following default workflows:

·         Closed Objects Filters to prevent processing of closed ob in the Alert Workflows and Situation Workflows.

·         Automated Ticketing to help you integrate with third party ticketing systems in the Situation Workflows.

To learn how to create workflows, see Manage Workflows. For information on creating actions within a workflow, see Manage Workflow Engine Actions.

Workflow Engine Strategies and Tips

The Cisco Crosswork Situation Manager Workflow Engine is a powerful tool that gives you access to your event, alert and Situation data and lets you control data processing flow. This topic covers some strategies to help you get the most out of the Workflow Engine.

Before you begin

Taking some time to prepare before you start creating workflows in Cisco Crosswork Situation Manager can help improve your experience. The following include some suggestions to help you get ready:

·         Take time to define outcomes you want to accomplish with the Workflow Engine. Read through the example use cases listed in the Workflow Engine topic to gather ideas.

·         Pick one use case and think about the supporting data that can help you define your workflow. For example, you want to stop processing of clear, severity 0, events if they are going to create a new alert. In this case, you need the severity field. For more information see Alert and Event Field Reference.

·         Look through the Workflow Engine Functions Reference to find the right function you need to accomplish your task. For example, to see if an event will create a new alert, you can use willCreateNewAlert.

·         If possible, set up a REST LAM so you can send sample data to test out your workflows. See REST LAM. You can use cURL or a graphical API client to send event data to the REST LAM.

·         If possible, get SSH access to the machine running Moogfarmd so you can access the log for troubleshooting. See Troubleshoot the Workflow Engine for more information.

Creating workflows

Workflows are containers for a set of actions to process your data. In a complex system, you may create hundreds of workflows to handle all kinds of scenarios. Consider the following as you define your workflows:

·         Workflow Engines process data through individual workflows in the numeric order from the UI. If possible, work with one active workflow at a time while you design and configure your workflow. This way you can focus on a single behavior without worrying about an upstream or downstream workflow.

·         Whenever possible, use an entry filter to limit the number of objects entering your workflow. Running a workflow takes processing power and time. Using an entry filter ensures that only pertinent objects pass through a workflow. Entry filters are also more performant than action based filters. For example, if I want to stop processing clear events, I can create a filter: severity = "Clear". See Filter Search Data for information on filter syntax.

·         If you want to use a sweep up filter to process multiple alerts or Situations, verify that the function you plan to use works with sweep up filters. You can check the Workflow Engine Functions Reference or the topics for individual functions.

·         When objects enter a workflow as part of a sweep up filter, the workflow processes each action on all objects in turn. For example, it executes action one for all objects before proceeding to action two.

·         When you enable multiple workflows, consider the impact of upstream workflows on downstream ones. For example, keep the default Closed Alerts Filter early in the workflow chain. This prevents those events from entering later workflows. Also consider the impact of event workflows on alert and Situation workflows.

See Manage Workflows for steps to create a workflow.

Adding Actions to workflows

You add actions to a workflow to manipulate data and control the data processing flow for events, alerts, enrichment, and Situations. Consider the following as you add actions to your workflow:

·         The UI validates that the data you enter for function arguments as JSON. Where possible, it checks validity of number and string data. For example 0-5 are the only valid severities. It does not check for valid field names or value data.

·         Unless instructed, you do not need to enter quotation marks around values, except strings where they represent elements in an array.

·         The Workflow Engine uses JavaScript regular expression notation. Consider using a third-party regular expression validator to help with syntax. For example, regex101. Cisco does not endorse any specific tool. Before using any tool, verify that it meets your organization's standard for security and privacy.

·         The forwarding behavior applies when the action returns false. For more detail, see Action types and forwarding behavior.

·         Actions execute in numeric order. Test your workflow for both negative and positive cases with the addition of each new action.

See Manage Workflow Engine Actions for steps to add actions to a workflow.

Action types and forwarding behavior

The Workflow Engine has several types of actions to choose from. Each action lets you set the forwarding behavior to control the downstream flow of the object in the case that the action returns false. The type of action should influence how you choose your forwarding behavior. Action types include:

·         Conditionals that return true or false to let you control whether or not to proceed with the current object. You can use these as an "action" based filter if you need to filter data within a workflow after the execution of previous actions. For maximum efficiency, you may consider breaking a workflow into multiple workflows and using the entry filter for the subsequent workflow.

·         Transformers that update object data. In general transformer actions should Always Forward unless doing so compromises the workflow. For example, when a failed action impacts the subsequent action.

·         Data actions that action retrieved data and pass it to a consecutive action or that send data asynchronously to an external data sink. Treat data retrievers like conditionals. For example, stop the workflow if it returns false since subsequent actions may not run. An asynchronous exporter always returns true regardless of the result from the asynchronous return.

Manage Workflows

You can create and configure individual workflows or chains of workflows in the UI.Cisco Crosswork Situation Manager.

Read through Workflow Engine Strategies and Tips for ideas about how to use the Workflow Engine.

To access the Workflow Engine, navigate to Settings > Automation.

When you open the Workflow Engine, you see workflow tabs for the different types of engines you can enable at startup.

Click +Add Worfklow to create a workflow or double-click an existing workflow to open it.

Edit the Workflow Definition as follows to control which data the engine processes:

You can use the slider in the title bar to set the engine to Active or Inactive.

If you have multiple workflows, you can use the up and down arrows in Edit mode to reorder them. Alternatively, you can drag and drop the workflows into order.

After you create an engine, you can add actions to process data. When the engine is active, it processes all objects that match the filtering criteria according to the actions. If a user has manually changed an alert or Situation, this may affect its processing by the Workflow Engine.

Manage Workflow Engine Actions

After you have defined the data you want to process using a Workflow Engine in Cisco Crosswork Situation Manager, you can set up actions to programmatically transform the data and control the data flow.

Read through Workflow Engine Strategies and Tips for ideas about how to use the Workflow Engine.

When you edit an engine, you can click +Add Action to create a new action, or double-click an existing action to edit it.

Delay

By default, each workflow has a delay action. It is mandatory and you can not delete it. You can set the delay for up to 86,400 seconds (24 hours). If you set the Reset option

Actions

Define workflow actions as follows to add custom processing to events, alerts, or Situations depending on the type of engine:

If you have multiple actions, you can drag and drop them to arrange them according to your requirements.

Workflow Engine Functions Reference

This is a reference for Workflow Engine functions in Cisco Crosswork Situation Manager.

Functions may be available for more than one object. For example, addItemToList is available in event, alert, enrichment, and Situation workflows. In this reference, the functions appear in the lists for all the objects they are valid for.

Event functions

The following functions are available in event workflows:

·         addItemToList: Adds an item or items to an array.

·         appendFields: Appends a concatenated set of fields to an existing field, using a separator character.

·         appendString: Appends a static string to an existing field separated by a space character.

·         ceventFilter: Returns true if the object matches a SQL-like filter. Sweep up filter applies.

·         classifyEvent: Sets the class, type, and severity fields of an event based upon its contents using a predefined classification algorithm.

·         concatFields: Sets the value of a field to a string representing a set of concatenated fields.

·         convertToJSON: Converts the object to JSON and adds it to the workflowContext for use in subsequent actions.

·         copyFieldFromAlertToEvent: Copies a single field from an existing alert to a deduplicating event for the same alert.

·         copyFromAlertToEvent: Copies multiple fields from an existing alert to a deduplicating event for the alert.

·         deltaEvent: Returns true: if the specified event fields differ from corresponding fields in an existing alert, or when an error occurs in the delta check, or when no alert exists. Returns false when it detects no changes.

·         dropEvent: Allows you to prevent further processing of an event.

·         existingAlertFilter: Returns true if the existing alert for a deduplicating event matches a SQL-like filter.

·         isClear: Returns true if the object's severity level is Clear (0).

·         isInSubnet: Returns true when an IP address is present within a specified subnet. Sweep up filter applies.

·         isNewerThan: Returns true when the object age in seconds is less than a specified age in seconds. Sweep up filter applies.

·         isNotNull: Returns true if the value for an object's cEvent field is not null, is not an empty object, or is not an empty array.

·         isNull: Returns true if the value for an object's cEvent field is null, is not set, is an empty object, or is an empty array.

·         isOlderThan: Returns true when the object age in seconds is older than a specified age in seconds. Sweep up filter applies.

·         listContains: Returns true when the array field you query contains some of your specified values. Sweep up filter applies.

·         listContainsAll: Returns true when the array field you query contains all of your specified values. Sweep up filter applies.

·         listDoesNotContain: Returns true when the array field you query contains none of your specified values. Sweep up filter applies.

·         logMessage: Logs a message to the Moogfarmd log.

·         logWorkflowDuration: Logs debug messages for the workflow execution duration.

·         prependFields: Prepends a concatenated set of fields to an existing field, using a separator character.

·         prependString: Prepends a string to an existing field, using a separator character.

·         restAsyncPost: Makes a HTTP POST request with a JSON payload to a named REST endpoint.

·         searchAndReplace: Matches a regular expression to an object field and maps the contents of subgroups to other fields. Sweep up filter applies.

·         setCoreEventField: Sets a single core event field to a value.

·         staticLookup: Searches for a key in a static lookup table, retrieves the corresponding value, and applies that value to a field in the object.

·         stripFQDN: Splits a fully qualified domain name (FQDN) into a hostname/short name and a domain name and updates fields with the values.

·         upperCase: Changes the value of a field to uppercase. Sweep up filter applies.

·         willCreateNewAlert: Returns true if the event will create a new alert.

·         willDeduplicateAlert: Returns true if the event will deduplicate into an existing alert.

Alert and enrichment functions

The following functions are available in alert and enrichment workflows:

·         addItemToList: Adds an item or items to an array.

·         alertInSituation: Returns true when the alert is a member of an active Situation. Sweep up filter applies.

·         alertNotInSituation: Returns true when the alert is not a member of an active Situation. Sweep up filter applies.

·         appendFields: Appends a concatenated set of fields to an existing field, using a separator character.

·         appendString: Appends a static string to an existing field separated by a space character.

·         between: Returns true if the object creation date falls between two times.

·         ceventFilter: Returns true if the object matches a SQL-like filter. Sweep up filter applies.

·         concatFields: Sets the value of a field to a string representing a set of concatenated fields.

·         convertToJSON: Converts the object to JSON and adds it to the workflowContext for use in subsequent actions.

·         forward: Forwards the object to the named Moolet.

·         isAssigned: Returns true if the object has an owner or moderator. Sweep up filter applies.

·         isClear: Returns true if the object's severity level is Clear (0).

·         isInSubnet: Returns true when an IP address is present within a specified subnet. Sweep up filter applies.

·         isNewerThan: Returns true when the object age in seconds is less than a specified age in seconds. Sweep up filter applies.

·         isNotAssigned: Returns true if the object does not have an owner or moderator. Sweep up filter applies.

·         isNotNull: Returns true if the value for an object's cEvent field is not null, is not an empty object, or is not an empty array.

·         isNull: Returns true if the value for an object's cEvent field is null, is not set, is an empty object, or is an empty array.

·         isOlderThan: Returns true when the object age in seconds is older than a specified age in seconds. Sweep up filter applies.

·         listContains: Returns true when the array field you query contains some of your specified values. Sweep up filter applies.

·         listContainsAll: Returns true when the array field you query contains all of your specified values. Sweep up filter applies.

·         listDoesNotContain: Returns true when the array field you query contains none of your specified values. Sweep up filter applies.

·         logMessage: Logs a message to the Moogfarmd log.

·         logWorkflowDuration: Logs debug messages for the workflow execution duration.

·         lookupAndReplace: Sets the alertField to a value when one of the fields in the inFields list matches a word or regular expression. Sweep up filter applies.

·         prependFields: Prepends a concatenated set of fields to an existing field, using a separator character.

·         prependString: Prepends a string to an existing field, using a separator character.

·         replaceString: Replaces a string or regular expression in a field with a specified string or regular expression.

·         restAsyncPost: Makes a HTTP POST request with a JSON payload to a named REST endpoint.

·         searchAndReplace: Matches a regular expression to an object field and maps the contents of subgroups to other fields. Sweep up filter applies.

·         sendMooletInform: Sends a Moolet inform with a subject and details.

·         setClass: Sets the class of the alert.

·         setCustomInfoJSONValue: Adds or updates a custom info key to the specified JSON value. Sweep up filter applies.

·         setCustomInfoValue: Adds or updates a custom info key to a specified string value. Sweep up filter applies.

·         setDescription: Sets the description of the object.

·         setSeverity: Sets the severity of the alert. Sweep up filter applies.Severity Reference

·         setType: Sets the type of the alert.

·         staticLookup: Searches for a key in a static lookup table, retrieves the corresponding value, and applies that value to a field in the object.

·         stripFQDN: Splits a fully qualified domain name (FQDN) into a hostname/short name and a domain name and updates fields with the values.

·         upperCase: Changes the value of a field to uppercase. Sweep up filter applies.

Situation functions

The following functions are available in Situation workflows:

·         addItemToList: Adds an item or items to an array.

·         appendFields: Appends a concatenated set of fields to an existing field, using a separator character.

·         appendString: Appends a static string to an existing field separated by a space character.

·         between: Returns true if the object creation date falls between two times.

·         ceventFilter: Returns true if the object matches a SQL-like filter. Sweep up filter applies.

·         checkSituationState: Returns true if the specified state exists for a Situation. Sweep up filter applies.

·         concatFields: Sets the value of a field to a string representing a set of concatenated fields.

·         containsAlertDetails: Returns true if all or any of the alerts in the Situation matches the filter condition. Sweep up filter applies.

·         convertToJSON: Converts the object to JSON and adds it to the workflowContext for use in subsequent actions.

·         createServiceTicket: Creates a ticket for the specified service.

·         forward: Forwards the object to the named Moolet.

·         hasCausalPRC: Returns true if one or more alerts in the Situation has a causal PRC flag set. Sweep up filter applies.

·         hasMerged: Returns true if the Situation has been merged or superseded.

·         hasNotMerged: Returns true if the Situation has not been merged or superseded.

·         hasSimilarSituations: Returns true when the Situation has a similar Situation above the specified threshold.

·         isAssigned: Returns true if the object has an owner or moderator. Sweep up filter applies.

·         isClear: Returns true if the object's severity level is Clear (0).

·         isNotAssigned: Returns true if the object does not have an owner or moderator. Sweep up filter applies.

·         isNewerThan: Returns true when the object age in seconds is less than a specified age in seconds. Sweep up filter applies.

·         isNotNull: Returns true if the value for an object's cEvent field is not null, is not an empty object, or is not an empty array.

·         isNull: Returns true if the value for an object's cEvent field is null, is not set, is an empty object, or is an empty array.

·         isOlderThan: Returns true when the object age in seconds is older than a specified age in seconds. Sweep up filter applies.

·         labelSituation: Labels the Situation using the Situation Manager Labeler macro language. Sweep up filter applies.

·         listContains: Returns true when the array field you query contains some of your specified values. Sweep up filter applies.

·         listContainsAll: Returns true when the array field you query contains all of your specified values. Sweep up filter applies.

·         listDoesNotContain: Returns true when the array field you query contains none of your specified values. Sweep up filter applies.

·         logMessage: Logs a message to the Moogfarmd log.

·         logWorkflowDuration: Logs debug messages for the workflow execution duration.

·         prependFields: Prepends a concatenated set of fields to an existing field, using a separator character.

·         prependString: Prepends a string to an existing field, using a separator character.

·         replaceString: Replaces a string or regular expression in a field with a specified string or regular expression.

·         restAsyncPost: Makes a HTTP POST request with a JSON payload to a named REST endpoint.

·         searchAndReplace: Matches a regular expression to an object field and maps the contents of subgroups to other fields. Sweep up filter applies.

·         sendMooletInform: Sends a Moolet inform with a subject and details.

·         setCustomInfoJSONValue: Adds or updates a custom info key to the specified JSON value. Sweep up filter applies.

·         setCustomInfoValue: Adds or updates a custom info key to a specified string value. Sweep up filter applies.

·         setDescription: Sets the description of the object.

·         setSituationState: Sets the state of the Situation. Sweep up filter applies.

·         sigActionFilter: Returns true if the Situation action is of the specified type.

·         sigActionToolFilter: Returns true if the specified tool has been run against a Situation.

·         staticLookup: Searches for a key in a static lookup table, retrieves the corresponding value, and applies that value to a field in the object.

·         upperCase: Changes the value of a field to uppercase. Sweep up filter applies.

addItemToList

A Workflow Engine function that adds an item or items to an array. You can specify more than one value. Does not add duplicate elements to the array, but maintains an array of unique elements. Returns an array of unique items.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function addItemToList takes the following arguments:

alertInSituation

A Workflow Engine function that returns true when the alert is a member of an active Situation.

This function is available for alert and enrichment workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function alertInSituation has no arguments.

alertNotInSituation

A Workflow Engine function that returns true when the alert is not a member of an active Situation.

This function is available for alert and enrichment workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function alertNotInSituation has no arguments.

appendFields

A Workflow Engine function that appends a concatenated set of fields to an existing field, using a separator character.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function appendFields takes the following arguments:

appendString

A Workflow Engine function that appends a static string to an existing field separated by a space character.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.Update the Workflow Engine

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function appendString takes the following arguments:

between

A Workflow Engine function that returns true if the object creation date falls between two times. Optionally between times on specific days.

This function is available for alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function between takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function between. If you want to check for objects created at any time on Monday or on Friday, set the following:

·         from: 00:00:00

·         to: 24:00:00

·         days: ["Mon","Fri"]

The UI translates your settings to the following JSON:

{"from":"00:00:00","to":"24:00:00","days":["Mon","Fri"]}

ceventFilter

A Workflow Engine function that returns true if the object matches a SQL-like filter. This function uses the CEvents to evaluate the SQL-like filter against the object. See APIFilter Search Data for more information on filters.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function ceventFilter takes the following arguments:

checkSituationState

A Workflow Engine function that returns true if the specified state exists for a Situation. For example TICKETED, LEAVE_MANUAL_DESCRIPTION. Not to be confused with Situation status.

This function is available for Situation workflows only.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function checkSituationState takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function checkSituationState. If you want to set the state to TICKETED, enter the following:

·         state: TICKETED

The UI translates your settings to the following JSON:

{"state":"TICKETED"}

Returns true for Situations that have a state of "TICKETED". For example a Situation 22 which returns the following for /document/preview/114972#UUIDe8a3290d9d634ccc46917e36863122bb :getSituationFlags

{"22": ["TICKETED"]}

classifyEvent

A Workflow Engine function that sets the class, type, and severity fields of an event based upon its contents using a predefined classification algorithm. Overwrites existing values. See Event and Alert Field Best Practice for information on object fields.

This function is available as a feature of the Workflow Engine v1.0 download. See  Update the Workflow Engine for information on how to upgrade to this.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function classifyEvent takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function classifyEvent. If you want the Workflow Engine to automatically populate the type, class, and severity fields of the event based upon the description, set the following:

·         eventFields: []

·         typeField: type

·         classField: class

·         severityField: severity

The UI translates your settings to the following JSON:

{"eventFields":[],"typeField":"type","classField":"class","severityField":"severity"}

Given an object with the following description:

"description":"App server APPSERVER2002 down."

The Workflow Engine updates the object fields as follows:

"severity": 5,
"type": "availability",
"class": "server"

concatFields

A Workflow Engine function that sets the value of a field to a string representing a set of concatenated fields.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function concatFields takes the following arguments:

containsAlertDetails

A Workflow Engine function that returns true if all or any of the alerts in the Situation matches the filter condition. Uses SQL-like filter syntax. See Filter Search Data for more information on filters.

Applies the scope to all Situations in the Workflow when there are multiple Situations in context. For example, if you used a sweep up filter in the workflow definition. In this case, if you have set the scope to 'any', every Situation must have at least one alert match the SQL-like filter for the function to return true.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for Situation workflows only.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function containsAlertDetails takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function containsAlertDetails. If you want to verify that a Situation contains at least one severity 3 or higher alert, set the following:

·         scope: any

·         filter: severity >= 3

The UI translates your settings to the following JSON:

{"scope":"any","filter":"severity >= 3"}

convertToJSON

A Workflow Engine function that converts the object to JSON and adds it to the workflowContext for use in subsequent actions such as the restAsyncPost function.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function convertToJSON has no arguments.

copyFieldFromAlertToEvent

A Workflow Engine function that copies a single field from an existing alert to a deduplicating event for the same alert. For example, if an update event doesn't include a 'source' attribute, the 'copyFromAlertToEvent' copies the 'source' from the existing alert. If the copy fails, the event would have an empty 'source' field causing the Alert Builder to reject it. In this case create a subsequent action to check for the existence of 'source enables you to set a default source for the event if it is undefined.

When using this function, the following applies:

·         You can specify both the source and destination fields. Choose the forwarding action for this carefully.

·         If the process fails and the forwarding behavior is 'Stop All Workflows', the Workflow Engine drops the event.

·         If you select 'Always Forward' there is a risk that the Alert Builder will reject an incomplete event. For example, if an update event does not include a 'source' property, this function copies the value for 'source' from the existing alert.

·         If the copy fails, the event has an empty 'source' field causing the Alert Builder to reject it. In this case, create a subsequent action to check for the existence of 'source' and set a default source for the event if it is undefined.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function copyFieldFromAlertToEvent takes the following arguments:

copyFromAlertToEvent

A Workflow Engine function that copies multiple fields from an existing alert to a deduplicating event for the alert.

When using this function, the following applies:

·         You can backfill fields from the alert to the event. Choose the forwarding action for this carefully.

·         Returns true when the event has no matching existing alert. The same behavior as if all specified fields copied successfully. To avoid this behavior, use another function to check for event deduplication before copyFromAlertToEvent.

·         If the forwarding behavior is 'Always Forward', there is a risk that the Alert Builder may reject an incomplete event.

·         Use subsequent actions to validate the event and add defaults to any backfilled event fields as needed. For example, if an update event does not include a 'source' attribute, you can use this function to copy 'source' from the existing alert. If the copy fails, the event would have an empty 'source' field causing the Alert Builder to reject it. In this case, create a subsequent action to check for the existence of 'source' which enables you to set a default source for the event if it is undefined.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function copyFromAlertToEvent takes the following arguments:

createServiceTicket

A Workflow Engine function that creates a ticket for the specified service.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function createServiceTicket takes the following arguments:

deltaEvent

A Workflow Engine function that returns true:

·         If the specified event fields differ from corresponding fields in an existing alert.

·         When an error occurs in the delta check.

·         When no alert exists.

Returns false when it detects no changes.

Uses shallow object inspection which compensates for list ordering and object key ordering, but may still result in an inaccurate response.

This function has an inherent call to willDeduplicateAlert so you do not need to call it first.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this. Update the Workflow Engine

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function deltaEvent takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function deltaEvent. If you want to check for differences in the services in the custom_info.eventDetails.services field between the alert and deduplicating events, set the following:

·         fields: ["custom_info.eventDetails.services"]

The UI translates your settings to the following JSON:

["custom_info.eventDetails.services"]

Given an alert with the following fields:

"custom_info": {"eventDetails":
                    {"services": ["APPSERVER"]}
                   }

An event with the following fields returns true:

"custom_info": {"eventDetails":
                    {"services": ["NETWORK"]}
                   }

An event with the following fields returns false:

"custom_info": {"eventDetails":
                    {"services": ["APPSERVER"]}
                   }

dropEvent

A Workflow Engine function that allows you to prevent further processing of an event. For example, you may wish to discard an event if it has a severity of 0 and willCreateNewAlert returns true.

Always returns false. Follows the Forwarding Behavior settings.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function dropEvent has no arguments.

existingAlertFilter

A Workflow Engine function that returns true if the existing alert for a deduplicating event matches a SQL-like filter. Uses the evaluateFilter method in the CEvents API.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function existingAlertFilter takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function existingAlertFilter. If you want to verify if a deduplicating event has an agent_location value of "London", set the following:

·         filter: agent_location = 'London'

The UI translates your settings to the following JSON:

agent_location = 'London'

Given an alert with the following signature:

"APPSERVER2002:APPLICATION:AVAILABILITY"

An event with the following fields returns true:

"signature": "APPSERVER2002:APPLICATION:AVAILABILITY",
"agent_location": "London"

An event with the following fields returns false:

"signature": "APPSERVER2002:APPLICATION:AVAILABILITY",
"agent_location": "SF"

forward

A Workflow Engine function that forwards the object to the named Moolet. Navigate to Settings> Self Monitoring > Event Processing to see the list of Moolets running on your system.

This function is available for alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function forward takes the following arguments:

hasCausalPRC

A Workflow Engine function that returns true if one or more alerts in the Situation has a causal PRC flag set.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for Situation workflows only.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function hasCausalPRC has no arguments.

hasMerged

A Workflow Engine function that returns true if the Situation has been merged with another Situation or superseded by a Situation. See Configure Merge Groups for more information.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function hasMerged has no arguments.

hasNotMerged

A Workflow Engine function that returns true if the Situation has not been merged with another Situation or superseded by another Situation. See Configure Merge Groups for more information.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function hasNotMerged has no arguments.

hasSimilarSituations

A Workflow Engine function that returns true when the Situation has a similar Situation above the specified threshold. Uses the Situation Similarity utility provided with the Workflow Engine to calculate similarity between .5 (50% similar) and 1 (100% similar).

The Situation Similarity requires some configuration. You can find the configuration file at: $MOOGSOFT_HOME/config/SimilarSigConfig.conf.

·         Verify the Graze API credentials are valid.

·         Verify the webhost is correct if the UI runs on a different host than Moogfarmd.

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function hasSimilarSituations takes the following arguments:

isAssigned

A Workflow Engine function that returns true if the object has an owner or moderator.

This function is available for alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isAssigned has no arguments.

isClear

A Workflow Engine function that returns true if the object's severity level is Clear (0).

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isClear has no arguments.

isInSubnet

A Workflow Engine function that returns true when an IP address is present within a specified subnet.

This function is available for event, alert, and enrichment workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isInSubnet takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function isInSubnet. To check if the source field for an IP address within the "198.51.100.0/24" subnet, set the following:

·         field: source_id

·         subnet: 198.51.100.0/24

The UI translates your settings to the following JSON:

{"field":"source_id","subnet":"(198.51.100.0/24)"}

An object with the following source_id value returns true:

source_id":"198.51.100.33"

An object with the following source_id value returns false:

source_id":"198.51.101.33"

isNewerThan

A Workflow Engine function that returns true when the object age in seconds is less than a specified age in seconds. Uses the difference between the current time and agent_time for events, int_last_event_time for alerts, and last_event_time for Situations.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isNewerThan takes the following arguments:

isNotAssigned

A Workflow Engine function that returns true if the object does not have an owner or moderator.

This function is available for alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isNotAssigned has no arguments.

isNotNull

A Workflow Engine function that returns true if the value for a field within an object is not null, is not an empty object , {}, or is not an empty array , []. Alert and Event Field Reference and Event and Alert Field Best Practice for more information on object fields.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isNotNull takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function isNotNull.

{"field":"custom_info"}

isNull

A Workflow Engine function that returns true if the value for a field within an object is null, is an empty object , {}, or is an empty array , []. See Alert and Event Field Reference and Event and Alert Field Best Practice for more information on object fields.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isNull takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function isNull.

{"field":"custom_info"}

isOlderThan

A Workflow Engine function that returns true when the object age in seconds is greater than a specified age in seconds. Uses the difference between the current time and agent_time for events, int_last_event_time for alerts, and last_event_time for Situations.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function isOlderThan takes the following arguments:

labelSituation

A Workflow Engine function that labels the Situation using the Situation Manager Labeler macro language. Does not override manual Situation descriptions. See Situation Manager Labeler for more information on the macro language.

This function is available for Situation workflows only.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function labelSituation takes the following arguments:

listContains

A Workflow Engine function that returns true when the array field you query contains some of your specified values. Define values as an array, for example [ a ] or [ a, b, c ]. You must specify an array field, not a string.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function listContains takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function listContains. If you want to check for the existence of "Webserver" or "Webapp" elements in an array of services in custom_info, set the following:

·         field: custom_info.eventDetails.service_list

·         values: ["Webserver","Webapp"]

The UI translates your settings to the following JSON:

{"field":"custom_info.eventDetails.service_list","values":["Webserver","Webapp"]}

An object with the following custom_info value returns true:

"custom_info": {"eventDetails":
                    {"service_list": ["Webapp"] }
                }

An object with the following custom_info value returns false:

"custom_info": {"eventDetails":
                    {"service_list": ["Network"] }
                }

listContainsAll

A Workflow Engine function that returns true when the array field you query contains all of your specified values. Define values as an array, for example [ a ] or [ a, b, c ]. You must specify an array field, not a string.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function listContainsAll takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function listContainsAll. If you want to check for the existence of both "Webapp" and "Webserver" elements in an array of services in custom_info, set the following:

·         field: custominfo.eventDetails.service_list

·         values: ["Webapp","Webserver"]

The UI translates your settings to the following JSON:

{"field":"custominfo.eventDetails.service_list","values":["Webapp","Webserver"]}

An object with the following custom_info value returns true:

"custom_info": {"eventDetails":
                    {"service_list": [ "Webserver", "Webapp"] }
                }

An object with the following custom_info value returns false:

"custom_info": {"eventDetails":
                    {"service_list": ["Webserver","Network"]}
                }

listDoesNotContain

A Workflow Engine function that returns true when the array field you query contains none of your specified values. Define values as an array, for example [ a ] or [ a, b, c ]. You must specify an array field, not a string.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function listDoesNotContain takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function listDoesNotContain. If you want to check for the absence of an element "Network" in an array of services in custom_info, set the following:

·         field: custominfo.eventDetails.service_list

·         values: ["Network"]

The UI translates your settings to the following JSON:

{"field":"custominfo.eventDetails.service_list","values":["Network"]}

An object with the following custom_info value returns true:

"custom_info": {"eventDetails":
                    {"service_list": ["Webapp","Webserver"] }
                }

An object with the following custom_info value returns false:

"custom_info": {"eventDetails":
                    {"service_list": ["Webapp","Webserver","Network"] }
                }

logMessage

A Workflow Engine function that logs a warning level message to the Moogfarmd log. See Configure Logging for information on log locations.

Prepends the message with the workflow function name and the alert or Situation ID, or signature for events.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function logMessage takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function logMessage.

{"message":"Urgent action required."}

logWorkflowDuration

A Workflow Engine function that logs debug messages for the workflow execution duration. To log duration you need to set at lest two actions with logWorkflowDuration in your workflow. The first starts the timer. subsequent instances log the elapsed time since the first logWorkflowDuration action within the workflow. For example, for an event:

DEBUG: [0:Event Workflows housekeeping][20191002 21:08:30.208 -0400] [WorkflowEngine.js:6907] +|Even
t Workflows::logWorkflowDuration: Workflow  for event : APPSERVER2002:APPLICATION:AVAILABILITY:  execution
 time = 10858ms|+

To enable debug logging for Moogfarmd, execute the following:

farmd_cntl --loglevel debug

When you are through logging, reset the log level to warn:

farmd_cntl --loglevel warn

This function is available as a feature of the Workflow Engine v1.0 download. See Update the Workflow Engine for information on how to upgrade to this.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function auditWorkflow takes the following arguments:

lookupAndReplace

Updates a specified alert field with a static value if any listed alert fields contain a text substring or match against a regular expression.

For example, you can search both the class and description fields for the words "router" or "switch" while also searching for the regular expression representing a network interface: "eth\\d+". In case of a match, you can update the custom_info.key field to the static value of "network." Then you can configure a Cookbook recipe to use the custom_info.key field for clustering.

This function is available for alert and enrichment workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function lookupAndReplace takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function lookupAndReplace. Set the following to search for network related terms in the class or description fields and set the custom_info.key field to "network":

·         wordList: ["router","switch"]

·         reList: ["eth\\d+","network"]

·         inFields: ["class","description"]

·         alertField: custom_info.services

·         value: network

The UI translates your settings to the following JSON:

{"wordList":["router","switch"],"reList":["eth\\d+","network"],"inFields":["class","description"],"alertField":"custom_info.key","key":"network"}

The following example data matches the lookup criteria:

"class":"network",
"description":"Communication link failure."

 "class":"",
 "description":"Router failed."

"class":"",
"description":"Interface eth0 down."

For all matching cases, the Workflow Engine updates the custom_info field as follows:

"custom_info": {"key": "network"}

The following data does not match the lookup criteria so the custom_info.key field remains unchanged:

"class":"",
"description":"Error establishing database connection."

prependFields

A Workflow Engine function that prepends a concatenated set of fields to an existing field, using a separator character.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function prepEndFields takes the following arguments:

prependString

A Workflow Engine function that prepends a string to an existing field, using a separator character.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function prependString takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function prependString.

{"string":"This is an example of prepending further description.","destination":"description"}

replaceString

A Workflow Engine function that replaces a string or regular expression in a field with a specified string or regular expression.

This function is available for alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function replaceString takes the following arguments:

restAsyncPost

A Workflow Engine function that makes a HTTP POST request with a JSON payload to a named REST endpoint. Expects the payload from a previous action, for example, from the convertToJSON function that converts an event, alert or Situation to a JSON blob. Returns false when no payload is found.

restAsyncPost is a non-blocking asynchronous call which returns true to the workflow immediately. It is best for a 'data sink' use case. It does not support setting authentication or other HTTP request fields or attributes.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function restAsyncPost takes the following arguments:

 

The following example demonstrates typical use of Workflow Engine function restAsyncPost.

{"URL":"https://example.com"}

searchAndReplace

A Workflow Engine function that matches a regular expression to an object field and updates the values for fields in the object based upon a map. You can map the contents of subgroups to other fields. For example, extract the 'source' value inside a description and map it to the source field. You can also map fields to a constant value.

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function searchAndReplace takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function searchAndReplace. You can check for an IP address, and a value of "memory" or "disk" in the object's description field. When the Workflow Engine finds a match, it maps the following fields:

·         source to the matching IP address: ((?:\\d+\\.){3}\\d+).

·         class to the matching value of "memory" or "disk": (memory|disk).

·         custom_info.support team to the constant "NOC".

Configure the function as follows:

·         field: description

·         expression: ^.+?((?:\d+\.){3}\d+).+?(memory|disk).+?$

·         map: {"source":"$extract.1","class":"$extract.2","custom_info.support_team":"NOC"}

The UI translates your settings to the following JSON:

{"field":"description",
"expression":"^.+?((?:\\d+\\.){3}\\d+).+?(memory|disk).+?$",
"map":{"source":"$extract.1","class":"$extract.2","custom_info.support_team":"NOC"}}

Note:

The code display for the Workflow Engine double-escapes characters. You do not need to double-escape in the data entry field. For example the IP address: "((?:\d+\.){3}\d+)".

When you have nested subgroups, as in the example with the IP address, they do not affect the extract numbering.

An object with the following description matches the regular expression test:

"description": "Host 198.51.100.0 high memory utilization on mytestbox.example.com"

The Workflow Engine updates the object fields as follows:

"source": "198.51.100.0",
"custom_info": {"support_team": "NOC"},
"class": "memory"

sendMooletInform

A Workflow Engine function that sends a Moolet inform with a subject and details. Adds the object to the payload, and so is always available to the receiver. See Moolet Informs for more information.

This function is available for alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function sendMooletInform takes the following arguments:

setClass

A Workflow Engine function that sets the class of the alert to a static value.

This function is available for alert and enrichment workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setClass takes the following arguments:

setCoreEventField

A Workflow Engine function that sets a single core event field to a static value. For custom info, use the setCustomInfoValue or setCustomInfoJSONValue functions. For example set the agent_location field to "London".

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setCoreEventField takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function setCoreEventField.

{"field":"signature","value":"mySource:myClass:myType"}

setCustomInfoJSONValue

A Workflow Engine function that adds or updates a custom info key to the specified JSON value. Accepts complex keys: a.b.c.d. The value must be a JSON object. Use the setCustomInfoValue function for to set string values.

This function is available for alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setCustomInfoJSONValue takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function setCustomInfoJSONValue. If you want to set the JSON value for the custom_info.services key, set the following:

·         key: services

·         value: {"service_list":["Network","Database"]}

The UI translates your settings to the following JSON:

{"service_list":["Network","Database"]}

The Workflow Engine updates the object fields as follows:

 "custom_info":
    {"services":
        {"service_list": ["Network","Database"]}
    }

setCustomInfoValue

A Workflow Engine function that adds or updates a custom info key to a specified string value. Accepts complex keys: a.b.c.d. The value must be a text string. Use the setCustomInfoJSONValue for JSON object values.

This function is available for alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setCustomInfoValue takes the following arguments:

setDescription

A Workflow Engine function that sets the description of the object. The action does not override manual descriptions.

This function is available for alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setDescription takes the following arguments:

setSeverity

A Workflow Engine function that sets the severity of the alert.

This function is available for alert and enrichment workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setSeverity takes the following arguments:

setSituationState

A Workflow Engine function that sets the state of the Situation. Not to be confused with Situation status.

This function is available for Situation workflows only.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setSituationState takes the following arguments:

setType

A Workflow Engine function that sets the type of the alert.

This function is available for alert and enrichment workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function setType takes the following arguments:

Example

The following example demonstrates typical use of Workflow Engine function setType. If you want to set the type for an object to "availablity", enter the following:

·         type: availability

The UI translates your settings to the following JSON:

{"type":"availability"}

sigActionFilter

A Workflow Engine function that returns true if the Situation action matches the specified type.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function sigActionFilter takes the following argument:

sigActionToolFilter

A Workflow Engine function that returns true if the specified tool has been run against a Situation. For example, a ticketing integration tool.

This function is available for Situation workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function sigActionToolFilter takes the following argument:

staticLookup

A Workflow Engine function that searches for a key in a static lookup table, retrieves the corresponding value, and applies that value to a field in the object. lookupName references a .lookup file in JSON format in the following folder: $MOOGSOFT_HOME/config/lookups/.

For example, Locations refers to $MOOGSOFT_HOME/config/lookups/Locations.lookup. On first use, the lookup loads into constants. You do not need to edit the Workflow Engine Moobot to load. The default lifespan for the lookup is 3600 seconds, after which the Workflow Engine reloads the file.

This function is available for event, alert, enrichment, and Situation workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function staticLookup takes the following arguments:

stripFQDN

A Workflow Engine function that splits a fully qualified domain name (FQDN) into a hostname/short name and a domain name and updates fields with the values.

If shortNameField begins with "www" or a derivative, sets the value to the subsequent segment of the domain. For instance, "www3.example.com" returns "example'.

If you don't want to map the domain name, enter null or an empty string, "", for the domainNameField .

This function is available for event, alert, and enrichment workflows.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function stripFQDN takes the following arguments:

upperCase

A Workflow Engine function that changes the value of a field to uppercase. For example, changes a value of "Network" to "NETWORK".

This function is available for event, alert, enrichment, and Situation workflows.

The workflow sweep up filter applies to this function.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function upperCase takes the following argument:

willCreateNewAlert

A Workflow Engine function that returns true if the event will create a new alert.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function willCreateNewAlert has no arguments.

willDeduplicateAlert

A Workflow Engine function that returns true if the event will deduplicate into an existing alert.

This function is available for event workflows only.

Back to Workflow Engine Functions Reference.

Arguments

Workflow Engine function willDeduplicateAlert has no arguments.

Troubleshoot the Workflow Engine

The Workflow Engine is a flexible tool that can help you transform your data and manage its flow through Cisco Crosswork Situation Manager data processing. Workflow Engine functions provide you programmatic control over your data, but sometimes your workflows may not behave the way you expect.

This topic contains ideas to help you debug your workflows and get the most from the Workflow Engine.

Troubleshoot from the UI

Consider the following options for troubleshooting workflows from the Cisco Crosswork Situation Manager UI:

·         If you have multiple workflows enabled, but one of them is behaving unexpectedly, try temporarily disabling the other workflows to see if it works on its own. If so, reactivate the other workflows one by one, testing at each step to see if one of the other workflows is affecting it.

·         Check the forwarding behavior for your workflow actions. The forwarding behavior controls subsequent processing when the function returns false. Stop This Workflow prevents the object passing to subsequent actions and Stop All Workflows prevents the object from passing to any subsequent action or workflow.

·         Test your entry filter for the workflow. If your objects are not meeting entry filter requirements, the workflow will not process them.

·         Verify your source fields and destination fields. Make sure that the names match up exactly. If you are using complex keys, make sure that you have the path exactly right. For example: custom_info.eventDetails.services.

·         If you are using an event that specifies a Moolet, check the Moolet name under Settings > Self Monitoring > Event Processing. For example, "Default Cookbook".

Troubleshoot from the moog_farmd.log

If you have access to the log for Moogfarmd, you have a lot more troubleshooting options to identify exactly what is happening with your objects as they progress through workflows.

To enable debug logging for Moogfarmd, execute the following:

farmd_cntl --loglevel debug

When you are through logging, reset the log level to warn:

farmd_cntl --loglevel warn

You can find the Moogfarmd log at /var/logs/moogsoft. See Configure Logging for more information.

The Workflow Engine includes the following logging functions to help you troubleshoot:

·         logMessage: Logs a message to the Moogfarmd log.

·         logWorkflowDuration: Logs debug messages for the workflow execution duration.

The log messages from the Worfklow Engine include the engine name along with details about the object processing in the workflow. This means that you can use the tail command to observe the activity within an engine. For example:

tail -f MOO.moog_farmd.log | grep ":Alert.Workflows"

Within the log output, you can search for specific things, including:

·         The function name you are troubleshooting.

·         Identifying data for the object you are processing, such as the event signature.

·         Identifying information about an entry or sweep up filter.

See Example Workflow Engine log for sample messages and their meanings within the log context.

Example Workflow Engine log

The following log segment includes comments to highlight the different aspects of a Workflow Engine log:

### Alert did not pass the entry filter ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:55.983 -0400] [CWorkflow.java:470] +|Moolet [Enrichment Workflows] - workflow [Closed Alerts Filter]: message [{"Elements":{active_sig_list=[67, 68], agent=DATA_SOURCE, agent_location=my_agent_location, alert_id=165, class=my_class, count=3, custom_info={eventDetails={agent=TestAgent1, first_occurred=1570047828, service=SAP, name=REST LAM Post 1, team=SAP Support}}, description=DESC: Host 1 Sig 1, entropy=0.8312803355385304, event_id=2899, external_id=my_external_id, first_event_time=1570047828, int_last_event_time=1570047828, last_event_time=1570047896, last_state_change=1570047828, manager=TestMgr1, owner=2, rc_probability=null, severity=5, sig_list=[67, 68], signature=lnux100:sig1, significance=3, source=lnux100, source_id=192.168.100.101, state=2, type=TestType1}, "Topic":"alerts", "Seq":"0", "SessId":"4769192054476008521", "Pdu":"E_MooMsg", "MessageId":"c2fc745a-8572-4982-a012-69fe64b84e96", "CorrelationId":"ff62fbbb-45ff-44f8-a1b5-9341bf33e729", "Metadata":{action=Event Added To Alert, clock_time=1570047895, message_type=1, previous_data={last_event_time=1570047869, count=2}, user_id=2}, "UsedCount":"null", "AckPoint":"0"}] failed to pass entry filter [state = 9].|+

### Workflow is inactive ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:55.983 -0400] [CWorkflow.java:463] +|Moolet [Enrichment Workflows] - workflow [Enrich From SNow] inactive, sending message to the next Workflow/Moolet.|+

### Active workflow begins ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:55.983 -0400] [CWorkflow.java:294] +|Moolet [Enrichment Workflows] - workflow [Test External DB]: starting delay of [0] seconds for msg [{"Elements":{active_sig_list=[67, 68], agent=DATA_SOURCE, agent_location=my_agent_location, alert_id=165, class=my_class, count=3, custom_info={eventDetails={agent=TestAgent1, first_occurred=1570047828, service=SAP, name=REST LAM Post 1, team=SAP Support}}, description=DESC: Host 1 Sig 1, entropy=0.8312803355385304, event_id=2899, external_id=my_external_id, first_event_time=1570047828, int_last_event_time=1570047828, last_event_time=1570047896, last_state_change=1570047828, manager=TestMgr1, owner=2, rc_probability=null, severity=5, sig_list=[67, 68], signature=lnux100:sig1, significance=3, source=lnux100, source_id=192.168.100.101, state=2, type=TestType1}, "Topic":"alerts", "Seq":"0", "SessId":"4769192054476008521", "Pdu":"E_MooMsg", "MessageId":"c2fc745a-8572-4982-a012-69fe64b84e96", "CorrelationId":"ff62fbbb-45ff-44f8-a1b5-9341bf33e729", "Metadata":{action=Event Added To Alert, clock_time=1570047895, message_type=1, previous_data={last_event_time=1570047869, count=2}, user_id=2}, "UsedCount":"null", "AckPoint":"0"}]|+

### Name of the function that is processing ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:55.984 -0400] [CWorkflowBotAction.java:196] +|Performing action [enrichOneToOne]|+

### Depending on the function, different logs here ###

###  Alert updated ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:56.096 -0400] [CMooMsg.java:1086] +|Encoded size [991] json[{"_MOOTADATA_":{"action":"Alert Updated","clock_time":1570047896,"message_type":1,"previous_data":{"custom_info":{"enrichment":null,"eventDetails":{}},"last_state_change":1570047828}},"active_sig_list":[67,68],"agent":"DATA_SOURCE","agent_location":"my_agent_location","alert_id":165,"class":"my_class","count":3,"custom_info":{"eventDetails":{"agent":"TestAgent1","first_occurred":1570047828,"service":"SAP","name":"REST LAM Post 1","team":"SAP Support"},"enrichment":{"ci":{"Name":"lnux100","AssetClass":"Linux Server"}}},"description":"DESC: Host 1 Sig 1","entropy":0.8312803355385304,"external_id":"my_external_id","first_event_time":1570047828,"int_last_event_time":1570047828,"last_event_time":1570047896,"last_state_change":1570047896,"manager":"TestMgr1","owner":2,"rc_probability":null,"severity":5,"sig_list":[67,68],"signature":"lnux100:sig1","significance":3,"source":"lnux100","source_id":"192.168.100.101","state":2,"type":"TestType1"}]|+

### Action completing with an exit status of 'true' ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:56.103 -0400] [CMDB-WFE.js:403] +|Enrichment Workflows::enrichOneToOne: Exiting action with a status of true|+

### Workflow Finished and sending to next Moolet ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:56.104 -0400] [CPassToNextMoolet.java:63] +|Moolet [Enrichment Workflows] - Sending message to the next Moolet|+
DEBUG: [3:Enrichment Workflows][20191002 16:24:56.104 -0400] [CMsgDispatch.java:516] +|Dispatching message from [Enrichment Workflows]|+

### Name of the next Moolet for the alert ###
DEBUG: [3:Enrichment Workflows][20191002 16:24:56.104 -0400] [CMsgDispatch.java:547] +|Dispatching to [MaintenanceWindowManager]|+

Action States

Action States are the different states which alerts are placed into as they pass from the Alert Builder into the Alert Rules Engine. See the Alert Rules Engine for a standard link up-link down example that uses Action States.

The different states define how long the alerts are retained in a certain state and whether they are forwarded to any Moolets or Sigalisers. The default or base state is called 'Ground'. It is required for the system to function correctly and cannot be deleted. This is the state that alerts have when they enter the Alert Rules Engine.

Create an Action State

Click + to create a new Action State. The available fields are as follows:

Click Save Changes to continue. Alternatively, click Revert Changes to undo any changes. The new Action State appears on the list to the left.

Delete an Action State

Select the Action State you want to delete from the list on the left.

Click - to delete the Action State. The pop-up confirmation window appears. Click Yes to confirm the deletion.

Transitions

Transitions are a user-configurable set of conditions that move an alert from one state to another within the Alert Rules Engine (ARE). Transitions result in the following:

·         Alerts move from one Moolet to the next Moolet in the chain.

·         Alerts pass to a clustering algorithm which clusters them into Situations.

To create and configure different transitions go to System Settings.

Create a New Transition

Click + to create a new transition and edit the fields to meet your requirements:

When you have configured the transition, click Save to continue.

Delete a Transition

To delete a transition from the list of available transitions:

·         Select the transition to delete.

·         Click - to delete the transition.

·         Click Yes to confirm the deletion.

Auto Close

The Auto Close feature lets you define criteria for automatically closing alerts and Situations. Auto Close enables you to use filtering rules to organize your data and keep it current so you can focus on the most important active alerts and Situations.

You also see performance improvements because automatically closing old alerts and Situations reduces the amount of data involved in statistic calculations.

Auto Close lets you define the conditions using filters and determine how often Cisco Crosswork Situation Manager checks which alerts and Situations to close. Any alerts and Situations older than a certain time and that meet the defined criteria are closed.

Configure Auto Close for Situations

The Housekeeper Moolet must be configured and running within Moogfarmd in order for Auto Close to work.

To configure which Situations should be auto closed, create a filter as follows:

·         In the Auto Close > System Settings window, click Edit Filter to open the filter editor.

·         Clear the filter using Empty Filter and Add Clause. Alternatively, you can manually type in your filter rules. You can set up as many auto close rules as you like. In a rule, you can either include or exclude Situations for auto closing but you cannot use both together. See Filter Search Data for reference.

·         Apply the changes to continue and click Done.

·         After you add the filter, define the behavior for automatically closing Situations as follows:

·         Close the Situation and all the alerts it contains.

·         Close the Situation and all the unique alerts it contains. Unique alerts are any alerts that are not part of any other Situations.

·         Close the Situation only.

You can create tasks to configure:

·         The age when Situations are suitable for Auto Close.

·         The number of Situations to close in each Auto Close run.

·         Situations only close if all associated alerts are closed.

Edit the default task or click Add Task. The available settings are as follows:

Once saved, the Auto Close task runs after a set period of time. This time period is between five minutes and four hours depending on the age of the Situation.. The older the Situation age, the closer the frequency of the task gets to four hours (see the example below). There is no limit on the number of tasks, so you can add any as many as you need meet your requirements.

The example below demonstrates how you can configure an Auto Close task to close a maximum of 1000 Situations per run that meet the following criteria:

·         Older than 23 hours.

·         All associated alerts are closed.

·         Have a clear severity.

Configure Auto Close for Alerts

To configure which alerts should be auto closed, on the System Settings > Auto Close window, select the Alerts tab and create a filter as follows:

·         Click Edit Filter to open the filter editor.

·         Clear the filter using Empty Filter and Add Clause. Alternatively, you can manually type in your filter rules. You can set up as many auto close rules as you like. In a rule, you can either include or exclude alerts for auto closing but you cannot use both together. See Filter Search Data for reference.Filter Search Data

·         Apply the changes to continue and click Done.

You can create tasks to configure:

·         The age when alerts are suitable for Auto Close.

·         The number of alerts to close in each Auto Close run.

Edit the default task or click Add Task. The available settings are:

Once saved, the Auto Close task runs after a set period of time. This time period is between five minutes and four hours depending on the age of the alert. The older the alert age, the closer the frequency of the task gets to four hours.

There is no limit on the number of tasks, so you can add any as many as you need meet your requirements.

The example below demonstrates how you can configure a task to Auto Close a maximum of 1000 alerts per run that meet the following criteria:

·         Older than 45 minutes.

·         Have a clear severity or a minor severity.

Configure Clustering Algorithms

Clustering algorithms in Cisco Crosswork Situation Manager group alerts based on factors such as time, language, similarity and proximity. See the Clustering Algorithm Guide for more information.

You can configure the following Cisco Crosswork Situation Manager clustering algorithms:

·         Cookbook: A powerful clustering algorithm that creates clusters defined by the relationships between alerts and their attributes. To configure Cookbook and Recipes, see Configure Cookbooks and Recipes.Cookbook

·         Tempus: A time-based algorithm that clusters alerts into Situations based on the similarity of their timestamps. To configure Tempus, see Configure Tempus.Tempus

·         Merge Groups: Enables you to set a similarity threshold so that Cisco Crosswork Situation Manager merges Situations created by different clustering algorithms that meet this threshold.

You can also configure Cookbook and its Recipes, Tempus, and Merge Groups via the Graze API.

The Import/Export option in the Cisco Crosswork Situation Manager UI, enables you to export clustering algorithm configurations from one Cisco Crosswork Situation Manager system and import them into another. For example, from a test environment to your production environment.

Configure Cookbooks and Recipes

Use the following steps to configure a Cookbook and its Recipes:

1.     First, you must configure the Recipes that you want to use in a Cookbook. See Configure a Cookbook Recipe for details.

2.     Then, you must create the Cookbook that will contain those Recipes. See #UUID66e2bced28a9be79f61b8a09e492c1ff_titleidm123132773577394 for details.

3.     Finally, you must activate the Cookbook so that it starts to cluster alerts into Situations. See #UUID66e2bced28a9be79f61b8a09e492c1ff_titleidm123132773635844 for details.

Configure a Cookbook Recipe

A Cookbook Recipe is a set of configurable filters, triggers, and calculations that defines the type of alerts and the alert relationships that Cookbook detects and clusters into Situations.

Cookbook requires at least one active Recipe in order to function and cluster alerts into Situations.

You can configure the following two recipe types from the UI:

·         Value Recipe v2: Default Recipe that extracts and analyzes groups of consecutive characters, called shingles, to measure text similarity between alerts.

·         Value Recipe: First version of the Value Recipe that uses a string comparison mechanism to determine text similarity between alerts.

See Recipe Types for more details on the different types of Recipes available in Cookbook. If you want to implement a Bot Recipe that allows you to call Moobot functions, you can use the Graze API.

Before you begin

Before you set up your Recipe via the UI, ensure you have met the following requirements:

·         Your LAMs or integrations are running and Cisco Crosswork Situation Manager is receiving events.

·         If you want to use Vertex Entropy or hop limit in your Recipes, you have imported your network topology. See Import a Topology.

Create a Cookbook Recipe

To create a new Cookbook Recipe from the Cisco Crosswork Situation Manager UI:

1.     Navigate to the Settings tab.

2.     Click Cookbook Recipes in the Algorithms section.

3.     Click the + icon to create a new Recipe.

4.     On the Recipe tab, enter the properties to name and describe the Recipe:

—      Name: Name of the Recipe. Use a unique and descriptive name.

—      Situation Description: Description that appears in Situations that the Recipe creates.

—      Recipe Type: Type of recipe. The options are Value Recipe and Value Recipe v2. See Recipe Types for more information.

5.     Configure the Recipe behavior and filters that define the alert relationships:

—      Trigger Filter: Determines the alerts that Cookbook considers for Situation creation. Cookbook includes alerts that match the trigger filter. By default Cookbook only includes alerts with a severity of 'Critical'. For details on creating a filter, see Filter Search Data. To set a vertex entropy trigger filter, see Set Up Vertex Entropy for more information.

—      Exclusion Filter: Determines the alerts to exclude from Situation creation. Cookbook ignores alerts that match the exclusion filter. For details on creating a filter, see Filter Search Data. To set a vertex entropy exclusion filter, see Set Up Vertex Entropy for more information.

—      Seed Alert Filter: Determines whether to create a Situation from a seed alert. The seed alert must meet both the Trigger Filter, Exclusion Filter and Seed Alert Filter criteria to create a Situation. Cookbook considers subsequent alerts for clustering if they meet the trigger and exclusion filter criteria. Alerts that arrive prior to the seed alert that met the trigger and exclusion filter criteria do not form Situations. For details on creating a filter, see Filter Search Data.To set a vertex entropy seed alert filter, see Set Up Vertex Entropy for more information.

·         The seed alert filter is a mechanism to ensure that only specific events create Situations. For example, if you create a seed alert filter where the description matches 'Switch failure', alerts are eligible for clustering into a Situation only after a seed alert with the matching description arrives.

—      Rate Filter: Determines whether Cookbook clusters alerts into Situations based on the rate the alerts arrive and the minimum and maximum sample size. To add a rate filter, check the checkbox and complete the following fields:

o    Rate: Rate, in number of alerts per second. Cookbook clusters alerts if they arrive at the rate specified here or higher.

o    Min Sample Size: Number of alerts that must arrive before the Cookbook starts to calculate the alert rate.

o    Max Sample Size: Maximum number of alerts that are considered in the alert rate calculation. When more than this number of alerts have arrived, Cookbook discards the oldest alerts and calculates the alert rate based on the number of alerts in the Max Sample Size.

—      Alert Threshold: Minimum number of alerts in a candidate cluster required before Cookbook creates a Situation. If left as '1', a single alert can generate a new Situation.

·         To determines the number of alerts required to create a Situation, Cookbook compares the alert threshold values in the Cookbook Recipe to those of the merge group that the Cookbook Recipe belongs to. It uses the higher value.

·         If you are using the default merge group which has an alert threshold of 2, Cookbook will never create a Situation containing a single alert. If you want Cisco Crosswork Situation Manager to create Situations with a single alert, change the alert threshold in the default merge group to 1 or create a custom merge group. See Configure Merge Groups for more information on updating the default merge group and setting up custom merge groups.

—      Cook For: Minimum time period, in seconds, that Cookbook clusters alerts for before the Recipe resets and starts a new cluster. See Cookbook and Recipe Examples for more information.

·         If you set a different Cook For time for a Recipe, it overrides the Cookbook value. Recipes without a Cook For time inherit the value from the Cookbook.

—      Cook For Extension: Time period that Cookbook can extend clustering alerts for before the Recipe resets and starts a new cluster. Setting this value enables the cook for auto-extension feature for this Recipe. As Cookbook receives related alerts, it continues to extend the total clustering time until the Max Cook For period is reached. Used in conjunction with the Max Cook For value, the Cook For Extension period helps to ensure that Cookbook continues to cluster alerts together that are related to the same failure. The Cook For Extension period only applies to new related alerts; it does not apply to existing alerts that are updated with new events. See Cookbook and Recipe Examples for more information.

·         If you set a different Cook For Extension time for a Recipe, it overrides the Cookbook value. Recipes without a Cook For Extension time inherit the value from the Cookbook.

—      Max Cook For: Maximum time period that Cookbook clusters alerts for before the Recipe resets and starts a new cluster. It works in conjunction with the Cook For Extension time to help to ensure that Cookbook continues to cluster alerts together that are related to the same failure. If Cook For Extension is set and this value is not set, it defaults to three times the Cook For value. See Cookbook and Recipe Examples for more information.

·         If you set a different Max Cook For time for a Recipe, it overrides the Cookbook value. Recipes without a Max Cook For value inherit the value from the Cookbook.

6.     Configure the alert matching properties for the Recipe:

—      Cluster By: Defines how Cookbook matches alerts to clusters. You can select the default option to cluster alerts based on Cookbook's First Recipe Match Only setting. The First Matching Cluster option adds alerts to the first cluster above the similarity threshold value. The alternative is Closest Matching Cluster to add alerts to the cluster with the highest similarity greater than the similarity threshold value. The latter option might be less efficient because it needs to compare alerts against each cluster in a Recipe.

—      Hop Limit: Maximum number of hops between the alert source nodes in order for the alerts to qualify for clustering. Cisco Crosswork Situation Manager measures hop limit from the first alert that formed the Situation and always follows the shortest possible route in the network. A hop is the jump between two directly connected nodes in a network. For more information on hops, see Vertex Entropy. To set a hop limit, see Set Up Vertex Entropy for more information.

·         You can only use a hop limit if you have imported your network topology into the system. See .Import a Topology for details.

7.     On the Clustering tab, add the fields that you want Cookbook to factor in when clustering alerts:

—      Click the + icon and select a field in the drop-down list.

—      Use the slider to set the similarity threshold for each field. The value determines the required percentage similarity for Cookbook to cluster a set of alerts.

—      If you want to use custom info fields, configure the Match List Items option. See Match List Items in Recipes for details.

—      If you are configuring a Value Recipe, check Case Sensitive if you want the text similarity calculation to factor in case sensitivity. See Recipe Types for more information.

—      If you are configuring a Value Recipe V2, select whether you want Cookbook to calculate text similarity using shingles or words. You can select Shingles from the drop-down list in the Language Processing field and enter a Shingle Size. The default value is the optimal shingle size for that field. Alternatively, you can select Words. See Recipe Types for more information.

8.     Click Save Changes.

When you have completed the configuration, Cisco Crosswork Situation Manager applies the changes to any active Cookbooks that use the Recipe as soon as you save the changes. If the Recipe has not been added to an active Cookbook, go to Settings > Cookbook and move the Recipe under Selected Recipes for that Cookbook.

If you change a Cookbook Recipe, see Cookbook Configuration Changes for information on how these changes affect the clusters that Cookbook creates.

Recipe Types

The Cookbook clustering algorithm uses the following Recipe types to define alert relationships and control how it clusters alerts:

·         Value Recipe v2

·         Value Recipe

·         Bot Recipe

The Value Recipe V2 and CValueRecipe use different methods to calculate the textual similarity between alerts. The CBotRecipe is a customizable recipe that allows you to call specific functions from a Moobot.

Value Recipe v2

Value Recipe v2 extracts and analyzes groups of consecutive characters to measure text similarity between alerts. It is the default Recipe in Cookbook for new Cisco Crosswork Situation Manager v7 installations and for any new Cookbooks you create.

This recipe uses the bag-of-words model and shingling natural language processing methods to calculate the text similarity between alerts. Shingling is the process in which Cookbook extracts groups of consecutive characters called shingles from a source string. Potential sources include the alert source ID or description. To measure similarity, Cookbook calculates the number of identical shingles. You can control the calculation using the shingle size property.

In the Clustering tab in the Cookbook Recipes window in Settings, you select whether Cookbook treats string values as shingles or words for each field you use to cluster alerts. If you select shingles, you can choose what you want the shingle size to be. The default shingle size settings in the Value Recipe v2 are optimal for most use cases.

For example, if you set the shingle size for source IDs to 2 and Cookbook receives two alerts with the source IDs:

webserver0100

webserver0200

Cookbook extracts the following shingles from the source ID strings:

we eb bs se er rv ve er r0 01 10 00

we eb bs se er rv ve er r0 02 20 00

Ten out of the 12 shingles are identical which indicates a high similarity.

If you set the shingle size to 0 or less, Cookbook treats the string values as words in its text similarity calculation.

For example, if Cookbook receives two alerts with the source IDs: "database01" and "database02", it treats them as:

database01

database02

These two words are not identical so the two alerts would be given a low similarity.

Value Recipe

The first version of the Value Recipe uses a string comparison mechanism to cluster alerts by textual similarity.

Value Recipe uses string metric algorithms to calculate similarity. The calculation breaks strings up into partitions and performs a character-by-character comparison of each partition to measure similarity.

For example, if you set a Cookbook Recipe to cluster alerts with source IDs and descriptions with a Similarity Threshold of 100%, in a scenario where Cookbook receives the following alerts:

Cookbook creates three clusters: one containing alert A, one containing alert B and one containing alerts C and D which have identical source IDs and descriptions. The string may contain non-alphabetical characters. Value Recipe can also convert numeric values to strings for comparison.

The Value Recipe uses the case sensitive property to enable or disable case sensitivity as a factor in text similarity matching. For example, you can enable the Case Sensitivity property for source ID so Cookbook only matches if the case is identical but you can disable it for descriptions if you do not want descriptions to be case sensitive.

If you enable case sensitivity, then an alert from a source called "WebServer1" and an alert from a source called "webserver1" would have a lower similarity.

To make Cookbook match each value in a list individually in custom info fields, check the Match List Items check box in the Cookbook Recipe Clustering tab. See Match List Items in Recipes for details.

Bot Recipe

Bot Recipe is a customizable Cookbook Recipe that allows you to call certain functions from the Cookbook.js Moobot. You can configure the Bot Recipe using the Cookbook Graze API.Graze API

You can configure the Bot Recipe to call functions defined in the Cookbook.js Moolet. The Cookbook Moolet defines two functions, an initialization function called initialize_function and a member_function.

You can call the initialize_function once to set up any necessary initialization of the algorithms you want to write in the Moobot.

You can call the member_function once for every event that passes the trigger. Cookbook considers each of these events for matching and for every candidate cluster in the system. For example, Cookbook calls the member_function 100 times if there are 100 candidate clusters for each alert that comes through the system. Cookbook compares the alert to candidate clusters that are potential Situations. If the alert's similarity matches or exceeds the matcher value, Cookbook adds the alert to the candidate cluster.

Match List Items in Recipes

You can create Recipes and configure clustering around the use of 'custom_info' list-based fields in Alert Custom Info.

You can also set whether list-based clustering of a custom_field is applied. If not, the field will be treated as string.

Configure Match List Items for a Custom Info Field

To match list items for a custom info field:

1.     On the Settings tab, select Cookbook Recipes from the Algorithms section, select the Recipe you want to configure, and click on the Clustering tab.

2.     In the Cluster By field, select the custom_info attribute from the drop-down list. Enter the custom_info field name in the box below.

3.     Check the Match List Items check box to match individual items in custom_info lists and use the slider to select the similarity threshold for this custom_info field.

·        

Example

You configure your Recipe to treat the custom_info field 'cities' as a list and set the similarity to 100%, as shown above.

After configuring the Recipe, Cisco Crosswork Situation Manager receives the following four alerts:

Alert 1: custom_info.offices = ["London"]
Alert 2: custom_info.offices = ["London", "San Francisco", "Venice", "Bangalore"]
Alert 3: custom_info.offices = ["Venice", "Bangalore"]
Alert 4: custom_info.offices = ["Bangalore"]

This configuration would produce four clusters:

·         Cluster A: Alert 1 and alert 2 match on "London".

·         Cluster B: Alert 2 matches on "San Francisco".

·         Cluster C: Alert 3 and alert 4 match on "Venice".

·         Cluster D: Alerts 2, 3 and 4 match on "Bangalore".

This can produce four separate Situations as per the four clusters above, or two Situations because cluster D contains all the alerts in clusters B and C.

If the Recipe does not see 'custom_info.cities' field as a list then it treats the field as a single string. This means in this example all four alerts would end up in separate Situations with no clustering.

Configure a Cookbook

Cookbook is a deterministic clustering algorithm in Cisco Crosswork Situation Manager that creates Situations defined by the relationships between alerts.

Cookbook requires at least one active Recipe in order to function and cluster alerts into Situations. See Configure a Cookbook Recipe for more details.

Before you begin

Before you set up your Cookbook via the UI, ensure you have met the following requirements:

·         You have set up the Recipes you want your Cookbook to use. See Configure a Cookbook Recipe for details.

·         Your LAMs or integrations are running and Cisco Crosswork Situation Manager is receiving events.

Create a Cookbook

To create a new Cookbook from the UI:

1.     Navigate to the Settings tab.

2.     Click Cookbooks in the Algorithms section.

3.     Click the + icon to create a new Cookbook.

4.     Fill in the properties to name and describe the Cookbook:

—      Name: Name of the Cookbook.

—      Description: Text description of the Cookbook.

5.     Configure the Cookbook's input and clustering behaviour:

—      Process Output Of: Defines the source of the alerts for the Cookbook.

—      Cluster By: Determines Cookbook's clustering behavior. You can select First Matching Cluster so Cookbook adds alerts to the first cluster in a Recipe over the similarity threshold value. This is the default behavior for Cookbook. Alternatively, select Closest Matching Cluster to add alerts to the cluster with the highest similarity greater than the similarity threshold value. This option may be less efficient because Cookbook needs to compare alerts against each cluster in a Recipe.

—      Entropy Threshold: Minimum entropy value an alert must have in order for Cookbook to consider it for clustering it into a Situation. Cookbook does not include any alerts with an entropy value below the threshold in Situations. If you set it to 0.0, Cookbook processes all alerts.

—      Cook For: Minimum time period that Cookbook clusters alerts for before the Recipe resets and starts a new cluster. See Cookbook and Recipe Examples for more information.

·         If you set a different Cook For time for a Recipe, it overrides the Cookbook value. Recipes without a Cook For time inherit the value from the Cookbook.

—      Cook For Extension: Time period that Cookbook can extend clustering alerts for before the Recipe resets and starts a new cluster. Setting this value enables the cook for auto-extension feature for this Cookbook. As Cookbook receives related alerts, it continues to extend the total clustering time until the Max Cook For period is reached. Used in conjunction with the Max Cook For value, the Cook For Extension period helps to ensure that Cookbook continues to cluster alerts together that are related to the same failure. The Cook For Extension period only applies to new related alerts; it does not apply to existing alerts that are updated with new events. See Cookbook and Recipe Examples for more information.

·         If you set a different Cook For Extension time for a Recipe, it overrides the Cookbook value. Recipes without a Cook For Extension time inherit the value from the Cookbook.

—      Max Cook For: Maximum time period that Cookbook clusters alerts for before the Recipe resets and starts a new cluster. It works in conjunction with the Cook For Extension time to help ensure that Cookbook continues to cluster alerts together that are related to the same failure. If Cook For Extension is set and this value is not set, it defaults to three times the Cook For value. See Cookbook and Recipe Examples for more information.

·         If you set a different Max Cook For time for a Recipe, it overrides the Cookbook value. Recipes without a Max Cook For value inherit the value from the Cookbook.

—      Scale By Severity: If checked, Cookbook ignores alerts with a severity of 0 (Clear).

6.     Configure which Recipes the Cookbook uses and how it uses them:

—      First Recipe Match Only: Enables you to set a priority order for Recipes in the Cookbook. If you select this check box, Cookbook assigns each alert to the highest priority Recipe where it satisfies the clustering criteria. If unselected, Cookbook assigns an alert to all Recipes where the alert satisfies the clustering criteria.

—      Selected Recipes: Move the Recipes from the Available column to the Selected column to include them in the Cookbook. If you have selected First Recipe Match Only, put the Recipes in the correct order so that Cookbook can determine which Recipe an alert should be assigned to. You should place the highest priority Recipe at the top of the list.

7.     Click Save Changes to create the Cookbook.

Activate the Cookbook

After completing the configuration, you can activate the new Cookbook to run alongside any existing active Cookbooks:

1.     Navigate to the Settings tab.

2.     Click Cookbook Selection in the Algorithms section.

3.     Move the new Cookbook from the Available Cookbooks column to the Active Cookbooks column to make it active.

4.     Click the Advanced tab if you want to configure Cisco Crosswork Situation Manager to purge closed and superseded Situations from Moogfarmd. Define how often you want the purge to occur in hours and minutes.

5.     Click Save Changes to activate the Cookbook.

When you have completed the configuration, Cisco Crosswork Situation Manager applies the changes to the Cookbook as soon as you save the changes.

If you change a Cookbook, see Cookbook Configuration Changes for information on how these changes affect the clusters that Cookbook creates.

Cookbook and Recipe Examples

The following examples provide further explanation of the functionality within your Cookbook and Recipe configurations. See Configure a Cookbook and Configure a Cookbook Recipe for further details.

 

 

Cook For auto-extension example

The Cook For auto-extension functionality uses the Cook For, Cook For Extension and Max Cook For properties in the Cookbook and Recipe configurations. These configuration properties help to ensure that Cookbook continues to cluster alerts together that are related to the same failure.

Consider the following example:

·         Cook For is set to 1 hour.

·         Cook For Extension is set to 30 minutes.

·         Max Cook For is set to 2 hours.

Cisco Crosswork Situation Manager receives an alert which meets the Cookbook and Recipe criteria so Cookbook starts a cluster. If Cisco Crosswork Situation Manager receives a new related alert 40 minutes after Cookbook started clustering alerts, Cookbook extends the total clustering time by 30 minutes from that time to 1 hour and 10 minutes, then:

·         If Cisco Crosswork Situation Manager receives another alert 1 hour and 5 minutes after Cookbook started clustering, because Cisco Crosswork Situation Manager received it within the extended time of 1 hour and 10 minutes, Cookbook further extends the total clustering time to 1 hour and 35 minutes. Cookbook continually extends the total clustering time as it receives more related alerts, provided that they are received within the extended time. Cookbook can extend the total clustering time until the Max Cook For time is reached. If Cookbook receives further related alerts after the Max Cook For time of 2 hours has elapsed, the Recipe resets and Cookbook adds them to a new cluster.

·         If Cisco Crosswork Situation Manager does not receive any further alerts, Cookbook stops clustering alerts after the extended time of 1 hour and 10 minutes elapses. If Cisco Crosswork Situation Manager then receives another alert after this time has elapsed, Cookbook starts a new cluster.

Cookbook Configuration Changes

If you change the configuration of a Cookbook or Cookbook Recipe, Cisco Crosswork Situation Manager may re-evaluate any Cookbook clusters, depending on your persistence setting and the severity of the configuration change. This applies to clusters of alerts that Cisco Crosswork Situation Manager holds in memory and has not yet formed into Situations. It does not affect clusters that have already become Situations that users can see and have been saved in the database.

Your persistence setting affects whether Cookbook re-evaluates clusters as follows:

·         If persistence is turned off, Cookbook resets every cluster and all new incoming alerts will form new Situations.

·         If persistence is turned on, Cookbook updates existing clusters depending on the configuration parameters that have been changed, as described below. Cookbook may remove clusters or create new Situations or it may persist the clusters so that new alerts are added after you have changed the configuration.

Configuration categories

Cisco Crosswork Situation Manager groups Cookbook and Recipe configuration changes into three categories:

·         Cosmetic changes: These configuration properties are non-functional and do not affect how Cookbook creates and maintains clusters. When you change these properties, there are no functional changes to existing clusters.

·         Property changes: These configuration properties affect how Cookbook maintains clusters and generates Situations.

·         Core changes: These configuration properties fundamentally govern how Cookbook creates clusters and groups alerts into Situations.

Cookbook and Recipe configuration properties are grouped into the following categories:

Cosmetic changes

Cosmetic changes to Cookbook and Recipe properties have the following effects on clusters:

Name

If you change the name of a Cookbook or a Recipe, Cookbook makes no operational changes to clustering.

Description

If you change the Situation Description for a Recipe, Cookbook applies the new description to all Situations created after the change. Cookbook maintains the old description for all Situations created before the change, regardless of whether new alerts are added to the Situation.

Property changes

Property changes to Cookbook and Recipe properties have the following effects on clusters:

Alert Threshold

Changes to the Alert Threshold

Reducing the Alert Threshold

For changes to the Alert Threshold, the main behavioral difference occurs when the Alert Threshold is reduced. In the example below, before the configuration change, the Alert Threshold was set to 3 and two alerts had arrived and formed a cluster in memory. If you change the Alert Threshold configuration from 3 to 1, the cluster satisfies the new configuration so Cookbook automatically creates a Situation in the database containing the these two alerts. New alerts coming into the system can continue to be added to this cluster.

Increasing the Alert Threshold

If you increase the Alert Threshold configuration, the cluster will persist in memory until the higher Alert Threshold is reached.

Cook For/Cook For Extension/Max Cook For

Cookbook adopts a similar logic for changes to all three of these attributes because they all affect the cluster's expiry time.

Extending Cook For/Cook For Extension/Max Cook For

If you extend any of these properties, the cluster expiry time is extended from the first event time. In the example below, before the configuration change, the Cook For time is 30 seconds and alerts 1 and 2 arrive at 0 and 25 seconds, respectively. After the Cook For time changes from 30 seconds to 40 seconds, alert 3 arrives at 35 seconds. Cookbook clusters this alert with the persisted cluster from the previous configuration. When alert 4 arrives at 45 seconds, Cookbook creates a new cluster because it satisfies the newly defined Cook For time. Cookbook behaves similarly if Cook For Extension or Max Cook For properties are extended.

Reducing Cook For/Cook For Extension/Max Cook For

If you reduce any of these properties, Cookbook relies on the first event time to establish whether clusters are still valid. In the example below, before the configuration change, the Cook For time is 30 seconds and alerts 1 and 2 arrive at 0 and 25 seconds, respectively. If you reduce the Cook For time from 30 seconds to 20 seconds, the arrival time of the most recent (second) alert exceeds the new Cook For time so Cookbook expires and removes the cluster.

In the second example below, before the configuration change, the Cook For time is 60 seconds and alerts 1 and 2 arrive at 0 seconds and 25 seconds, respectively. If you reduce the Cook For time from 60 seconds to 40 seconds, the cluster still persists because it is still within the new Cook For time so that, when alert 3 arrives at 35 seconds, it joins the existing cluster. Alert 4 arrives at 45 seconds which exceeds the new Cook For time so Cookbook places it in a new cluster.

In the same example, if alerts 3 and 4 had arrived (at 35 and 45 seconds) and the configuration change occurred at 50 seconds, Cookbook would close the cluster immediately with the four alerts in it, as shown below.

Core changes

For all properties that are in the core changes group, any changes cause Cookbook to remove the associated clusters. This is because the fundamental rules on which Cookbook clusters alerts have changed, and it is no longer meaningful to cluster new alerts with old ones.

As an example, consider the example below in which you change the similarity of a Recipe. Initially, the recipe uses a 50% similarity on source ID. Alerts 1 and 2 arrive and Cookbook clusters them together. If you increase the similarity from 50% to 100%, Cookbook removes the cluster from memory. The diagram below shows how confusing it would be if the cluster persisted, visibly seeing a cluster containing alerts which clearly contradict a 100% match on source ID.

This behavior, to remove any old clusters and start new clusters when new alerts arrive, is consistent across all core configuration changes.

Configure Tempus

Tempus is an algorithm in Cisco Crosswork Situation Manager that clusters alerts based on the similarity of their timestamps. See Tempus for more information on how the clustering algorithm works.

Before you begin

Before you set up Tempus via the UI, ensure you have met the following requirements:

·         Your LAMs or integrations are running and Cisco Crosswork Situation Manager is receiving events.

Configure a basic Tempus algorithm

To configure a basic Tempus algorithm from the UI, using the default settings:

1.     Navigate to the Settings tab.

2.     Click Tempus in the Algorithms section.

3.     Click the Active switch to turn on the Tempus algorithm that uses the default settings.

4.     Use the default or enter a description of the Tempus algorithm.

Configure an advanced Tempus algorithm

To configure an advanced Tempus algorithm from the UI, using your own settings:

1.     Navigate to the Settings tab.

2.     Click Tempus in the Algorithms section.

3.     Use the default or enter a description of the Tempus algorithm.

4.     Click Show Advanced to display the Tempus settings.

5.     Configure the algorithm settings for Tempus:

—      Entropy Threshold: Sets the minimum entropy value for an alert to be clustered into a Situation. Tempus does not cluster any alerts with an entropy value below the threshold into Situations. The default of 0 means that all alerts are processed.

6.     Configure the trigger settings for Tempus:

—      Execution Interval: Executes the Tempus algorithm after a defined number of seconds.

7.     Configure the correlation time period settings for Tempus:

—      Time Period: Determines the length of time when Tempus analyzes alerts and clusters them into a Situation each time it runs. Default time period is 1200 seconds (20 minutes). The default time period and bucket size provides 240 buckets per time period.

—      Bucket Size: Determines the time span of each bucket in which alerts are captured. Default bucket size is 5 seconds. The default time period and bucket size provides 240 buckets per time period.

Note:

Cisco does not recommend you change the bucket size. If you do want to change it, then change with caution because Tempus is designed to use small bucket sizes.

—      Arrival Spread: Sets the acceptable latency or arrival window for each alert, in seconds. Use this to minimise or reduce the impact of multiple alerts arriving over a small amount of time and landing in separate buckets. A larger value means a wider distribution to multiple time buckets, and hence more tolerance on arrival time.

1.     Click the Active switch to turn on the Tempus algorithm that uses these settings.

2.     If you want to configure other settings for Tempus, such as the alert threshold, use the Graze API endpoint updateTempus.

Reset to default settings

Click Show Advanced and then click Reset to Defaults to override any changes you have made to the settings and return to the default values.

Set Up Vertex Entropy

Vertex Entropy is a Cisco Crosswork Situation Manager algorithm that indicates the critical nodes within your network and their tendency to produce important events. See Vertex Entropy for more information.

Before you begin

Before you perform the Vertex Entropy calculation and enable its associated features, ensure you have met the following requirements:

·         You have a map of the connected nodes in your network in a comma-separated value (.csv) file.

·         Your .csv topology file contains all of the nodes that you expect to send events.

·         You have run the topology builder utility to import your network topology. See Import a Topology.

Calculate the Vertex Entropy

To enable the features associated with Vertex Entropy, follow these steps:

1.     Import your network topology .csv file into the database using the topology builder. Import a Topology for more information.

2.     Run the graph analyser from $MOOGSOFT_HOME/bin to calculate the Vertex Entropy for the nodes or "vertices" in your network. This is a one-off calculation:

—      ./graph_analyser

—      Graph analyser disregards any weight values included in your imported network topology. See Graph Analyser Command Reference.

Once the graph analyser has been run and each node has a Vertex Entropy value, you can start using these values in other areas of Cisco Crosswork Situation Manager.

Add a Vertex Entropy filter

You can use Vertex Entropy as either a trigger or an exclusion filter in a Cookbook Recipe. These filters include or exclude alerts with similar importance in terms of network connectivity. For example, you can set a trigger so Cookbook considers alerts with a Vertex Entropy value of over 0.5 for Situation creation.

To add a trigger filter:

1.     In your Cookbook Recipe, enter the following in the Trigger Filter:

·         'vertex_entropy' > 0.5

Alternatively, you can create an exclusion filter to prevent alerts with a Vertex Entropy value of less than 0.3 from creating a Situation:

1.     In your Cookbook Recipe, enter the following in the Exclusion Filter:

·         'vertex_entropy' < 0.3

Add a seed alert

You can add a seed alert filter to a Recipe to ensure Cookbook only creates a new Situation if an alert matches the provided filter value.

For example, if you only want to create Situations when there is an issue with the most critical nodes in your network, you can set the seed alert filter to only create Situations from alerts with a Vertex Entropy value of 1.0:

1.     In your Cookbook Recipe, enter the following in the Seed Alert Filter:

·         'vertex_entropy' = 1

If enabled, the initial seed alert must meet both the trigger and seed alert conditions. For more information, see Configure a Cookbook Recipe.

The seed alert filter is not specific to Vertex Entropy and can be used for other conditions such as severity.

Add a hop limit

You can add a hop limit so that Cookbook clusters alerts from nodes within a certain number of hops from each other. You can use this alongside Vertex Entropy trigger or exclusion filters.

In this diagram, a hop limit of '3' means Cookbook includes alerts from all nodes between node A and node D.

To add a hop limit:

1.     In your Cookbook Recipe, check the Hop Limit checkbox.

2.     Enter '3' in the Hop Limit field.

The hop limit ensures that Cookbook only clusters alerts that originated from nodes that are close together. For more information, see Configure a Cookbook Recipe.

Graph Analyser Command Reference

This is a reference for the graph_analyser utility used to calculate /document/preview/11796#UUID8635a39b79fdd302137e104ae42562e8. The graph_analyser command-line utility accepts the following arguments:Vertex Entropy

Import/Export Algorithm Configurations

The Import/Export option enables you to migrate your algorithm configurations from one Cisco Crosswork Situation Manager instance to another. For example, from a test system to your production environment. This option exports Cookbooks and their Recipes, Tempus, the default merge group and all custom merge groups, from one Cisco Crosswork Situation Manager instance and imports them into another.

Export your algorithm configurations

In the Cisco Crosswork Situation Manager instance that you want to export the algorithm configurations from:

1.     Click Import/Export in the Algorithms section of the Settings tab in Cisco Crosswork Situation Manager and select the Export tab.

2.     Click Download to download the file containing all the configurations into a file.

—      The file name format is systemconfig_<date><time>.export.

Import your algorithm configurations

In the Cisco Crosswork Situation Manager instance that you want to import the algorithm configurations into:

1.     Click Import/Export in the Algorithms section of the Settings tab in Cisco Crosswork Situation Manager to display the Import tab.

2.     Click Select File ... and find the export file that you want to use.

—      A list of configuration items that will be imported displays.

3.     Select whether you want to overwrite existing configurations or leave them unaltered. The import function uses the name of the configuration to determine whether it already exists. For example, if it imports Cookbook1 and Cookbook1 already exists, if you have selected Overwrite duplicates, the imported Cookbook1 overwrites the existing one. If you have selected Skip duplicates, the imported Cookbook1 is ignored. If an imported Cookbook has a different name but identical properties to an existing one, the new one will be imported.

4.     Click Import to import your algorithm configurations.

Note:

If you have customized your process_output_of value for any Cookbook or Tempus algorithm so that it processes the output of non-default Moolets, it is best practice to ensure that your imported configuration matches your data flow in Moogfarmd.

Configure Merge Groups

Cisco Crosswork Situation Manager uses merge groups to control the minimum number of alerts in a Situation and how it merges Situations that different clustering algorithms create.

Use merge groups to control:

·         Clustering algorithms that you want Cisco Crosswork Situation Manager to merge similar Situations together.

·         The alert threshold which defines the minimum number of alerts that Cisco Crosswork Situation Manager will cluster into a Situation.

·         Situation similarity threshold which defines the percentage of alerts two Situations must share before they are merged.

You can use the default merge group in Cisco Crosswork Situation Manager or you can create custom merge groups. If you use the default merge group, Cisco Crosswork Situation Manager merges all the Situations that all of your clustering algorithms create if they meet the alert threshold and Situation similarity threshold criteria. You can create custom merge groups to override the default behavior of the default merge group. This is useful not only for adjusting the alert threshold and the Situation similarity threshold, but also if you want Cisco Crosswork Situation Manager to merge Situations with more granularity.

In addition to the alert threshold in a merge group, you can also set an alert threshold in Tempus (via the Graze API) and in Cookbook Recipes (using the Cisco Crosswork Situation Manager UI or the Graze API). When a clustering algorithm considers whether or not to cluster alerts into a Situation, it compares the alert threshold in the merge group and the clustering algorithm. It then uses the higher value to determine how many alerts it requires to create a Situation.addTempusaddValueRecipe

Default merge group

If you do not create any custom merge groups, all the clustering algorithms use the default merge group settings.

The default merge group has a Situation similarity threshold of 0.7. This means that Cisco Crosswork Situation Manager merges two Situations if they have at least 70% of the same alerts.

The default merge group has an alert threshold of 2. If you have a clustering algorithm with an alert threshold of 1, that uses this default value of 2, since Cisco Crosswork Situation Manager uses the higher alert threshold value to determine the number of alerts required to create a Situation, Cisco Crosswork Situation Manager will never create Situations containing a single alert regardless of the alert threshold setting in the clustering algorithm.

If you want the clustering algorithms to create Situations containing a single alert, change the alert threshold in the default merge group to 1. You must use the Graze API if you want to change the default merge group values.

Custom merge groups

If you create a custom merge group for one or more clustering algorithms, they will only merge the Situations they produce among themselves. Situations from clustering algorithms outside of a merge group cannot merge with Situations inside a merge group.

You can configure custom merge groups in the Cisco Crosswork Situation Manager UI or using the Graze API.

Example

You have defined the following clustering algorithms:

·         Tempus algorithm that clusters alerts that arrive in Cisco Crosswork Situation Manager at a similar time.

·         Cookbook 1 with three Recipes; one Recipe clusters alerts on 'Description', another Recipe clusters alerts on 'Host', and the third Recipe clusters alerts with a 'Severity' of Critical (5).

·         Cookbook 2 with a single Recipe that clusters alerts on 'Impacted Services'.

·         Cookbook 3 with a single Recipe that creates Situations containing a single alert with a high entropy value.

If you use the default merge group only, all the Situations created by all these clustering algorithms will be merged if they meet the alert threshold and Situation similarity threshold criteria. But you want greater granularity than that so you create the following custom merge groups:

·         Custom merge group 1 - Cookbooks 1 and 2: Merges clusters created by Cookbook 1 and Cookbook 2 if they meet the following criteria:

—      Alert threshold = null, so it uses the default merge group value of 2. If you create a custom merge group in the UI, the alert threshold is set to null so it automatically uses the default merge group value.

—      Situation similarity threshold = 80%, so it will only merge clusters from Cookbook 1 and Cookbook 2 if they have 80% or more of the same alerts.

·         Custom merge group 2 - Cookbook 3: You want to keep these Situations with a single alert separate so you configure this merge group as follows:

—      Alert threshold = 1, so a single alert clusters into a Situation. This overrides the default merge group value of 2. You must use the Graze API endpoint updateMergeGroup to change this value.

—      Situation similarity threshold = 100%, so unless the alerts in two Situations are identical, the Situations will not be merged.

—      You do not create a custom merge group for Tempus so it will use the default merge group values of:

—      Alert threshold = 2.

—      Situation similarity threshold = 70%.

Updating the default merge group

You must use the Graze API to update the default merge group. You cannot update it using the UI. See the following topics for instructions on updating or viewing the details of the default merge group:

·         getDefaultMergeGroup: Returns details of the default merge group in Cisco Crosswork Situation Manager.

·         updateDefaultMergeGroup: Updates the default merge group in Cisco Crosswork Situation Manager.

Configuring custom merge groups

You can configure a custom merge group in the UI or using the Graze API.

Before you begin

Before you create a custom merge group, ensure you have met the following requirements:

·         You have configured at least two different clustering algorithms, for example, Cookbook and Tempus.

·         Your LAMs or integrations are running and Cisco Crosswork Situation Manager is receiving events.

Configure a custom merge group in the UI

To configure a custom merge group in the UI:

1.     Navigate to the Settings tab.

2.     Click Merge Groups in the Algorithms section.

3.     Select an existing custom merge group and click Edit, or click Add Merge Group to add a new one.

4.     Configure the custom merge group settings:

—      Name: Enter a name for the custom merge group.

—      Sigaliser: Select the clustering algorithms to include in the custom merge group. To include additional clustering algorithms, click Add Sigaliser.

—      Similarity Threshold: The percentage of alerts two Situations must share before they are merged. Enter a value between 0 and 100. The default similarity threshold in the default merge group is 70%. Set a lower value if you want Cisco Crosswork Situation Manager to merge Situations with a lower percentage of alerts shared between them, which is likely to increase the number of Situations that will merge. Set a higher value if you want to decrease the number of Situations that Cisco Crosswork Situation Manager will merge. If you set this value to 0, Cisco Crosswork Situation Manager uses the default merge group value.

5.     Click Save to finish configuring the custom merge group.

6.     If you want to change the alert threshold for this custom merge group, use the Graze API endpoint updateMergeGroup.

After you configure a custom merge group, Moogfarmd automatically restarts and begins using it.

Configure a custom merge group using the Graze API

See the following topics for instructions on creating custom merge groups via the Graze API:

·         addMergeGroup: Adds a new custom merge group.

·         deleteMergeGroup: Deletes an existing custom merge group.

·         getMergeGroups: Returns details of all the custom merge groups in Cisco Crosswork Situation Manager.

·         updateMergeGroup: Updates a custom merge group.

Field behavior

When Cisco Crosswork Situation Manager merges two or more Situations, it updates the fields of the Situations as follows:

Troubleshooting and Diagnostic Tools

As an Administrator, you can set up tools that enable operators to troubleshoot and diagnose Situations.

You can configure server tools to execute utilities on a remote host. These can be generic tools, or specific to Situations or alerts.

You can also set up client tools, to retrieve or send information to a client, to help diagnose problems in Situations or alerts.

Cisco Crosswork Situation Manager provides a number of default hotkeys but you can set up additional hotkey shortcuts to make navigation easier for Cisco Crosswork Situation Manager users.

You can also set up ChatOps shortcuts to enable users to run tools from the Collaborate tab in the Situation Room. This gives Cisco Crosswork Situation Manager users quick access to the available tools.

Implement Generic Server Tools

Generic server tools in Cisco Crosswork Situation Manager are tools that allow a user to execute a utility on a remote host.

These tools specify a command that is run using ToolRunner, which is configured to connect to the remote host. The command can be anything that is executable from the Linux command line. For example a ping or cat or a custom bash script. See Configure ToolRunner for more information.

In Cisco Crosswork Situation Manager, the generic server tools managed here are available from Situation Room ChatOps feature. See Take Additional Actions for details.

The steps below describe how to create a generic server tool and its command. Any arguments required are defined by the user when the tool is run.

You can make the tool available to all users, specified teams, specified roles, or yourself only.

Create a new generic server tool

1.     To create a new generic server tool:

2.     Click Generic Server Tools in the Tools section of the Settings tab.

3.     Click + to create a new tool.

4.     Fill in the available fields to define the tool:

5.     On the Shared With tab, select whether you want to share this tool with everyone, specific teams, specific roles or only yourself. You must have the permission share_filters_public to share a tool with all users. You must have the permission share_filters_teams to share a tool with specific teams. See Manage Roles for more information.

6.     If you select to share the tool with specific teams or specific roles, add the teams or roles you want to share the tool with to the list below.

7.     Click Save Changes to add the tool to the list of generic server tools in the left hand pane.

Example

A generic server tool with the following command runs the script myTests.sh which is located on the remote host at the path /home/moog/bin, using remote host access information defined in the ToolRunner servlet:

/home/moog/bin/myTests.sh

Implement Situation Server Tools

Situation server tools in Cisco Crosswork Situation Manager are tools that enable a user to execute a utility on a remote host.

These tools specify a command that is run using ToolRunner, which is configured to connect to the remote host. The command can be anything that is executable from the Linux command line. For example a ping or cat or a custom bash script. See Configure ToolRunner for more information.

·         The command can be anything you can run on the host in a Linux terminal command line, such as an inbuilt part of the operating system, such as ping, or your own script.

·         The arguments are extracted from Situation attributes by prefixing the attribute name with '$', such as $description for the Situation description.

In Cisco Crosswork Situation Manager, the Situation server tools managed here are only available from ChatOps in the Situation Room. See Take Additional Actions in the Operator Guide for more information.

The steps below describe how to create a Situation server tool, its availability filter, command and arguments. You can also create Situation server tools via a command prompt.

You can make the tool available to all users, specified teams, specified roles, or yourself only.

Create a new Situation server tool

To create a new Situation server tool:

1.     Click Situation Server Tools in the Tools section of the Settings tab.

2.     On the Tool tab, click the + to create a new tool.

3.     Fill in the available fields to define the tool:

·         To prevent substitution with potentially malicious commands, arguments are escaped using a backslash.

·         For example:

·         Command: echo

·         Argument: $args, where $args is echo_something; rm file.txt

·         This results in the following command being executed:

·         echo echo_something\; rm file.txt

·         The semi-colon is escaped to prevent the rm command from being run.

4.     On the Shared With tab, select whether you want to share this tool with everyone, specific teams, specific roles or only yourself. You must have the permission share_filters_public to share a tool with all users. You must have the permission share_filters_teams to share a tool with specific teams. See Manage Roles for more information.

5.     If you select to share the tool with specific teams or specific roles, add the teams or roles you want to share the tool with to the list below.

6.     Click Save Changes to add the tool to the list of Situation server tools in the left hand pane.

Example

The screenshot below shows a Situation server tool called 'LogSitnDetails' with the Command: /home/moog/bin/logger.sh.

This tool runs the script logger.sh on the remote host which logs Situation details to a file. The details logged are the Situation ID, created time, description and total number of alerts, which are defined with the Arguments: ig_id $created_at $description $total_alerts.

Each Situation attribute name is prefixed with $. The Context Filter makes this tool available only for Closed Situations.

Create a tool with a command prompt

You can create Situation server tools via a command prompt. This is useful for efficient creation of multiple tools using a scripted process, for example:

·         Open a new Terminal window on the Cisco Crosswork Situation Manager system and type the following:

—      moog_add_sitn_server_tool

·         Type any flags and arguments for the tool settings. See the examples below.

Note:

Cisco Crosswork Situation Manager command line tools are located here:

$MOOGSOFT_HOME/utils

To display the help information for this tool, type:

—      moog_add_sitn_server_tool and press Enter.

—      Use a double-dash prefix "--" to define all following text as arguments. This ensures arguments are not misinterpreted as flags.

—      For example, "-- -c" to define the argument "-c", which would otherwise be interpreted as the command flag.

—      If included, the Run For time must be 5 seconds or longer.

·         When you have defined the tool, press Enter. If successful, "Tool was added" appears.

Once the UI is refreshed, newly created tools appear in the Situation server tools configuration window.

Examples

The following example creates a Situation server tool to return the Situation ID:

moog_add_sitn_server_tool --name "Sitn Id" --desc "Get the Situation ID" --cmd echo --args "Situation ID = \$sig_id" --run_for 42

This command creates a tool with the following settings:

·         Name: Sitn ID (--name "Sitn ID").

·         Description: Get the Situation ID (--desc "Get the Situation ID")

·         Context Filter: none

·         Command: echo (--cmd echo)

·         Arguments: display 'Situation ID = ID'

·         (--args "Situation ID = \$sig_id"). The backslash is required to escape the '$' because it is an environment variable.

·         Run for: 42 seconds (--run_for 42)

The following example creates a Situation server tool that pings the server five times:

moog_add_sitn_server_tool -d "five pings" -m "sig_id<10" -c ping -a -- -c 5

This command creates a tool with the following settings:

·         Description: five pings (-d "five pings")

·         Context Filter: ID < 10 (-m "sig_id<10")

·         Command: ping (-c ping)

·         Arguments: ping five times (-- -c 5). The argument starts with -c which is itself a tool flag. Therefore the "--" double-dash prefix is used to interpret -c 5 as an argument, and not a flag.

·         Run for: no time set (no -r flag and argument)

·         Name: ping. The name is not defined here (no -n flag and argument) so the command is used as the name by default.

Implement Alert Server Tools

Alert server tools allow you to execute a utility on a remote host. You define the host when you run the tool. You can control which tools are available for different types of alerts.

There are two ways of running alert server tools:

·         If you run them via ChatOps shortcuts, they always run on the ToolRunner host.

·         If you run them from the Tools menu in an Alert List, they run on the host of your choice.

These tools specify a command that is run using ToolRunner, which is configured to connect to the remote host. The command can be anything that is executable from the Linux command line. For example a ping or cat or a custom bash script. See Configure ToolRunner for more information.

Alert server tools pass arguments to utilities based upon alert attributes. For example, testing the reachability (ping) of hardware using the source attribute of the alert.

You can make the tool available to all users, specified teams, specified roles, or yourself only.

Create a new alert server tool

To create a new alert server tool:

1.     Click Alert Server Tools in the Tools section of the Settings tab.

2.     On the Tool tab, click the + icon to create a new tool.

3.     Fill in the available fields to define the tool:

4.     On the Shared With tab, select whether you want to share this tool with everyone, specific teams, specific roles or only yourself. You must have the permission share_filters_public to share a tool with all users. You must have the permission share_filters_teams to share a tool with specific teams. See Manage Roles for more information.

5.     If you select to share the tool with specific teams or specific roles, add the teams or roles you want to share the tool with to the list below.

6.     Click Save Changes to add the tool to the list of alert server tools in the left hand pane.

Example

The following screenshot shows an alert server tool that tests the reachability of the source alert and returns the results.

The Command ping is used with Arguments$source and -c5 which specify the source, from the alert attribute, and the number of times to ping (five).

The Alert Type Filter uses a regular expression '.*' to make the tool available for all alerts.

Implement Client Tools

You can create client tools in Cisco Crosswork Situation Manager to execute actions through a specified URL. The client tools can send Situation and alert data to the URL.

Client tools can return HTTP response data. The response data includes the HTTP response and HTTP message body. See HTTP Response for more information. Providing a more detailed response in the UI, that includes response status and data, can yield useful and important information. For example, tools to link to an external trouble ticket system (via its URL) can open a new ticket using data from the selected Situation.

There are two different types of client tools: alert client tools and Situation client tools.

You can make the tool available to all users, specified teams, specified roles, or yourself only.

Create a new client tool

To set up a new alert client tool or a new Situation client tool.

1.     Click Alert Client Tools or Situation Client Tools in the Tools section of the Settings tab.

2.     On the Tool tab, click the + icon to create a new tool.

3.     Fill in the available fields to define the tool:

4.     Select whether you want to create a URL Tool or you want to Merge Custom Info.

·         When creating a client tool, you can enter Situation or alert attributes, and prompt variables, in the URL, Custom Info, and URL Encoded Content fields. For example, you can enter $description for the contents of the Situation or alert description field.

5.     To create a tool that uses a URL, complete the following fields:

6.     To create a tool that uses custom info fields, enter valid JSON for the custom info you want the tool to add. The JSON can include Situation and alert attributes and prompt variables. For example, the following JSON blob adds a set of custom info called "TPS data" that contains a string "From system", the Situation ID and the timestamp for when the Situation was created:

—      {"TPS data": ["From system", "$sig_id", "$created_at"]}

—      To add a prompt variable, type '$' and select an existing variable or enter your own variable. If you enter a new prompt variable, see Add prompt variables for details.

7.     Prompts appear in the Prompts table if you have added them in the URL, URL Formatted Content, or Custom Info fields. See Add prompt variables for details.

8.     On the Shared With tab, select whether you want to share this tool with everyone, specific teams, specific roles or only yourself. You must have the permission share_filters_public to share a tool with all users. You must have the permission share_filters_teams to share a tool with specific teams. See Manage Roles for more information.

9.     If you select to share the tool with specific teams or specific roles, add the teams or roles you want to share the tool with to the list below.

10.  Click Save Changes to add the tool to the list of alert client tools or Situation client tools in the left hand pane.

Add prompt variables

Prompt variables open a message box when the tool is run, prompting the user to type text, a number, or select from a list.

To add a new prompt variable:

1.     In the URL, URL Formatted Content, or Custom Info fields, enter prompt variables in the following format:

—      $<prompt_name>

—      The prompt name cannot be any of the existing Situation or alert attribute names. The prompt name appears in the Prompts table.

2.     To edit the prompt, double-click on it, or select it and then click Edit Prompt. A new window displays.

3.     Enter a Display Name. This is what appears in the prompt message.

4.     Select whether the user will be prompted for text, a number or a list.

—      If you select Text or Number, enter the default value and minimum and maximum length of the prompt. Numbers can be integers or floating point, in which case they are truncated to two decimal places.

—      If you select List, enter the default value and add the available list options.

5.     Click Save Changes to add the new tool appears in the list in the left hand pane.

Client custom info tool example with a prompt variable

You can configure client tools to change custom info fields. For example, you can configure a tool to raise a ticket on a third party system to prompt for entries of pre-defined (custom info) values to provide more information in the ticket.

To create a client custom info tool with a prompt variable, select the Merge Custom Info option:

In this example, the custom info is:

{"LEVEL": $prompt1}

The screenshot below shows how the prompt variable settings can be configured:

Run client tools

·         The client tools can be accessed from the following areas:

·         Alert Client Tools: On the Alert Tools Menu, see Alerts Overview (Right-Click menu). Or via "Situation Alerts" in a Situation Room.

·         Situation Client Tools: The Situation Tools Menu, from the Tools menu on the Situation Room or via ChatOps in the Collaborate tab.

If you want to run client tools using Safari, go to Safari > Preferences > Security and uncheck 'Block pop-up windows' as this is checked by default.

Run client tool example

To run a tool 'Set LEVEL data for TPS':

1.     Go to an alert, right-click or click Tools > Tools > 'Set LEVEL data for TPS'.

The following prompt appears:

2.     Click OK to run the tool.

Configure Hotkeys

Hotkeys are keyboard shortcuts you can use on the Alert View, Situation View and Situation Room screens in Cisco Crosswork Situation Manager. The default hotkeys are as follows:

You can add more custom hotkeys for additional actions to make navigation easier for you and your team.

Add a Hotkey

Click + Create Hotkey in the top left corner of the window to add a new hotkey shortcut.

Under 'Key', select a number between 0-9 or a letter between A-Z then select an action for it to represent. The new hotkey is highlighted with orange markers.

Click Save Changes to continue.

Change an existing Hotkey

Select any assigned Action from the list, click on the KEY dropdown menu and select from any available alphanumeric character.

The new key assigned to the action in the list will have a red indicator informing the user that the change is ready to be saved.

Delete a Hotkey

Select any unwanted hotkey from the list, default hotkeys included, and click - Remove Hotkey. The selected hotkey disappears from the list.

Click Save Changes to continue or click Revert Changes to undo the action.

Configure ChatOps Shortcuts

The ChatOps feature enables Cisco Crosswork Situation Manager users to run tools from the Collaborate tab in the Situation Room. For more information, see Take Additional Actions in the Operator Guide.Take Additional Actions

The ChatOps shortcuts can be set up to give your users quick access to the available tools. To do this, go to System Settings > ChatOps Shortcuts:

Any existing shortcuts are listed under the Tools panel to the right of the window.

Create a ChatOps Shortcut

Click the + Create Shortcut button to get started.

Next enter a name for your new ChatOps shortcut in the Shortcut text box. This is what users enter to run the ChatOps tool.

Note: You can use 0-9, lowercase a-z, dash (-), underscore (_) and period(.) If the name already exists, the name highlights in red in the list on the left and you cannot save until the name is edited to be unique.

Select the checkbox for the tool you want. To search for a tool, type text to the right of the search icon. Default alert and Situation workflow tools, such as ping, nslookup, are available.

Repeat the steps above to create as many shortcuts as required.

Click Save Changes when you are finished.

Remove a ChatOps Shortcut

Select the ChatOps shortcut you want to delete, from the list on the left.

Click the - Remove Shortcut button to remove the shortcut from the list.

If you accidentally remove the wrong shortcut, you can click Revert Changes to undo this. This discards all changes since the last save.

Additional System Configuration

As an administrator, you can perform additional system configuration to create an effective Cisco Crosswork Situation Manager working environment for your organization.

You can perform the following actions:

·         Configure which columns display on Situation and alert views and the order in which they appear.

·         Select the landing page that displays when users open Cisco Crosswork Situation Manager.

·         Create global filters for all users, and team filters for selected teams, to restrict the alerts and Situations that users can access. Individual users can create their own personal filters.

·         Set the default tab that displays when a user enters a Situation Room.

·         Define the effect on alerts when actions are performed on the Situations they are in.

·         Set the period of inactivity before a user is logged out.

·         Select system settings such as date format and time zone.

·         Enable Probable Root Cause (PRC) and select which users can access it.

·         Enable and disable the collection of statistics for Insights.

·         Configure Early Access Features. These features are for evaluation only and should not be used in production environments.

Create Shared Alert and Situation Filters

You can use filters to configure which alerts and Situations you want to access and display from the Cisco Crosswork Situation Manager Workbench.

You can choose whether the filter is available to all users, specified teams, or yourself only.

Create a Filter

To create an alert or Situation filter:

1.     Click Alert Filters or Situation Filters in the Display Options section of the Settings tab.

2.     On the Filter tab, click the + icon to create a new filter.

3.     Fill in the available fields to configure the filter:

4.     Define the filter that you want to use:

—      Click Add Clause to start building the filter.

—      Click the drop-down menu arrow and select a parameter.

—      Click the drop-down menu below this and select an operator.

—      Depending on the parameter selected, enter or select a value in the final box and then click Apply.

—      To add more clauses, click the clause and then click AND, OR, or NOT and fill in the boxes as before.

5.     On the Shared With tab, select whether you want to share this tool with everyone, specific teams, or only yourself.

6.     If you select to share the filter with specific teams, add the teams you want to share the filter with to the list below.

7.     Click Save Changes to create the new filter.

See Filter Search Data for filter examples.

Configure the Landing Page

You can configure the landing page users land on when they open Cisco Crosswork Situation Manager. You can set the landing page for each user's role, for each team and for the system. The default system landing page is the Summary screen. The landing page a user adopts follows the priority order: role > team > system.

Set a Role's Landing Page

To configure the landing page for a role:

1.     Go to System Settings > Roles.

2.     Select the Role you want to edit.

3.     Set the 'Landing Page' and save your changes.

See Manage Roles for more details.

Configure other Landing Pages

You can also configure landing pages for teams and the Cisco Crosswork Situation Manager system in the System Settings. To add landing page for a team see Manage Teams. To add a landing page for your system see Customize User Experience and Workflow.

Customize User Experience and Workflow

You can configure the functionality and default tabs of Cisco Crosswork Situation Manager for Situation Rooms, Workflow, System Settings and Interface Settings. To access these options, click Customization in the System section of the Settings tab.

Situation Room

Use the Situation Room tab to select the default tab that displays when any user opens the Situation Room and the columns that appear in the Situation Room header.

Default tab

You can choose between Next Steps, Alerts, Collaborate, Topology, and Visualize as the default tab that displays. By default, the tab is Next Steps. To change the default tab, select the default tab that you want from the drop-down list.

Situation Header Columns

You can select up to eight columns to display in the Situation Room header. To change the columns that you want to display in the Situation Room header:

·         To add a new column, click Add Field and select a column from the drop-down list.

·         To delete an existing column, click x next to the column name.

·         To move a column, click on a column and drag it to the place where you want it to display.

Workflow

The Workflow tab allows you to alter the standard workflow of all Situations.

Propagate Situation actions to alerts

When enabled, this mirrors any actions to the Situation down to its alerts. Click Enable to continue and then define the scope:

Close Situations

These settings determine the behavior when closing Situations:

System Settings

The System Settings tab is where you can set when Cisco Crosswork Situation Manager times out, either with the system default or a custom timeout:

Interface Settings

The Interface Settings section is where you configure the default time format, landing page and theme for the Cisco Crosswork Situation Manager interface.

Note:

* These are the default settings for Cisco Crosswork Situation Manager.

Upload a logo

You can upload a custom image such as a company or brand logo to appear on the login page and on the top bar of the workbench:

·         Click Upload and select the desired image file from your computer. Currently PNG, JPG and GIF files up to 5MB in size are supported.

·         Click Save Changes for the changes to be applied.

·         Click the image to go to the default or configured landing page.

Create Link Definitions

Link Definitions allow you to format alert and Situation view custom info values as links to external tools for individual alerts or Situations.

You can add Link Definitions to columns in alert and Situation views. You can configure links using Situation and alert data and custom info fields. You can use them to perform queries and to link directly to corresponding tickets in third-party ticketing systems.

Create a Link Definition

To create a new Link Definition:

·         Go to Settings > Link Definitions and click + to create a new definition. Complete the details for the Link Definition as follows:

—      Name. Name of the Link Definition.

—      Link. Link format, including query syntax if required. It can include a $reference to one or more alert or Situation fields.

—      Display. Link text to appear in Situation and alert views. It can include the $value of a referenced field.

·         To display the Link Definition in a Situation or alert view column, go to Settings > Situation Columns and Settings - Alert Columns. See Configure Alerts and Situation Columns for more information.

The following screenshot displays two example Link Definitions, ServiceNow Ticket and Google Link.

Third Party Ticketing Example

The Link Definition for ServiceNow Ticket is configured to take the value of the custom info servicenow field, and also displays this value as part of the link itself:

·         Name: ServiceNow

·         Link: https://instance.service-now.com/nav_to.do?uri=incident.do?number=$value

·         Display: ServiceNow $value

The Situation column is configured to display data from the custom_info servicenow field and links to the ServiceNow link definition:

·         Field: custom_info.servicenow

·         Header: ServiceNow Ticket

·         Type: Text

·         Link Definition: ServiceNow

Situation #25 has the following custom_info:

·         Name: servicenow

·         Value: INC1190341

In the screenshot example, the ServiceNow INC1190341 link for Situation #25 contains the following URL: https://instance.service-now.com/nav_to.do?uri=incident.do?number=INC1190341

Query Example

The Link Definition for Google Link is configured to perform a google query for the text strings in the $description Situation field and the custom info impact field:

·         Name: Google Description

·         Link: https://www.google.com/search?q=$description&q=$custom_info.impact

·         Display: Google Description

The Situation column is configured to link to the Google Description link definition. In this example, the Field setting is ignored, but you must still enter a valid custom info attribute into it to make the link appear:

·         Field: custom_info.impact

·         Header: Google Link

·         Type: Text

·         Link Definition: Google Description

Situation #26 has the following custom_info:

·         Name: impact

·         Value: documentation

In the screenshot example, the Google Description link for Situation #26 contains the following URL: https://www.google.com/search?q=Service%20Pack%20Update%20SP21&q=documentation

Configure Alert and Situation Columns

You can change the columns to display on Situation and Alert Views and add new columns based on custom_info fields. Optionally add link definitions to custom_info columns, for example, to link the custom_info data to a third-party system. See Link Definitions.

Navigation

Click the Columns drop-down menu in the top right corner to display which columns are displayed by default. Check or uncheck columns to add or remove them from the default column layout.

Click any column and edit the text next to 'Header' to change the column header.

Create a New Column

Click Columns > Add Column to add a new custom column to the default layout. Edit the available fields to configure the column:

Adjust the column width as required and change the order by dragging and dropping the new column where you would like it to be. Click Save Changes to continue and confirm when prompted. Alternatively, click Revert Changes to discard your changes.

Example

This example walks you through setting up an Alert Column with Custom_info Data from Prompt. The custom_info field 'TPSLEVEL' is added to Alerts using a client tool with a prompt variable: 'Set TPS Level'. See Client Tools for information on how to set up the TPS Level client tool.

1.     Right-click and select Tools > Set TPS Level tool or Tools > Tools > Set TPS Level to run the tool on an alert.

2.     Select the TPS level on the prompt window.

3.     Right-click on the alert, select Show Details... and Custom Info...

4.     Navigate to System Settings > Columns > Alert Columns to create the custom_info column.

5.     Click Columns > Add Column and then configure the column.

6.     Click Save Changes to continue.

The Alert Views window displays the TPS Level column display custom_info data in the second column.

Configure and Retrain Probable Root Cause

Probable root cause (PRC) for Cisco Crosswork Situation Manager is enabled by default. This means that you can mark Situation alerts as having a root cause or not and Cisco Crosswork Situation Manager shows a root cause estimate in Next Steps in the Situation Room.

Navigate to System Settings > Configure & Retrain to enable or disable PRC.

To configure which users can mark alerts for PRC, go to Security > Roles. Select the role you want to edit and under Permissions, move 'prc_feedback' to 'Selected' using the direction arrows. This permission is enabled for Administrator and Super User roles by default.

Root Cause

The PRC Model gives each alert within a Situation a probable root cause estimate. Retrain recalculates the estimates with the current data. You can choose which features your model uses when predicting the probable root cause of an alert. The default PRC configuration uses two types of Severity. The other features are listed below:

Statistics Collection

You can disable and enable the collection of different statistics for Insights. See Grafana Dashboards for more details.

Navigate to Settings - Insights to configure statistics collection. Available collections that populate the default Grafana dashboards include:

·         Team Room Overview

·         Ops Insights

·         Noise Reduction Insights

·         Team Ops Insights

·         Team Performance Insights

·         Individual Performance Insights

·         UI Dashboards

All collections are active by default. Uncheck a collection if you do not want to use that particular dashboard. You might want to disable collections and their related processes if you are only interested in retrieving a specific set of statistics or you want to keep the load on your system to a minimum.

Configure Labs Features

Cisco Crosswork Situation Manager Labs offers a preview of unreleased features. Navigate to Settings > Labs > Configure to view available Labs features for the current release.

Available Early Access features are:

·         Visualize: Allows administrators and implementers to view the Visualize tab in a Situation Room that shows which process created the Situation, diagrams showing the similarity of the alerts in the Situation, and a list of the alerts in the Situation. See Situation Visualization for details.

·         Workflow Engine: Allows you to define custom workflows for events, alerts and Situations. See Workflow Engine for more information.

·         New Grid: Activate the new grid for Situation and alert lists. It has improved performance, better scrolling, and is easier to use compared to the old grid. Some actions, such as drag and select rows, and copy and paste, behave slightly differently in the new grid.

Schedule Maintenance Downtime

You can schedule maintenance windows so that events created during these maintenance periods are not included in Situations. Defining maintenance windows for scheduled downtimes, such as during server or software upgrades, reduces unnecessary noise.

During a maintenance window, events continue to be correlated into alerts and labeled as 'In Maintenance' but you can choose not to group them into Situations. If an alert under a maintenance schedule receives an event, it is tagged as such.

View Maintenance Windows

To view maintenance windows, click the Maintenance Schedule heading in the Side Menu. If you are an Administrator, you can edit one of the displayed windows by double-clicking it.

Historical, expired and manually deleted windows are not displayed here.

Create a Maintenance Window

Click Create Maintenance Window to create a new window. Complete the following:

Cisco Crosswork Situation Manager adds the new maintenance window in the time zone of the user who created it. When scheduling recurring maintenance windows, Cisco Crosswork Situation Manager takes into account any daylight savings time changes for the time zone. See IANA Time Zone Database for more information on valid time zones.

Situation Visualization

Caution

The Visualize tab is a "Labs Feature". Enable the Visualize feature in Labs to view it in the Situation Room. See Configure Labs Feature for details. Your role must have the "sig_visualize" permission to use this feature. See Manage Roles for details.

The Visualize tab in Situation Rooms allows administrators and implementers to see:

·         The Cookbook and Recipe used to create the Situation.

·         A visual representation of the similarity of the alerts within the Situation to the reference alert. The reference alert is either the seed alert, if a Cookbook Recipe is configured this way, or it is the first alert that the clustering algorithm assigned to the Situation.

·         A list of all the associated alerts in the Situation.

You can use this information to adjust your Cisco Crosswork Situation Manager configuration to improve the relevance of the Situations it creates.

To view the Visualize feature, go to a Situation Room and click the Visualize tab.

Note:

Currently, Cisco Crosswork Situation Manager does not fully handle alerts that a user has manually added to a Situation in this Early Access version of the Visualize feature. For example, manually added alerts do not display in a similarity diagram if their similarity to the reference alert is below the threshold for a component but they will appear in another diagram if their similarity is above the threshold for that component.

Currently, the similarity diagrams only work with core alert fields. If Cisco Crosswork Situation Manager uses custom info fields to cluster alerts, the similarity diagrams do not display.

Similarity diagrams

The Visualize tab shows diagrams of the alerts in the Situation according to how the Cookbook Recipe has clustered them. In the example below, this Situation has ten alerts which are clustered by two components: Description and Source ID. The Cookbook Recipe clusters alerts whose Description is at least 20% similar to the reference alert and whose Source ID is also at least 20% similar to the reference alert. The reference alert may be a seed alert or simply the first alert that the Cookbook Recipe added to the cluster.

Each diagram shows the similarity of the alert to the reference alert for one of the components. Each alert displays as a dot on the diagram on a spoke representing the sequence it was clustered into the Situation. The reference alert has a similarity of 100% and displays at the center of the circle. Alerts with a high similarity display closer to the center of the circle and alerts with a low similarity display nearer the edge of the circle. In the example below, alerts that are only 20% similar would display at the edge of the circle.

Representation of the alert in the center of each diagram is as follows:

·         Yellow dot: Single reference alert, with no other alerts having 100% similarity.

·         Blue dot with a single concentric blue circle: Reference alert plus one alert which has a 100% similarity match to the reference alert.

·         Blue dot with two concentric blue circles: Reference alert plus two or more alerts which have a 100% similarity match to reference alert.

You can perform the following actions on the similarity diagrams:

·         Hover over an alert in a diagram to display the similarity of that alert to the reference alert for that field.

·         Click on an alert in a diagram to display the details of that alert in a pane on the right hand side of the window.

·         Use the sliders below each diagram to increase the similarity value. Alerts that are outside the selected similarity appear gray. This feature enables you to determine whether a higher similarity would improve the Situation. In the example above, the Cookbook Recipe clusters alerts into a Situation if the Description has more than 20% similarity to the reference alert. You may find that alerts with a similarity of less than 29% are not really relevant to the Situation. In this case, you could consider changing the Description similarity to 29%.

Alert list

The Visualize tab displays a list of all the alerts in the Situation. The reference alert always displays at the top of the alert list.

You can enter a filter in the Filter field to display alerts in the Situation that match it. See Filter Search Data for information on creating filters.

Self Monitoring

Administrators can use Self Monitoring to view the status, health, and processing metrics of the Cisco Crosswork Situation Manager processes. The different tabs show the state of Processing Metrics, Event Processing, Web Services, Event Ingestion and Message Bus.

Heartbeats are one of the key concepts in Self Monitoring. A heartbeat is an internal message sent by a process every 10 seconds to inform Self Monitoring that it is still running.

All data displayed in this screen is live and updates continually.

Package States

The table below describes the possible states for a package:

You can set processes as reserved or unreserved in the system.conf file ($MOOGSOFT_HOME/config/system.conf. If a package's 'reserved' setting is 'true', the self monitoring reports a warning if the package is not running. Stopped unreserved processes do not generate warnings.

Controls

There are a number of controls in Self Monitoring that can be used to stop, start and restart Moogfarmd and the LAM services:

These can be configured by users with Super User permissions.

Self Monitoring Tabs

The Self Monitoring screen is divided into a number of tabs. Each section displays the states of the various processes, indicating which are running or which have issues:

·         Processing Metrics

·         Event Processing

·         Web Services

·         Event Ingestion

·         Message Bus

Processing Metrics

This tab, which is open by default when Self Monitoring is launched, displays event processing times and other metrics.

The icon in the top left corner indicates the overall state of event processing. This is determined by the Current Maximum Event Processing Time in seconds. This time is indicated by the position of the gray bar on the colored bullet graph shown below. The Current Maximum Event Processing Time is 1.917s in this example:

The default bullet chart color values are as follows:

·         GREEN (0 - 10 seconds) Good performance

·         YELLOW (10 - 15 seconds) Marginal performance

·         RED (15 - 20 seconds) Poor performance

The time values are configurable in the web.conf file.

Using Processing Metrics

To use the Processing Metrics tab, open the LAMs and moog_farmd folders and look for deviations from normal values.

The numeric value itself may not be an absolute measurement of health, so as a general rule, look for unusual or sudden changes in the values or behavior. See the examples below:

·         If a particular LAM becomes a data flow bottleneck, expect to see substantial increases in the values for the Message Queue Size and/or Socket Backlog metrics for that LAM. This leads to an increasing Event Processing Time for the appropriate Moogfarmd (which is expecting data from the LAM).

·         If an AlertRulesEngine in a Moogfarmd instance becomes a data flow bottleneck, expect to see a substantial increase in the Message Backlog and possibly the Messages Processed decreasing for that AlertRulesEngine. This also leads to an increasing Event Processing Time for the Moogfarmd.

Both of these result in the bullet chart (at the top) showing increasing Current Maximum Event Processing Time, from green to yellow to red.

Event Processing

This tab contains a process group including Moogfarmd (the core Cisco Crosswork Situation Manager application) and the Moolets, such as AlertBuilder, Alert Rules Engine, Sigalisers.

The icon in the top left corner indicates the overall state (running normally in the example above). The group and cluster names are displayed in the top right corner. The time and date of the last heartbeat is displayed above the list of Moolet processes.

Web Services

This tab contains all processes related to Tomcat web applications: moogsvr, moogpoller, toolrunner and Graze.

Each row displays the following information:

Note:

* These only apply to High Availability deployments where there are more than one instance of Cisco Crosswork Situation Manager and its component processes.

Event Ingestion

This tab displays information about the state of all processes relating to the LAMs and the individual processes which process raw data and create events:

The controls in the far right column can be used to stop and restart active LAM processes or to start inactive LAMs.

Message Bus

The final tab provides a link to the Message Bus Console, also known as the MooMs (Moogsoft Messaging System). This is hosted by message-queueing software RabbitMQ.

Click the link to proceed to the RabbitMQ management console.

The username and password to log in are specified and can be configured in $MOOGSOFT_HOME/config/system.conf (under mooms.username and mooms.password in the JSON) and correspondingly in RabbitMQ. See Configure the Message Bus for more information.

Once logged in, RabbitMQ displays information about message rates, connections, channels, queued messages, etc.

Configuration

The 'Restart/Stop/Start' feature uses the moogfarmd/LAM service scripts under /etc/init.d, for example, /etc/init.d/moogfarmd and /etc/init.d/logfilelamd, in combination with the Apache Tomcat 'toolrunner'.

You need Super User role permissions to configure this feature. Create a user in the 'moogsoft' group. This user must be used by the toolrunner and the services in order to start/stop services via the UI. For example:

·         /etc/init.d/moogfarmd - PROCESS_OWNER set to 'controluser'

·         $APPSERVER_HOME/webapps/toolrunner/WEB-INF/web.xml - toolrunneruser set to 'controluser' (toolrunnerpassword needs to be the password for that user)

Cisco recommends that you do not use the default 'moogsoft' user because that is a system user and does not allow you to log in using a password. Update the /etc/init.d/ service scripts to have the correct:

·         SERVICE_NAME (to make the services unique)

·         PROCESS_OWNER (must be the same user as the toolrunner user)

·         INSTANCE/CLUSTER/GROUP (unless already configured via relevant the LAM/Moogfarmd/system.conf configuration file). These need to be provided to the 'daemon' lines as command line parameters. For example --instance MY_INSTANCE --group MY_GROUP --cluster MY_CLUSTER.

Add the name of the service script into the 'service_name' field in $MOOGSOFT_HOME/config/system.conf for that Cisco Crosswork Situation Manager process. To ensure the service appears in the right Self Monitoring tab, the process_type field must be set. See the default system.conf file for examples.

If a Moogfarmd service or LAM service is run that does not match a configuration block in system.conf/'processes', then it still appears within the UI 'Self Monitoring' dialog, but it is not possible to start/stop/restart the service.

The 'toolrunner' is used to control the services (requires configuring $APPSERVER_HOME/webapps/toolrunner/WEB-INF/web.xml):

·         The 'toolrunneruser' must match the PROCESS_OWNER specified within the relevant service script. This is because only root can run services as a different user.

·         The 'toolrunnerpassword' must be the password of the 'toolrunneruser'.

·         The 'toolrunnerhost' value must match the host of the machine which contains the moogfarmd/LAM services and the PROCESS_OWNER user.

It is more likely that an existing LAM/Moogfarmd service will have been run already in upgrade scenarios. If the service is one which needs to be controlled via the UI, then the service log file and PID (if present) need to be 'chowned' to the new service script PROCESS_OWNER/toolrunner user before it will work. For example:

chown toolrunneruser /var/log/moogsoft/moogfarmd.log

See the example of a $MOOGSOFT_HOME/config/system.conf file below:

{
"group"         : "moog_farmd",
"instance"      : "",
"service_name"  : "moogfarmd",
"process_type"  : "moog_farmd",
"reserved"      : true,
"subcomponents" :
    [
        "AlertBuilder",
        "Sigaliser",
        "Default Cookbook",
        "Journaller",
        "TeamsMgr"
        #"AlertRulesEngine",
        #"SituationMgr",
        #"Notifier"
    ]
},

Obtaining Documentation and Submitting a Service Request

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation.

To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What’s New in Cisco Product Documentation RSS feed. The RSS feeds are a free service.

 

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

All printed copies and duplicate soft copies are considered un-Controlled copies and the original on-line version should be referred to for latest version.

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco website at www.cisco.com/go/offices.

Cisco Trademark

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

 

Cisco Copyright

© 2019 Cisco Systems, Inc. All rights reserved.