Troubleshoot Issues Between FireSIGHT System and eStreamer Client (SIEM)

 

Introduction

The Event Streamer (eStreamer) allows you to stream several kinds of event data from a FireSIGHT System to a custom-developed client application. After you create a client application, you can connect it to an eStreamer server (For example, a FireSIGHT Management Center), start the eStreamer service, and begin exchanging data. eStreamer integration requires custom programming, but allows you to request specific data from an appliance. This document describes how an eStreamer client communicates and how to troubleshoot an issue with a client.

 

Communication Method Between eStreamer Client and Server

There are four major stages of communication that occur between a client and the eStreamer service:

Step 1: Client Establishes a Connection with eStreamer Server

First, a client establishes a connection with eStreamer server and the connection is authenticated by both parties. Before a client can request data from eStreamer, the client must initiate an SSL-enabled TCP connection with the eStreamer service. When the client initiates the connection, the eStreamer server responds, initiating an SSL handshake with the client. As part of the SSL handshake, the eStreamer server requests the client's authentication certificate, and verifies that the certificate is valid.
 
After the SSL session is established, the eStreamer server performs an additional post-connection verification of the certificate. After the post-connection verification is finished, the eStreamer server awaits a data request from the client.
 

Step 2: Client Requests Data from the eStreamer Service

In this step, client requests data from the eStreamer service and specifies the types of data to be streamed. A single event request message can specify any combination of available event data, including event metadata. A single host profile request can specify a single host or multiple hosts. Two request modes are available for requesting event data:

• Event Stream Request: The client submits a message containing request flags that specify the requested event types and version of each type, and the eStreamer server responds by streaming the requested data.
• Extended Request: The client submits a request with the same message format as for Event Stream requests but sets a flag for an extended request. This initiates a message interaction between client and eStreamer server through which the client requests additional information and version combinations not available via Event Stream requests.

 

Step 3: eStreamer Establishes the Requested Data Stream

In this stage, eStreamer establishes the requested data stream to the client. During periods of inactivity, eStreamer sends periodic null messages to the client to keep the connection open. If it receives an error message from the client or an intermediate host, it closes the connection.

 

Step 4: The Connection Terminates

The eStreamer server can also close a client connection for the following reasons:

• Any time sending a message results in an error. This includes both event data messages and the null keep-alive message eStreamer sends during periods of inactivity.
• An error occurs while processing a client request.
• Client authentication fails (no error message is sent).
• eStreamer service is shutting down (no error message is sent).

Client Shows No Event

If you do not see any events on your eStreamer client application, please follow the steps below to troubleshoot this issue:

Step 1: Verify the Configuration

Step 2: Verify the Certificate

Make sure that the required certificates are added. Before eStreamer can send eStreamer events to a client, the client must be added to the eStreamer server's peers database using the eStreamer configuration page. The authentication certificate generated by the eStreamer server must also be copied to the client.

Step 3: Check Error Messages

Identify any obvious eStreamer related errors in /var/log/messages by using the following command:
 

admin@FireSIGHT:~$ grep -i estreamer /var/log/messages | grep -i error

Step 4: Verify the Connection

Verify that the server is accepting incoming connections.
 

admin@FireSIGHT:~$ netstat -an | grep 8302

The output should look like below. If not, then the service may not be running.
 

tcp 0 0 <local_ip>:8302 0.0.0.0:* LISTEN

Step 5: Check the Status of the Process

To verify if there is an sfestreamer process running, please use the following command:
 

admin@FireSIGHT:~$ pstree -a | grep -i sfestreamer

Client Shows Duplicate Events

Handle Duplicate Events Displayed in a Client

The eStreamer server does not keep a history of the events it sends, so the client application must check for duplicate events. Duplicate events can occur for a variety of reasons.  For example, when starting a new streaming session, the time specified by the client as the starting point for the new session can have multiple messages, some of which may have been sent in the previous session and some of which were not. eStreamer sends all messages that meet the specified request criteria. EStreamer client applications should be designed to detect and de-duplicate any resulting duplicates.

Manage Duplicate Requests for Data

If you request multiple versions of the same data, either by multiple flags or multiple extended requests, the highest version is used. For example, if eStreamer receives flag requests for discovery events version 1 and 6 and an extended request for version 3, it sends version 6.

Client Shows Incorrect Snort Rule ID (SID)

This usually happens due to an SID conflict when a rule is imported into the system, the SID is re-mapped internally.

To use the SID you entered, rather than the re-mapped SID, you have to enable extended header. Bit 23 requests extended event headers. If this field is set to 0, events are sent with a standard event header that only includes the record type and record length.

Figure: The diagram illustrates the message format used to request data from eStreamer. Fields specific to the request message format are highlighted in gray.

Figure: The diagram illustrate the format of rule message information for an event that is transmitted within a Rule Message record. It shows the RuleID (which you are using now) and the Rendered Signature ID (which is the number you expect).

Tip: In order to find the detail description of each bit and message, read the eStreamer Integration Guide.

Collect and Analyze Additional Troubleshoot Data

Test Using the ssl_test.pl Script

Utilize the ssl_test.pl script provided in the Event Streamer Software Development Kit (SDK) to identify the problem. The SDK is available in a zip file on the support site. Instructions for the script is available in the README.txt, which is included in that zip file.

Capture Packet (PCAP)

Capture packets on the management interface of the eStreamer server and analyze it. Verify that the traffic is not blocked or denied somewhere in your network.

Generate Troubleshoot File

If you completed the above troubleshooting steps, and you are still unable to determine the problem, please generate a troubleshoot file from your FireSIGHT Management Center. Provide all of the additional troubleshoot data to Cisco Technical Support for further analysis.