- Setting Up the eStreamer Perl Reference Client
- Understanding the eStreamer Perl Reference Client
- Configuring Communications for the eStreamer Reference Client
- Loading General Prerequisites for the Perl Reference Client
- Loading Prerequisites for the Perl SNMP Reference Client
- Understanding the Data Requested by a Test Script
- Modifying the Type of Data Requested by a Test Script
- Creating a Certificate for the Perl Reference Client
- Running the eStreamer Perl Reference Client
Configuring eStreamer
After you create a client application, you can connect it to the eStreamer server, start the eStreamer service, and begin exchanging data.
Note
An eStreamer server is a Management Center or managed device (version 4.9 or higher) where the eStreamer service is running.
Perform the following tasks to manage eStreamer and client interaction:
1. Enable eStreamer on the eStreamer server.
See Configuring eStreamer on the eStreamer Server for information about allowing access to the eStreamer server, adding clients, and generating authentication credentials to establish an authenticated connection.
2. If required, manually run the eStreamer service (eStreamer). You can stop, start, and view the status of the service, and use command line options to debug client-server communication.
See Managing the eStreamer Service for more information.
3. Optionally, to use the eStreamer reference client to troubleshoot a connection or data stream, set up the reference client on the computer where you plan to run your client.
Configuring eStreamer on the eStreamer Server
Before the Management Center or managed device you want to use as an eStreamer server can begin streaming events to a client application, you must configure the eStreamer server to send events to clients, provide information about the client, and generate a set of authentication credentials to use when establishing communication. You can perform all of these tasks from the Management Center or managed device user interface.
See the following sections for more information:
Configuring eStreamer Event Types
You can control which types of events the eStreamer server is able to transmit to client applications that request them.
Available event types on a managed device or a Management Center include:
Available event types on a Management Center include:
- Discovery events (this also enables connection events)
- Correlation and allow list events
- Impact flag alerts
- User activity events
- Malware events
- File events
Note that the primary and secondary in a stacked 3D9900 pair report intrusion events to the Management Center as if they were separate managed devices. If you configure communication with an eStreamer client on the primary in a 3D9900 stack, you also need to configure the client on the secondary; the client configuration is not replicated. Similarly, when you delete the client, delete it in both places. If you configure an eStreamer client for a Management Center managing 3D9900s in a stack configuration, note that the Management Center reports all events received from both managed devices, even if the same event is reported by both.
If you configure an eStreamer client on a Management Center in a high availability
configuration, the client configuration is not replicated from the primary Management Center to the secondary Management Center.
To configure the types of events captured by eStreamer:
Step 1 Select System > Integration > eStreamer.
The eStreamer page appears with the eStreamer Event Configuration menu.
Step 3 Select the check boxes next to the types of events you want eStreamer to capture and forward to requesting clients. Note that if a check box is currently unchecked, that data is not being captured. Unchecking a check box does not delete data that has already been captured.
You can select any or all of the following on a Management Center or managed device:
- Intrusion Events to transmit intrusion events generated by managed devices.
- Intrusion Event Packet Data to transmit packets associated with intrusion events.
- Intrusion Event Extra Data to transmit additional data associated with intrusion events, such as the URI associated with the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
You can also select any or all of the following on a Management Center:
- Discovery Events to transmit host discovery events
- Correlation Events to transmit correlation and allow list events.
- Impact Flag Alerts to transmit impact alerts generated by the Management Center.
- User Activity Events to transmit user events.
- Intrusion Event Extra Data to transmit additional data for intrusion events, such as the URI associated with the originating IP address of a client connecting to a web server through an HTTP proxy or load balancer.
Note Note that this controls which events the eStreamer server can transmit. Your client application must still specifically request the types of events you want it to receive. For more information, see Request Flags.
Your settings are saved and the events you selected will be forwarded to eStreamer clients when requested.
Adding Authentication for eStreamer Clients
Before eStreamer can send events to a client, you must add the client to the eStreamer server’s peers database. You must also copy the authentication certificate generated by the eStreamer server to the client.
Step 1 Select System > Integration > eStreamer.
The Create Client page appears.
Step 3 In the Hostname field, enter the host name or IP address of the host running the eStreamer client.
Note If you use a host name, the host input server must be able to resolve the host to an IP address. If you have not configured DNS resolution, you should configure it first or use an IP address.
Step 4 If you want to encrypt the certificate file, enter a password in the Password field.
The eStreamer server allows the client computer to access port 8302 on the Management Center and creates an authentication certificate to use during client-server authentication. The eStreamer Client page re-appears, with the new client listed under eStreamer Clients.
Step 6 Click the download icon (
) next to the certificate file.
Step 7 Save the certificate file to the directory used by your client computer for SSL authentication.
The client can now connect to the Management Center.
Tip To revoke access for a client, click the delete icon (
) next to the host you want to remove. Note that you do not need to restart the host input service on the Management Center; access is revoked immediately.
Managing the eStreamer Service
You can manage the eStreamer service from the user interface. However, you can also use the command line to start and stop the service. The following sections describe eStreamer command line options:
- Starting and Stopping the eStreamer Service describes how to start and stop the eStreamer service.
- eStreamer Service Options describes the command line options available for the eStreamer service and how to use them.
Starting and Stopping the eStreamer Service
You can manage the eStreamer service using the manage_estreamer.pl script, which allows you to start, stop, reload, and restart the service.
Tip You can also add command line options to the eStreamer initialization script. See eStreamer Service Options for more information.
The following table describes the options in the manage_estreamer.pl script you can use on the Management Center or managed device.
|
|
|
|
|---|---|---|
eStreamer Service Options
eStreamer provides many service options that allow you to troubleshoot the service. You can use the options described in the following table with the eStreamer service.
Use the above options by first stopping the eStreamer service, then running it with the options you want, and finally restarting the service. For example, you can follow the instructions provided in Running the eStreamer Service in Debug Mode to debug eStreamer functionality.
Running the eStreamer Service in Debug Mode
You can run the eStreamer service in debug mode to view each status message the service generates on your terminal screen. Use the following procedure to do debugging.
To run the eStreamer service in debug mode:
Step 1 Log into the Management Center or managed device using SSH.
Step 2 Use manage_estreamer.pl and select option 2 to stop the eStreamer service.
Step 3 Use ./usr/local/sf/bin/sfestreamer --nodaemon --debug to restart the eStreamer service in debug mode.
Status messages for the service appear on the terminal screen.
Step 4 When you are finished debugging, restart the service in normal mode using manage_estreamer.pl and selecting option 4.
Configuring the eStreamer Reference Client
The reference client provided with the eStreamer SDK is a set of sample client scripts and Perl modules included to illustrate how the eStreamer API can be used. You can run them to familiarize yourself with eStreamer output, or you can use them to debug problems with installations of your custom-built client.
For more information on setting up the reference client, see the following sections:
Setting Up the eStreamer Perl Reference Client
To use the eStreamer Perl reference client, you must first configure the sample scripts to fit your environment and requirements.
For more information, see the following sections:
- Understanding the eStreamer Perl Reference Client
- Configuring Communications for the eStreamer Reference Client
- Loading General Prerequisites for the Perl Reference Client
- Loading Prerequisites for the Perl SNMP Reference Client
- Understanding the Data Requested by a Test Script
- Modifying the Type of Data Requested by a Test Script
- Creating a Certificate for the Perl Reference Client
Understanding the eStreamer Perl Reference Client
You can download the eStreamerSDK.zip package, which contains the eStreamer Perl reference client, from the Cisco support site. The following files are included in the eStreamerSDK.zip package:
This MIB file is used by the snmp.pm file to set up traps for SNMP.
This Perl module contains definitions of discovery message record blocks.
This Perl module contains the functions called by the Perl clients.
This Perl module parses the client certificate and allows the client to connect to the eStreamer server.
This Perl module contains definitions of discovery data blocks.
You can use this Perl script to test an intrusion event request over an SSL connection.
This Perl module prints intrusion events to a comma-separated value (CSV) format.
This Perl module prints events to a human-readable format.
This Perl module sends events to the specified SNMP server.
Configuring Communications for the eStreamer Reference Client
The reference client uses the Secure Sockets Layer (SSL) for data communication. You must install OpenSSL on the computer you plan to use as a client and configure it appropriately for your environment.
Note For initial installations on Linux operating systems, you must install the libssl-dev component as part of this download.
Step 1 Download OpenSSL from http://openssl.org/source/.
Step 2 Unpack the source to /usr/local/src.
Step 3 Configure the source by running the Configure script.
Step 4 Make and install the compiled source.
Loading General Prerequisites for the Perl Reference Client
Before you can run the eStreamer Perl reference client, you must install the IO::Socket::SSL Perl module on the client computer. You can install the module manually or use cpan to do so.
Note If the Net::SSLeay module is not installed on the client computer, install that module as well. Net::SSLeay is required for communication with OpenSSL.
You also need to install and configure OpenSSL to support an SSL connection to the eStreamer server. For more information, see Configuring Communications for the eStreamer Reference Client.
Loading Prerequisites for the Perl SNMP Reference Client
Before you can run the eStreamer SNMP module of the Perl reference client, you must install the latest net-snmp Perl modules available for the client operating system on the client computer.
Downloading and Unpacking the Perl Reference Client
You can download the EventStreamerSDK.zip file that contains the eStreamer Perl reference client the Cisco support site.
Unpack the zip file to a computer running the Linux operating system, where you plan to run the client.
Understanding the Data Requested by a Test Script
By default, when you use the ssl_test -o setting in the reference client, you request data as indicated in the following table.
|
|
|
|
|
|---|---|---|---|
Host data (see Host Data and Multiple Host Data Message Format) |
|||
|
Streaming event information for the specified domain (see Domain Streaming Request Message Format) |
||
Event stream request, message type 2, with bits 2 and 20-24 set to |
Event data (see Event Stream Request Message Format, Correlation Policy Record, Correlation Rule Record, Metadata for Discovery Events, Host Discovery Structures by Event Type, and Host IOC Set Messages) eStreamer transmits type 1 intrusion events because bit 2 is set on the event stream request. |
||
Event stream request, message type 2, with bits 0 and 23 set to |
Packet data (see Event Data Message Format and Packet Record 4.8.0.2+) eStreamer transmits only packet data because bit 0 is set on the event stream request. |
||
Event stream request, message type 2, with bits 2 and 23 set to |
Intrusion event data (see Event Data Message Format and Intrusion Event Record 6.0+) eStreamer transmits type 1 intrusion events because bit 2 is set on the event stream request. |
||
Event stream request, message type 2, with bits 2, 20, and 23 set to |
Intrusion event data (see Event Data Message Format and Intrusion Event Record 6.0+) eStreamer transmits type 1 intrusion events because bit 2 is set on the event stream request. |
||
Event stream request, message type 2, with bits 2, 20, and 23 set to |
Intrusion event data (see Event Data Message Format and Intrusion Event Record 6.0+) eStreamer transmits type 1 intrusion events because bit 2 is set on the event stream request. |
Modifying the Type of Data Requested by a Test Script
The SFStreamer.pm Perl module defines several request flag variables that you can use in the sample scripts to request data. The following table indicates what request flag variable to call to set each request flag in an event stream request message. If you want to request different data using one of the output modules, you can edit the $FLAG settings in the module.
For more information on the request flags, the data they request, and the product versions corresponding to each flag, see Request Flags.
In all event types, prior to version 5.x, the reference client labels
detection engine ID fields as
sensor ID.
Creating a Certificate for the Perl Reference Client
Before you can use the Perl reference client, you need to create a certificate on the Management Center or managed device for the computer where you want to run the client. You then download the certificate file to the client computer and use it to create a certificate ( server.crt) and RSA key file ( server.key).
To create a certificate for the Perl Reference Client:
Step 1 Select System > Integration > eStreamer.
The Create Client page appears.
Step 3 In the Hostname field, enter the host name or IP address of the host running the eStreamer client.
Note If you use a host name, the host input server must be able to resolve the host to an IP address. If you have not configured DNS resolution, you should configure it first or use an IP address.
Step 4 If you want to encrypt the certificate file, enter a password in the Password field.
The eStreamer server allows the client computer to access port 8302 on the Management Center and creates an authentication certificate to use during client-server authentication. The eStreamer Client page re-appears, with the new client listed under eStreamer Clients.
Step 6 Click the download icon () next to the certificate file.
Step 7 Save the certificate file to the directory used by your client computer for SSL authentication.
The client can now connect to the Management Center.
Tip To revoke access for a client, click the delete icon () next to the host you want to remove. Note that you do not need to restart the host input service on the Management Center; access is revoked immediately.
Running the eStreamer Perl Reference Client
The eStreamer Perl reference client scripts are designed for use on a 64-bit operating system with the Linux kernel but should work on any POSIX-based 64-bit operating system, as long as the client machine meets the prerequisites defined in Setting Up the eStreamer Perl Reference Client.
Testing a Client Connection over SSL Using a Host Request
You can use the ssl_test.pl script to test the connection between the eStreamer server and the eStreamer client. The ssl_test.pl script handles any record type and prints it to STDOUT or to an output plugin you specify. When you use the -h option without an output option, it streams host data for the specified hosts to your terminal.
Note You cannot use this script to stream packet data without directing it to an output plugin because printing raw packet data to STDOUT interferes with your terminal.
Use the following syntax to use the ssl_test.pl script to send host data to the standard output:
-h HostIPAddresses
For example, to test receipt of host data for the hosts in the 10.0.0.0/8 subnet over a connection to an eStreamer server with an IP address of 10.10.0.4:
Capturing a PCAP Using the Reference Client
You can use the reference client to capture streamed packet data in a PCAP file to see the structure of the data the client receives. Note that you must use -f to specify a target file when you use the -o pcap output option.
Use the following syntax to capture streamed packet data in a PCAP file using the ssl_test.pl script:
For example, to create a PCAP file named test.pcap using events streamed from an eStreamer server with an IP address of 10.10.0.4:
Capturing CSV Records Using the Reference Client
You can also use the reference client to capture streamed intrusion event data in a CSV file to see the structure of the data the client receives.
Use the following syntax to run the streamer_csv.pl script:
./ssl_test.pl eStreamerServerIPAddress -o csv -f ResultingCSVFile
For example, to create a CSV file named test.csv using events streamed from an eStreamer server with an IP address of 10.10.0.4:
Sending Records to an SNMP Server Using the Reference Client
You can also use the reference client to stream intrusion event data to an SNMP server. Use the -f option to indicate the name of the SNMP trap server that should receive events. Note that this output method requires a binary named snmptrapd in the path and therefore only works on UNIX-like systems.
Use the following syntax to send intrusion events to an SNMP server:
-f SNMPServerName
For example, to send events to an SNMP server at 10.10.0.3 using events streamed from an eStreamer server with an IP address of 10.10.0.4:
Logging Events to the Syslog Using the Reference Client
You can also use the reference client to stream intrusion events to the local syslog server on the client.
Use the following syntax to send events to the syslog:
For example, to log events streamed from an eStreamer server with an IP address of 10.10.0.4:
Connecting to an IPv6 Address
You can use the reference client to connect to a Management Center with an IPv6 address through the primary management interface. You must have the Socket6 and IO::Socket::INET6 Perl modules installed on the client machine and use the -ipv6 option or the shortened form -i.
Use the following syntax to specify an IPv6 address using the ssl_test.pl script:
For example, to connect to a Management Center with the IPv6 address 2001:470:e09c:20:7c1e:5248:1bf7:2ea0 use the following:
Feedback