Bulk Statistics

This chapter provides configuration information for:

Feature Summary and Revision History

Summary Data

Applicable Product(s) or Functional Area

All

Applicable Platform(s)

ASR 5500

VPC-DI

VPC-SI

Feature Default

Disabled - Configuration Required

Related Changes in This Release

Not Applicable

Related Documentation

  • ASR 5500 System Administration Guide

  • Command Line Interface Reference

  • VPC-DI System Administration Guide

  • VPC-SI System Administration Guide

Revision History


Note

Revision history details are not provided for features introduced before releases 21.2 and N5.5.

Revision Details

Release

New functionality was added to replace or supplement the configured bulkstats schema with the option of preserving bulkstats configuration parameters.

New functionality was added to collect bulkstats samples in the SSD. Refer to the Collecting Bulk Statistics Samples in SSD section for more information.

The bulkstat Global Configuration Mode command added the config [ schema | supplement ] keywords to enable this functionality. Refer to the Configuring a Separate Bulkstats Config File section for more information.

show configuration bulkstats brief command output was expanded to include all bulkstats configuration details except for schema.

21.3

First introduced.

Pre 21.2

Configuring Communication with the Collection Server

Two configuration methods are available for defining how bulk statistics are collected and managed. A "standard" configuration allows the system to automatically assign a number to the bulk statistics file. Optionally, a number can be specified by an administrator in the optional configuration method. Command details and descriptions of keywords and variables for commands in this chapter are located in the Bulk Statistics Configuration Mode Commands and Bulk Statistics File Configuration Mode Commands chapters in the Command Line Interface Reference.

Configuring Standard Settings

The configuration example in this section defines basic operation of the bulk statistics feature. Use the following example configuration to set up the system to communicate with the statistic collection server: 

configure 
   bulkstats mode 
      schema name format format_string 
      sample-interval time_interval 
      transfer-interval xmit_time_interval 
      limit mem_limit 
      exit 
    bulkstats collection 
    end

Configuring Optional Settings

This section describes optional commands that can be used within the Bulk Statistics Configuration mode. Specifically, you can configure bulk statistic "files" under which to group the bulk statistics. "Files" are used to group bulk statistic schema, delivery options, and receiver configuration. Because multiple "files" can be configured, this functionality provides greater flexibility because it allows you to configure different schemas to go to different receivers. 

configure 
  bulkstats mode 
    file number 
      receiver ip_address { primary | secondary }  
       [ mechanism { { { ftp | sftp } login user_name  
       [ encrypted ] password pwd } | tftp } } ] } 
      receiver mode { redundant | secondary-on-failure } 
      remotefile format naming_convention [ both-receivers | primary-receiver | secondary-receiver ] 
      header format header_format 
      footer format footer_format 
      exit 
    schema_type schema format format_string 
    sample-interval time_interval 
    transfer-interval xmit_time_interval 
    limit mem_limit 
    exit 
  bulkstats collection 
  end

Important

In release 20.0 and higher Trusted StarOS builds, FTP is not supported. SFTP is the recommended transfer protocol.

Configuring Bulk Statistic Schemas

In each configuration example described in Configuring Standard Settings and Configuring Optional Settings, the following is the primary command used to configure the type of schema and the statistics collected: 

configure 
  bulkstats mode 
    schema_type schema format format_string

Refer to the Bulk Statistics Configuration Mode Commands and Bulk Statistics File Configuration Mode Commands chapters in the Command Line Interface Reference for more information regarding supported schemas, available statistics, and proper command syntax.

Configuring a Separate Bulkstats Config File

You can configure a separate destination file for storing the bulk statistics sub-mode configuration. Run the show configuration bulkstats command to confirm the configuration.

The bulkstats configuration file stores the configuration that was previously stored in the system configuration file under the bulk statistics sub-mode.

The Global Configuration mode bulkstats config command creates the separate configuration file in the system configuration. 

configure 
   [no] bulkstats config [ schema | supplement ] url 
   end

The optional schema keyword allows you to replace only the schema using the file provided and preserve the server configuration. The optional supplement keyword allows you to supplement the running bulkstats configuration with the contents of the configuration file provided. These keywords only work on existing files.

url specifies the location of the bulkstats configuration file. If the destination file already exists, it is replaced with the new file (except when the schema or supplement keywords are used). The new file will only be created if you save the configuration after completing changes. The Exec mode show configuration bulkstats command displays the URL for the bulkstats configuration mode destination file if it has been configured.

You can copy the bulkstats configuration file from the url, edit it and copy it back to /flash. Changes can be applied by using the no form of the bulkstats config command followed by reconfiguring the bulkstats config command.

When the bulkstats config command is enabled, StarOS removes the existing bulk statistics sub-mode configuration from the system configuration file. You must save the system configuration to retain the configuration change.

If no bulkstats config is used to disable the new destination file after it has been enabled. StarOS does not remove the file. You must save the system configuration to retain the configuration change.


Important

After completing changes to the bulk statistics configuration, you must save the system configuration to save the changes. If the bulkstats config command is enabled, the bulkstats configuration file will be updated.

Using show bulkstats Commands

There are several Exec mode show bulkstats commands that display information about defined parameters.

  • show bulkstats data – displays criteria contained in the statistics gathering scheme for up to four files. See Viewing Collected Bulk Statistics Data.

  • show bulkstats schemas – displays the scheme used to gather statistics including collection and transmission statistics. See Verifying Your Configuration.

  • show bulkstats variables – displays available bulkstat variables (%variable%) by schema type that can be incorporated into a schema format.

In addition, show configuration bulkstats brief displays the bulkstats configuration at a global scope, as well as all server configuration. It does not display the schema configuration.

Verifying Your Configuration

After configuring support for bulk statistics on the system, you can check your settings prior to saving them.

Follow the instructions in this section to verify your bulk statistic settings. These instructions assume that you are at the root prompt for the Exec mode.

Check your collection server communication and schema settings by entering the following Exec mode command: 
show bulkstats schemas

The following is an example command output: 

Bulk Statistics Server Configuration: 
    Server State:                            Enabled  
    File Limit:                              6000 KB  
    Sample Interval:                         15 minutes (0D 0H 15M) 
    Transfer Interval:                       480 minutes (0D 0H 15M) 
    Collection Mode:                         Cumulative  
    Receiver Mode:                           Secondary-on-failure  
    Local File Storage:                      None 
Bulk Statistics Server Statistics: 
    Records awaiting transmission:  114 
    Bytes awaiting transmission:    8092 
    Total records collected:        59926 
    Total bytes collected:          4190178  
    Total records transmitted:      59812 
    Total bytes transmitted:        4188512  
    Total records discarded:        0 
    Total bytes discarded:          0 
    Last collection time required:  2 second(s) 
    Last transfer time required:    0 second(s) 
    Last successful transfer:       Wednesday December 7 12:14:30 EDT 2011 
    Last successful tx recs:        190 
    Last successful tx bytes:       13507 
    Last attempted transfer:        Wednesday December 7 12:14:30 EDT 2011 
File 1 
    Remote File Format:     /users/ems/server/data/chicago/bulkstat%date%%time%.txt 
    File Header:            "CHI_test %time%" 
    File Footer:            "" 
Bulkstats Receivers: 
    Primary: 192.168.0.100 using FTP with username administrator 
    Records awaiting transmission:    0 
    Bytes awaiting transmission:      0 
    Total records collected:          0 
    Total bytes collected:            0  
    Total records transmitted:        0  
    Total bytes transmitted:          0  
    Total records discarded:          0  
    Total bytes discarded:            0  
    Last transfer time required:      0 second(s)  
    No successful data transfers 
    No attempted data transfe 
 
File 2 not configured 
 
File 3 not configured 
 
File 4 not configured

Saving Your Configuration

Save the configuration as described in the Verifying and Saving Your Configuration chapter.

Viewing Collected Bulk Statistics Data

The system provides a mechanism for viewing data that has been collected but has not been transferred. This data is referred to as "pending data".

View pending bulk statistics data per schema by entering the following Exec mode command: 

show bulkstats data

The above command also shows the statistics of remote files, if configured as described in Configuring Optional Settings.

Collecting Bulk Statistics Samples in SSD

The output of the show support details (SSD) command is collected and provided to the Technical Assistance Center (TAC) for troubleshooting purposes. Bulkstats information in the SSD enables customers to provide readily available bulk statistics records for analysis, and faster resolution to issues.

By default, the system does not include bulkstats samples in the SSD. This functionality can be enabled using the bulkstats ssd-samples command under the Global Configuration Mode. 

config 
    bulkstats ssd-samples { 1 | 2 } 
    end

If previously enabled, the { no | default } bulkstats ssd-samples command disables collection of bulkstats samples in the SSD archive. Each bulkstats sample contains bulkstats from one transfer history. Currently, a maximum of two bulkstats sample can be included in the SSD archive. The sample files are collected in a temporary storage location at /var/tmp/bulkstats under the file name ssd_bulkstats_file<bulkstat_file_number>_sample1.txt (and ssd_bulkstats_file<bulkstat_file_number>_sample2.txt).

While the SSD archive is being created in the temporary storage, the bulk statistics samples might occupy a large amount of the storage space. As a result, the SSD archive creation might fail. During such scenarios, the no-bulkstats keyword in the show support details command can exculde the bulkstats samples from the SSD archive.

Manually Gathering and Transferring Bulk Statistics

There may be times where it is necessary to gather and transfer bulk statistics outside of the scheduled intervals. The system provides commands that allow you to manually initiate the gathering and transferring of bulk statistics.

To manually initiate the gathering of bulk statistics outside of the configured sampling interval, enter the following Exec mode command: 

bulkstats force gather

To manually initiate the transferring of bulk statistics prior to reaching the of the maximum configured storage limit, enter the following Exec mode command: 

bulkstats force transfer

Clearing Bulk Statistics Counters and Information

It may be necessary to periodically clear counters pertaining to bulk statistics in order to gather new information or to remove bulk statistics information that has already been collected. The following Exec mode command can be used to perform either of these functions: 

clear bulkstats { counters | data }

The clear bulkstats data command clears any accumulated data that has not been transferred. This includes any "completed" files that have not been successfully transferred.

Bulkstats Schema Nomenclature

This section describes the nomenclature associated with configuring and viewing bulkstats.

Statistic Types

The following statistic types are defined in the Statistics and Counters Reference user document (published prior to Release 20.0) and displayed in the output of the Exec mode show bulkstats variables command"
  • Counter: A counter records incremental data cumulatively and rolls over when the counter limit is reached.

    • All counter statistics are cumulative and reset only by one of the following methods: roll-over when the limit is reached, after a system restart, or after a clear command is performed.

    • The limit depends upon the data type.

  • Gauge: A gauge statistic indicates a single value; a snapshot representation of a single point in time within a defined time frame. The gauge changes to a new value with each snapshot though a value may repeat from one period to the next. The limit depends upon the data type.

  • Information: This type of statistic provides information, often intended to differentiate sets of statistics; for example, a VPN name or IP address. The type of information provided depends upon the data type.

The following statistic types are included in the Statistics and Counters Reference spreadsheet published after release 20.0 to replace the original user document:
  • Incremental: An incremental data type records incremental data cumulatively and rolls over when the counter limit is reached.

    • All incremental statistics are cumulative and reset only by one of the following methods: roll-over when the limit is reached, after a system restart, or after a clear command is performed.

    • The limit depends upon the data type.

  • Gauge: A gauge statistic indicates a single value; a snapshot representation of a single point in time within a defined time frame. The gauge changes to a new value with each snapshot though a value may repeat from one period to the next. The limit depends upon the data type.

  • Primary-key: This type of statistic provides information, often intended to differentiate sets of statistics; for example, a VPN name or IP address. The type of information provided depends upon the data type.

Data Types

The data type defines the format of the data for the value provided by the statistic. The following data types appear in the Statistics and Counters Reference and the output of the Exec mode show bulkstats variables command:

  • Int32: A 32-bit integer; the roll-over to zero limit is 4,294,967,295.

  • Int64: A 64-bit integer; the roll-over to zero limit is 18,446,744,073,709,551,615.

  • Float: A numeric value that includes decimal points; for example, 1.345.

  • String: A series of ASCII alphanumeric characters in a single grouping, usually pre-configured.

Key Variables

Every schema has some variables which are typically referred to as "key variables". These key variables provide index markers to identify to which object the statistics apply. For example, in the card schema the card number (variable %card%) uniquely identifies a card. For an HA service, the keys would be "%vpnname%" plus "%servname%', as the combination uniquely identifies an HA service. So, in a given measurement interval, one row of statistics will be generated per unique key.

There are also a number of common variables shared across schema that identify time, date, place, etc. These common variables are identified in the table below.

Table 1. Common Variables Across Schema
Variables Description Statistic Type Data Type

version-no

Contains complete version information that can be used in the header of the bulkstats file

Information

String

uptime

The total uptime (in seconds) of the system that created the file.

Information

Int32

host

The system hostname that created the file.

Information

String

ipaddr

The default management (local context) IP address in dPv4 dotted-decimal format. An empty string is inserted if no address is available.

Information

String

date

The UTC date that the collection file was created in YYYYMMDD format where YYYY represents the year, MM represents the month and DD represents the day.

Information

String

date3

The UTC date that the collection file was created in YYMMDD format where YY represents the year, MM represents the month and DD represents the day.

Information

String

time

The UTC time that the collection file was created in HHMMSS format where HH represents the hours, MM represents the minutes, and SS represents the seconds.

Information

String

time2

The UTC time that the collection file was created in HH:MM:SS format where HH represents the hours, MM represents the minutes, and SS represents the seconds.

Information

String

time3

The UTC time that the collection file was created in HH:MM format where HH represents the hours, MM represents the minutes.

Information

String

epochtime

The number of seconds since Jan 1, 1970, 00:00:00 GMT.

Information

In32

schemas

Lists all bulkstat schemas available on this platform.

Information

String

schemas-delta

Lists all bulkstats schemas that have changed the schema list was last output.

Information

String

localdate

The date (adjusted for the local timezone) that the collection file was created in YYYYMMDD format where YYYY represents the year, MM represents the month and DD represents the day.

Information

String

localdate3

The date that the collection file was created in YYYYMMDD format where YYYY represents the year, MM represents the month and DD represents the day. The date displays in local time, not UTC.

Information

String

localtime

The time (adjusted for the local timezone) that the collection file was created in HHMMSS format where HH represents the hours, MM represents the minutes, and SS represents the seconds.

Information

String

localtime2

The time (adjusted for the local timezone) that the collection file was created in HH:MM:SS format where HH represents the hours, MM represents the minutes, and SS represents the seconds.

Information

String

localtime3

The time that the collection file was created in HH:MM:SS format where HH represents the hours, MM represents the minutes, and SS represents the seconds. The time displays in local time, not UTC.

Information

String

localtz

The local timezone set for this platform.

Information

String

localtzoffset

The offset from UTC/GMT for the local timezone. Format = "+" or "-" HHMM.

Information

String

swbuild

The build number of the StarOS version.

Information

String

Bulk Statistics Event Log Messages

The stat logging facility captures several events that can be useful for diagnosing errors that could occur with either the creation or writing of a bulk statistic data set to a particular location.

The following table displays information pertaining to these events.
Table 2. Logging Events Pertaining to Bulk Statistics
Event Event ID Severity Additional Information
Local File Open Error 31002 Warning "Unable to open local file filename for storing bulkstats data"
Receiver Open Error 31018 Warning "Unable to open url filename for storing bulkstats data"
Receiver Write Error 31019 Warning "Unable to write to url filename while storing bulkstats data"
Receiver Close Error 31020 Warning "Unable to close url filename while storing bulkstats data"