The CPS vDRA supports exposing of Prometheus API queries on OAM network using HAProxy. vDRA allows operators to fetch necessary statistics from the system through the Prometheus API and further analyze in a single consolidated view. The following functions are supported:

• vDRA data gets pulled from Prometheus API and loaded directly into system for visualization. This includes data from the following three data stores:

  • Prometheus Hi-Res

  • Prometheus Trending

  • Prometheus Planning

• The Maximum TPS that is required for enabling these queries at required intervals are:

  • TPS: The TPS value is 160K.

  • Intervals: one minute and five minutes

• GET queries: GET /api/v1/query

• TLS or HTTPS-based authentication

Following statistics are collected from target sources and are available through Prometheus APIs:

• Host Node Exporter for Host VM statistics

• Mongo DB Exporter for Database statistics

• Application Statistics

• Scrape interval takes fewer samples and keep data for a longer interval. This is performed through CLI configurations:

  • To change the scrape interval timings for Hi-Res container, Planning container, Trending container. For example, you can change interval timing for Hi-res to 30s, Planning to 125s, Trending to 60s.

  • Through default Scrape interval timings. For example, default scrape interval timing for Hi-res is 5s, Planning is 120s Trending is 20s , and Maximum scrape interval timing for Hi-res is 30s, Planning is 125s and Trending is 60s.

  • Observes disc utilization improvements after changing the scrape interval.

  • Verifies the hi-res, trending and planning data disc pattern with default for Hi-res 5s, Planning 120s Trending 20s, Maximum scrape interval timing for Hi-res 30s, Planning 125s Trending 60s.

    An example for reduction in log size.

    #### Disk Size with Default Scrape Interval with 20KTPS Traffic ####
    Hi-res [5s], Planning[120s], Trending [20s]
    
    du -sh /stats
    2 GB -> prometheus-hi-res
    9 GB -> prometheus-planning
    4.5 GB -> prometheus-trending
    
    **** Disk Size with Maximum Scrape Interval with 20KTPS Traffic ****
    Hi-res [30s], Planning[125s], Trending [60s]
    
    du -sh /stats
    419 MB  ->  prometheus-hi-res---70% disk got reduced
    4.7 GB ->  prometheus-planning---50% disk got reduced
    2.3 GB  ->  prometheus-trending----50% disk got reduced
    

Important: Scrape interval configuration is important because it determines the number of data points per minute (DPM) scraped in your active series. However, using scrape intervals of more than 2 minutes in Prometheus is not recommended. Due to the default staleness period of 5 minutes, a scrape interval of 2 minutes allows for one failed scrape without the metrics being treated as stale.

For more information see the CLI Commands chapter in the vDRA Operations Guide.

To expose the Prometheus API metrics from Prometheus clusters, you should configure the Prometheus Federate Server. You can configure the Prometheus Clusters of multiple vDRA sites in one Prometheus external Server.

Here, the Prometheus server which contains service-level metrics pull in the cluster resource usage metrics about its specific service from the cluster Prometheus. Hence, both the sets of metrics can be used within that server.

Install the Prometheus packages on a remote server and verify if the remote server is reachable to the DRA Environment.

This chapter provides information about the CPS implementation of Grafana. For more information about Grafana, or access the general Grafana documentation, refer to: http://docs.grafana.org.

The CPS implementation uses the Prometheus data source and does not use Graphite for queries. This requires the definition of queries to use the Prometheus query format as defined in https://prometheus.io/docs/querying/basics/.

Note	After changing respective KPI panel's width to 24 (which is maximum), you can get all the spikes captured for 6 hours duration. So, if tou need to analyse longevity report for 12 hours or more, you can grep data by grouping in 6 hours interval.

Note

If the control VM that hosts Grafana goes down, then the Prometheus data also not available during that downtime after the same control VM (hosting Grafana) is back. This results in some missing data. As a workaround, you can add the Prometheus datasource of other control VM in Grafana UI that was up during that downtime and view the missing statistics.

Note	The top command output must not be compared with the Grafana CPU statistics panel display.

Note	In Grafana, admin users can invite new users by email or a link. However, this is not supported in CPS vDRA.

Perform the following to add a new Grafana:

1. Enter config mode

   scheduler# config
   Entering configuration mode terminal
   scheduler(config)# 
   

2. Enter the aaa authentication command to create the user:

   scheduler(config)# aaa authentication users user test2 gid 100 uid 9000 homedir / password testpassword ssh_keydir /
   scheduler(config-user-test2)# commit
   scheduler(config-user-test2)# exit
   

   Note	The gid, uid, homedir and ssh_keydir are required but not used by the application.

scheduler(config)# nacm groups group oper user-name test2 
scheduler(config-group-oper)# commit

scheduler(config)# nacm groups group grafana-editor user-name test2 
scheduler(config-group-grafana-editor)# commit

scheduler(config)# nacm groups group grafana-admin user-name test2 
scheduler(config-group-grafana-admin)# commit

scheduler# aaa authentication users user test2 change-password 
Value for 'old-password' (<string>): ************
Value for 'new-password' (<string>): ********
Value for 'confirm-password' (<string>): ********
scheduler# 
System message at 2017-03-08 21:17:18...
Commit performed by system via system using system.