13. Metrics
A metrics endpoint is available in the API: /api/controller/v2/metrics/ that surfaces instantaneous metrics about automation controller, which can be consumed by system monitoring software like the open source project Prometheus.
The type of data shown at the metrics/ endpoint is Content-type: text/plain and application/json as well. This endpoint contains useful information, such as counts of how many active user sessions there are, or how many jobs are actively running on each automation controller node. Prometheus can be configured to scrape these metrics from automation controller by hitting automation controller metrics endpoint and storing this data in a time-series database. Clients can later use Prometheus in conjunction with other software like Grafana or Metricsbeat to visualize that data and set up alerts.
13.1. Set up Prometheus
To set up and use Prometheus, you will need to install Prometheus on a virtual machine or container. Refer to the Prometheus documentation for further detail.
Create a token of a platform user that can read the metrics (for example a platform auditor) in Access Management > API Tokens, or through
/api/gateway/v1/tokens/(see API tokens and OAuth applications). In the Prometheus config file (typicallyprometheus.yml), scrape the metrics of automation controller through the platform address with that token:
scrape_configs: - job_name: 'koios-controller' metrics_path: /api/controller/v2/metrics/ scrape_interval: 30s scheme: https authorization: credentials: <token_value> tls_config: ca_file: /etc/prometheus/koios-ca.crt # remove tls_config if the certificate is publicly trusted static_configs: - targets: - <koios_host>For help configuring other aspects of Prometheus, such as alerts and service discovery configurations, refer to the Prometheus configuration docs.
If Prometheus is already running, reload its configuration: send it
SIGHUP, or make a POST to/-/reloadwhen it runs with--web.enable-lifecycle.
Use a browser to navigate to your graph in the Prometheus UI at
http://your_prometheus:9090/graphand test out some queries. For example, you can query the number of sessions stored by automation controller withawx_sessions_total{type="user"}; platform logins are sessions of the platform gateway, which this metric does not count.
Refer to the metrics endpoint of the automation controller API (/api/controller/v2/metrics/) for more ways to query.