You can ship HashiCorp Consul metrics using Metricbeat.

You will need to repeat the following procedure for each Consul server and client that will be sending metrics to Logz.io.

Metricbeat setup

Before you begin, you’ll need:

Update Consul configuration with the telemetry stanza

Go to one of the Consul servers and create a new file named prometheus.json under your Consul config library. ( It’s usually under /etc/consul.d.)

Update the file with the following telemetry stanza:

{
  "telemetry": {
    "disable_hostname": true,
    "prometheus_retention_time": "72h"
  }
}

Save the file and restart the Consul on the server.

Test the localhost on the Consul server

Now the metrics for this Consul server will be exposed locally in Prometheus format under the following endpoint:

http://127.0.0.1:8500/v1/agent/metrics?format=prometheus
Download the Logz.io public certificate to your Metricbeat server

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

You’ll need to run this command on the server that hosts Metricbeat.

Set metricbeat modules on Consul servers

Open the Metricbeat configuration file (<<PATH_TO_METRICBEAT>>/metricbeat.yml) with your preferred text editor.

Copy and paste the code block below, overwriting the previous contents, to replace the general configuration with the required module settings:

metricbeat.modules:
- module: prometheus
  period: 10s
  hosts: ["localhost:8500"]
  metrics_path: /v1/agent/metrics
  query:
    format: prometheus
  processors:
    - add_fields:
        fields:
          module: consul

- module: system
  metricsets:
    - cpu              # CPU usage
    - load             # CPU load averages
    - memory           # Memory usage
    - network          # Network IO
    - diskio
  enabled: true
  period: 10s
  cpu.metrics:  ["percentages","normalized_percentages"]  # The other available option is ticks.
  processors:
    - add_fields:
        fields:
          module: consul
Add Logz.io to your Metricbeat configuration

Still in the same configuration file, copy and paste the code block below with the following Logz.io settings:

# ===== General =====
fields:
  logzio_codec: json
  token: <<METRICS-SHIPPING-TOKEN>>
fields_under_root: true

Replace <<METRICS-SHIPPING-TOKEN>> with a token for the Metrics account you want to ship to.
Look up your Metrics token.

Set Logz.io as the output

Still in the same configuration file, check if Logz.io is already an output. If not, add it now.

# ===== Outputs =====
output.logstash:
  hosts: ["<<LISTENER-HOST>>:5015"]
    ssl.certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']

Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.

One last validation - make sure Logz.io is the only output and appears only once. If the file has other outputs, remove them.

The final metricbeat.yml file should look like this:

- module: prometheus
  period: 10s
  hosts: ["localhost:8500"]
  metrics_path: /v1/agent/metrics
  query:
    format: prometheus
  processors:
    - add_fields:
        fields:
          module: consul

- module: system
  metricsets:
    - cpu              # CPU usage
    - load             # CPU load averages
    - memory           # Memory usage
    - network          # Network IO
    - diskio
  enabled: true
  period: 10s
  cpu.metrics:  ["percentages","normalized_percentages"]  # The other available option is ticks.
  processors:
    - add_fields:
        fields:
          module: consul

fields:
  logzio_codec: json
  token: <<METRICS-SHIPPING-TOKEN>>
fields_under_root: true
ignore_older: 3hr
type: metrics
output.logstash:
  hosts: ["<<LISTENER-HOST>>:5015"]
  ssl.certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']
Start Metricbeat

Start or restart Metricbeat for the changes to take effect.

Open your Consul dashboard in Logz.io

Give your metrics a few minutes to get from your system to ours, and then open Logz.io.

Your metrics should appear in the preconfigured dashboard in your Metrics account. To open it, search your Logz.io Metrics dashboards for Consul.