Skip to main content

Jenkins

Logs

Shipping Jenkins logs with Filebeat

Before you begin, you'll need:

  • Filebeat installed
  • Root access
  • Port 5015 open
Download the Logz.io public certificate to your credentials server

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

Add Jenkins as an input

In the Filebeat configuration file (/etc/filebeat/filebeat.yml), add Jenkins to the filebeat.inputs section.

Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to.

Replace "JENKINS-HOME" with home location of your Jenkins installation

note

Filebeat requires a file extension specified for the log input.

# ...
filebeat.inputs:
- type: filestream
paths:
- /var/log/jenkins/jenkins.log
- /var/<<JENKINS-HOME>>/jobs/*/builds/lastFailedBuild/log
fields:
logzio_codec: plain

# You can manage your tokens at
# https://app.logz.io/#/dashboard/settings/manage-tokens/log-shipping
token: <<LOG-SHIPPING-TOKEN>>
type: jenkins
fields_under_root: true
encoding: utf-8
ignore_older: 3h
multiline:
pattern: '^[A-Z]{1}[a-z]{2} {1,2}[0-9]{1,2}, [0-9]{4} {1,2}[0-9]{1,2}:[0-9]{2}:[0-9]{2}'
negate: true
match: after

If you're running Filebeat 7 to 8.1, paste the code block below instead:

# ...
filebeat.inputs:
- type: log
paths:
- /var/log/jenkins/jenkins.log
- /var/<<JENKINS-HOME>>/jobs/*/builds/lastFailedBuild/log
fields:
logzio_codec: plain

# You can manage your tokens at
# https://app.logz.io/#/dashboard/settings/manage-tokens/log-shipping
token: <<LOG-SHIPPING-TOKEN>>
type: jenkins
fields_under_root: true
encoding: utf-8
ignore_older: 3h
multiline:
pattern: '^[A-Z]{1}[a-z]{2} {1,2}[0-9]{1,2}, [0-9]{4} {1,2}[0-9]{1,2}:[0-9]{2}:[0-9]{2}'
negate: true
match: after

The above configuration assumes the following defaults:

  • Log location - /var/log/jenkins/jenkins.log
  • Log type - jenkins

Set Logz.io as the output

If Logz.io is not an output, add it now. Remove all other outputs.

Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.

# ...
output.logstash:
hosts: ["<<LISTENER-HOST>>:5015"]
ssl:
certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']

Start Filebeat

Start or restart Filebeat for the changes to take effect.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards. You can search for type:jenkins to filter for your Jenkins logs. Your logs should be already parsed thanks to the Logz.io preconfigured parsing pipeline.

If you still don't see your logs, see Filebeat troubleshooting.

Metrics

To send your Prometheus-format Jenkins metrics to Logz.io, you need to add the inputs.prometheus and outputs.http plug-ins to your Telegraf configuration file.

Configure Telegraf to send your metrics data to Logz.io

Set up Telegraf v1.17 or higher

For Windows

wget https://dl.influxdata.com/telegraf/releases/telegraf-1.27.3_windows_amd64.zip

After downloading the archive, extract its content into C:\Program Files\Logzio\telegraf\.

The configuration file is located at C:\Program Files\Logzio\telegraf\.

For MacOS

brew install telegraf

The configuration file is located at /usr/local/etc/telegraf.conf.

For Linux

Ubuntu & Debian

sudo apt-get update && sudo apt-get install telegraf

The configuration file is located at /etc/telegraf/telegraf.conf.

RedHat and CentOS

sudo yum install telegraf

The configuration file is located at /etc/telegraf/telegraf.conf.

SLES & openSUSE

# add go repository
zypper ar -f obs://devel:languages:go/ go
# install latest telegraf
zypper in telegraf

The configuration file is located at /etc/telegraf/telegraf.conf.

FreeBSD/PC-BSD

sudo pkg install telegraf

The configuration file is located at /etc/telegraf/telegraf.conf.

Enable the metrics plugin from the Jenkins Web UI

First you need to add the metrics plug-in to your Jenkins configuration. To do this:

  1. Login to the Jenkins Web UI and navigate to Manage Jenkins > Manage Plugins > Available.
  2. Select Prometheus metrics and click Install without restart.
Add the inputs.prometheus plug-in

Now you need to configure the input plug-in to enable Telegraf to scrape the Jenkins data from your hosts. To do this, add the following code to the configuration file:

[[inputs.prometheus]]
# An array of urls to scrape metrics from.
urls = ["http://localhost:8080/prometheus/"]
metric_version = 2
Add the outputs.http plug-in

After you create the configuration file, configure the output plug-in to enable Telegraf to send your data to Logz.io in Prometheus-format. To do this, add the following code to the configuration file:

[[outputs.http]]
url = "https://<<LISTENER-HOST>>:8053"
data_format = "prometheusremotewrite"
[outputs.http.headers]
Content-Type = "application/x-protobuf"
Content-Encoding = "snappy"
X-Prometheus-Remote-Write-Version = "0.1.0"
Authorization = "Bearer <<PROMETHEUS-METRICS-SHIPPING-TOKEN>>"

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<LISTENER-HOST>> with the Logz.io Listener URL for your region, configured to use port 8052 for http traffic, or port 8053 for https traffic. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.
  • Replace <<PROMETHEUS-METRICS-SHIPPING-TOKEN>> with a token for the Metrics account you want to ship to.
    Here's how to look up your Metrics token.

Check Logz.io for your metrics

Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboards to enhance the observability of your metrics.

To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.