Ship Puppet logs

Setup

Configuration tl;dr

Files

Sample configuration
Encryption certificate

Listener

Port 5015. For help finding your region’s listener host, see Account region.

Default log location

Puppet produces lots of different logs. See Log files installed from Puppet Labs for more information.

Log type (for preconfigured parsing)

Puppet server: puppetserver
Puppet server access: puppetserver-access

Guided configuration

You’ll need: Filebeat 7 or Filebeat 6, root access

1. Download the Logz.io certificate

   For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

   sudo wget https://raw.githubusercontent.com/logzio/public-certificates/master/COMODORSADomainValidationSecureServerCA.crt -P /etc/pki/tls/certs/
   

2. Add Puppet as an input

   In the Filebeat configuration file (/etc/filebeat/filebeat.yml), add Puppet to the filebeat.inputs section.

   We recommend configuring Puppet to output JSON logs. See Advanced Logging Configuration from Puppet for more information.

   Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.

   # ...
   filebeat.inputs:
   - type: log
     paths:
   
     # If you configured Puppet to output JSON logs, replace the filename with
     #  `puppetserver.log.json`
     - /var/log/puppetlabs/puppetserver/puppetserver.log
   
     fields:
   
       # If you configured Puppet to output JSON logs, set logzio_codec to
       #  `json`
       logzio_codec: plain
   
       # Your Logz.io account token. You can find your token at
       #  https://app.logz.io/#/dashboard/settings/manage-accounts
       token: <<SHIPPING-TOKEN>>
       type: puppetserver
     fields_under_root: true
     encoding: utf-8
     ignore_older: 3h
   
   - type: log
     paths:
   
     # If you configured Puppet to output JSON logs, replace the filename with
     #  `puppetserver-access.log.json`
     - /var/log/puppetlabs/puppetserver/puppetserver-access.log
   
     fields:
   
       # If you configured Puppet to output JSON logs, set logzio_codec to
       #  `json`
       logzio_codec: plain
   
       # Your Logz.io account token. You can find your token at
       #  https://app.logz.io/#/dashboard/settings/manage-accounts
       token: <<SHIPPING-TOKEN>>
       type: puppetserver-access
     fields_under_root: true
     encoding: utf-8
     ignore_older: 3h
   

   If you’re running Filebeat 7, paste this code block. Otherwise, you can leave it out.

   # ... For Filebeat 7 only ...
   filebeat.registry.path: /var/lib/filebeat
   processors:
   - rename:
       fields:
       - from: "agent"
         to: "filebeat_agent"
       ignore_missing: true
   - rename:
       fields:
       - from: "log.file.path"
         to: "source"
       ignore_missing: true
   

   If you’re running Filebeat 6, paste this code block.

   # ... For Filebeat 6 only ...
   registry_file: /var/lib/filebeat/registry
   

3. Add Logz.io as an output

   If Logz.io is not an output, add it now.

   Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.

   # ...
   output.logstash:
     hosts: ["<<LISTENER-HOST>>:5015"]
     ssl:
       certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']
   

4. Start Filebeat

   Start or restart Filebeat for the changes to take effect.

5. Check Logz.io for your logs

   Give your logs a few minutes to get from your system to ours, and then open Kibana.

   If you still don’t see your logs, see log shipping troubleshooting.