Filebeat configuration

Before you begin, you’ll need:

Download the Logz.io public certificate to your Filebeat server

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt
Add nginx as an input

In the Filebeat configuration file (/etc/filebeat/filebeat.yml), add nginx to the filebeat.inputs section.

Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to.

# ...
filebeat.inputs:
- type: log
  paths:
  - /var/log/nginx/access.log

  fields:
    logzio_codec: plain

    # Your Logz.io account token. You can find your token at
    #  https://app.logz.io/#/dashboard/settings/manage-accounts
    token: <<LOG-SHIPPING-TOKEN>>
    type: nginx_access
  fields_under_root: true
  encoding: utf-8
  ignore_older: 3h

- type: log
  paths:
  - /var/log/nginx/error.log

  fields:
    logzio_codec: plain

    # Your Logz.io account token. You can find your token at
    #  https://app.logz.io/#/dashboard/settings/manage-accounts
    token: <<LOG-SHIPPING-TOKEN>>
    type: nginx_error
  fields_under_root: true
  encoding: utf-8
  ignore_older: 3h

If you’re running Filebeat 7, paste this code block. Otherwise, you can leave it out.

# ... For Filebeat 7 only ...
filebeat.registry.path: /var/lib/filebeat
processors:
- rename:
    fields:
    - from: "agent"
      to: "filebeat_agent"
    ignore_missing: true
- rename:
    fields:
    - from: "log.file.path"
      to: "source"
    ignore_missing: true

If you’re running Filebeat 6, paste this code block.

# ... For Filebeat 6 only ...
registry_file: /var/lib/filebeat/registry

The above assumes the following defaults for Access logs:

  • Log location - /var/log/nginx/access.log
  • Log type - nginx, nginx_access, or nginx-access

Defaults for Error logs:

  • Log location - /var/log/nginx/error.log
  • Log type - nginx-error
Set Logz.io as the output

If Logz.io is not an output, add it now. Remove all other outputs.

Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe.

# ...
output.logstash:
  hosts: ["<<LISTENER-HOST>>:5015"]
  ssl:
    certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']
Start Filebeat

Start or restart Filebeat for the changes to take effect.

Check Logz.io for your logs

Confirm you’re shipping logs by opening an nginx-hosted webpage in your browser. Give your logs some time to get from your system to ours, and then open Kibana.

You can search for type:nginx OR nginx_access OR nginx-access OR nginx-error to filter for your logs. Your logs should be already parsed thanks to the Logz.io preconfigured parsing pipeline.

If you still don’t see your logs, see log shipping troubleshooting.