Skip to main content

Filebeat

Filebeat is an easy way to send logs from your system to Logz.io. Use the dedicated configuration wizard for a simple setup.

Configure Filebeat on MacOS or Linux​

Pre Requirements​

Before you begin, you'll need:

  • Filebeat installed
  • Port 5015 open to outgoing traffic
  • Root access
note

While support for Filebeat 6.3 and later versions is available, Logz.io recommends that you use the latest stable version

Download the Logz.io public certificate​

For HTTPS shipping, download the Logz.io public certificate to your certificate authority folder.

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

Disabling SSL​

Filebeat uses SSL/TLS to secure the communication between Filebeat and Logz.io. To disable SSL, modify the Filebeat configuration accordingly:

  1. Open the Filebeat configuration file, typically located at /etc/filebeat/filebeat.yml (Linux) or C:\ProgramData\Filebeat\filebeat.yml (Windows).

  2. Find the output.logstash section in the file.

  3. Remove the # character at the beginning of the #ssl.enabled line to disable SSL. The line should now look like this: #ssl.enabled: false

  4. Save the changes and restart the Filebeat service to apply the changes.

Configure Filebeat with Logz.io configuration wizard​

note

Filebeat requires a file extension specified for the log input.

Log in to Logz.io and navigate to the Filebeat log shipping page.

Adding log sources to the configuration file​

  • Select your operating system - Linux or Windows.
  • Specify the full log Path.
  • Select a log Type from the list or select Other to create and specify a custom log type.
  • Select the log format - Plaintext or Json.
  • (Optional) Enable the Multiline option if your log messages span multiple lines. You’ll need to give a regex that identifies the beginning line of each log.
  • (Optional) Add a custom field. Click + Add a field to add additional fields.
note

The wizard makes it simple to add multiple log types to a single configuration file. So to add additional sources, click + Add a log type to fill in the details for another log type. Repeat as necessary.

Filebeat 8.1+​

If you're running Filebeat 8.1+, there are some adjustment you need to make in the config file:

  1. Change type of the filebeat.inputs to filestream instead of logs:
filebeat.inputs:
- type: filestream
  paths:
    - /var/log/*.log
  1. To configure multiline nest the multiline settings under parsers:
- type: filestream
  paths:
    - /var/log/*.log
  parsers:
  - multiline:
      type: pattern
      pattern: '^\d{4}-'
      negate: true
      match: after

Download and validate confiuration​

When you're done adding your sources, click Make the config file to download it.

You can compare it to our sample configuration if you have questions.

Validate the file using a YAML validator tool, such as (Yamllint.com.

Move the configuration file to the Filebeat folder​

Move your configuration file to /etc/filebeat/filebeat.yml.

Start Filebeat and view logs​

Start or restart Filebeat for the changes to take effect.

Allow some time for data ingestion, then open Open Search Dashboards.

If you don't see your logs, see Filebeat's troubleshooting guide.

Supported Modules​

Beat shippers make use of modules to ship data from various sources. Refer to the list below to see which modules each shipper supports.