Question: Can we integrate our logs for a specific service or application with Logz.io?

TL;DR (the short answers)

  • If the logs are only viewable within the 3rd-party service or application, the most likely answer is “no“, because there is no way to have the logs ingested and indexed on our clusters.

  • If the logs are accessible from outside your service or application, then it’s much more likely that you can ship those to Logz.io - depending on how that access is managed.

We offer a variety of Send Your Data integrations to enable you to ship logs directly from your code, based on the language you’re using.

This information is also available in the Logz.io Docs, via Ship data > Logs, when you select the From your code filter.

The detailed answers

One time upload

cURL file upload enables single file shipping and comes native with both MacOs and Linux. The full command syntax can be found in our cURL shipping instructions. Windows users can download the files from the official website.

This method requires the following outside communication ports:

  • Text files: 8021 (HTTP) and 8022 (HTTPS)
  • JSON files: 8070 (HTTP) and 8071 (HTTPS)

This information is also available in the Logz.io Docs, in Ship your data > Logs > cURL file upload.

Invoke-RestMethod file upload allows you to interact with REST APIs in PowerShell to upload JSON or plain text files. You can use Invoke_RestMethod to test your configuration or troubleshoot your connectivity to Logz.io. The full command syntax can be found in Invoke-RestMethod file upload.

This information is also available in the Logz.io Docs, in Ship your data > Logs > Upload log files using Invoke-RestMethod.

Continuous shipping

Filebeat is your best option: It’s lightweight, reliable, and easy to set up. This method requires outside communication on port 5015.

Other options:

  • Logstash: This method requires outside communication on port 5006 (encrypted) or port 5050 (unencrypted).
  • Rsyslog: This method requires outside communication on port 5001.
  • Fluentd: This method requires outside communication on port 8071.

This information is also available in the Logz.io Docs Ship your data section in the Filebeat, Logstash, Rsyslog over TLS, and Fluentd topics.

We have existing integrations with both S3 and Azure Blob storage.

This information is also available in the Logz.io Docs Ship your data section, in the S3 Bucket and Azure Blob Storage topics.

One-time upload

Run the relevant API query per the instructions provided by the application or service and send the response as payload to the Logz.io JSON bulk uploads over HTTP/HTTPs endpoint.

This method requires outside bulk HTTP/S endpoint communication on ports 8070 (HTTP) and 8071 (HTTPS).

The payload must be formatted as inline JSON objects separated by a newline. Arrays or pretty-printed JSON payloads will fail.

This information is also available in the Logz.io Docs Ship your data section, in the Upload JSON logs topic.

Continuous shipping

This method follows the same instructions as the one-time upload, but we recommend that you script the process to save time and effort.