Configuration

Before you begin, you’ll need:

  • s3:ListBucket and s3:GetObject permissions for the required S3 bucket

  • File names in ascending alphanumeric order. This is important because the S3 fetcher’s offset is determined by the name of the last file fetched. We recommend using standard AWS naming conventions to determine the file name ordering and to avoid log duplication.

Send your logs to an S3 bucket

Logz.io fetches your S3 access logs from a separate S3 bucket. S3 access logs are not enabled S3 by default, so you’ll need to set this up.

For help with this, see Amazon S3 Server Access Logging from AWS.

Add your S3 bucket information

To use the S3 fetcher, log into your Logz.io account, and go to the S3 access log shipping page.

  1. Click + Add a bucket
  2. Select your preferred method of authentication - an IAM role or access keys.

The configuration wizard will open.

  1. Select the hosting region from the dropdown list.
  2. Provide the S3 bucket name
  3. Optional You have the option to add a prefix.
  4. Save your information.

S3 bucket configuration wizard

Logz.io fetches logs that are generated after configuring an S3 bucket. Logz.io cannot fetch past logs retroactively.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana.

If you still don’t see your logs, see log shipping troubleshooting.