With this integration, you can collect logs from an S3 bucket and forward them to Logz.io.

Create new stack

To deploy this project, click the button that matches the region you wish to deploy your Stack to:

Region Deployment
us-east-1 Deploy to AWS
us-east-2 Deploy to AWS
us-west-1 Deploy to AWS
us-west-2 Deploy to AWS
eu-central-1 Deploy to AWS
eu-north-1 Deploy to AWS
eu-west-1 Deploy to AWS
eu-west-2 Deploy to AWS
eu-west-3 Deploy to AWS
sa-east-1 Deploy to AWS
ap-northeast-1 Deploy to AWS
ap-northeast-2 Deploy to AWS
ap-northeast-3 Deploy to AWS
ap-south-1 Deploy to AWS
ap-southeast-1 Deploy to AWS
ap-southeast-2 Deploy to AWS
ca-central-1 Deploy to AWS
Specify stack details

Specify the stack details as per the table below, check the checkboxes and select Create stack.

Parameter Description Required/Default
bucketName Name of the bucket you wish to fetch logs from. Will be used for IAM policy. Required
logzioListener The Logz.io listener URL for your region. (For more details, see the regions page Required
logzioToken Your Logz.io log shipping token. Required
logLevel Log level for the Lambda function. Can be one of: debug, info, warn, error, fatal, panic. Default: info
logType The log type you’ll use with this Lambda. This is shown in your logs under the type field in Kibana. Logz.io applies parsing based on the log type. Default: s3_hook
pathsRegexes Comma-seperated list of regexes that match the paths you’d like to pull logs from. -
pathToFields Fields from the path to your logs directory that you want to add to the logs. For example, org-id/aws-type/account-id will add each of the fields ord-id, aws-type and account-id to the logs that are fetched from the directory that this path refers to. -
Add trigger

Give the stack a few minutes to be deployed.

Once your Lambda function is ready, you’ll need to manually add a trigger. This is due to Cloudformation limitations.

Go to the function’s page, and click on Add trigger.

Step 5 screenshot

Then, choose S3 as a trigger, and fill in:

  • Bucket: Your bucket name.
  • Event type: Choose option All object create events.
  • Prefix and Suffix should be left empty.

Confirm the checkbox, and click *Add.

Step 5 screenshot

Send logs

That’s it. Your function is configured. Once you upload new files to your bucket, it will trigger the function, and the logs will be sent to your Logz.io account.

Parsing

S3 Hook will automatically parse logs in the following cases:

  • The object’s path contains the phrase cloudtrail (case insensitive).
Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana.

If you still don’t see your logs, see log shipping troubleshooting.