With this integration, you can collect logs from an S3 bucket and forward them to Logz.io.
Create new stack
To deploy this project, click the button that matches the region you wish to deploy your Stack to:
| Region | Deployment |
|---|---|
us-east-1 |
 |
us-east-2 |
|
us-west-1 |
|
us-west-2 |
|
eu-central-1 |
|
eu-north-1 |
|
eu-west-1 |
|
eu-west-2 |
|
eu-west-3 |
|
sa-east-1 |
|
ap-northeast-1 |
|
ap-northeast-2 |
|
ap-northeast-3 |
|
ap-south-1 |
|
ap-southeast-1 |
|
ap-southeast-2 |
|
ca-central-1 |
|
Specify stack details
Specify the stack details as per the table below, check the checkboxes and select Create stack.
| Parameter | Description | Required/Default |
|---|---|---|
bucketName |
Name of the bucket you wish to fetch logs from. Will be used for IAM policy. | Required |
logzioListener |
The Logz.io listener URL for your region. (For more details, see the regions page | Required |
logzioToken |
Your Logz.io log shipping token. | Required |
logLevel |
Log level for the Lambda function. Can be one of: debug, info, warn, error, fatal, panic. |
Default: info |
logType |
The log type you’ll use with this Lambda. This is shown in your logs under the type field in Kibana. Logz.io applies parsing based on the log type. | Default: s3_hook |
pathsRegexes |
Comma-seperated list of regexes that match the paths you’d like to pull logs from. | - |
pathToFields |
Fields from the path to your logs directory that you want to add to the logs. For example, org-id/aws-type/account-id will add each of the fields ord-id, aws-type and account-id to the logs that are fetched from the directory that this path refers to. |
- |
Add trigger
Give the stack a few minutes to be deployed.
Once your Lambda function is ready, you’ll need to manually add a trigger. This is due to Cloudformation limitations.
Go to the function’s page, and click on Add trigger.
Then, choose S3 as a trigger, and fill in:
- Bucket: Your bucket name.
- Event type: Choose option
All object create events. - Prefix and Suffix should be left empty.
Confirm the checkbox, and click *Add.
Send logs
That’s it. Your function is configured. Once you upload new files to your bucket, it will trigger the function, and the logs will be sent to your Logz.io account.
Parsing
S3 Hook will automatically parse logs in the following cases:
- The object’s path contains the phrase
cloudtrail(case insensitive).
Check Logz.io for your logs
Give your logs some time to get from your system to ours, and then open Kibana.
If you still don’t see your logs, see log shipping troubleshooting.