Create a new Lambda function
This Lambda function will collect CloudWatch logs and sends them to Logz.io in bulk over HTTP.
Open the AWS Lambda Console, and click Create function. Choose Author from scratch, and use this information:
- Name: We suggest adding the log type to the name, but you can name this function whatever you want.
- Runtime: Choose Python 2.7
- Role: Click Create new role from template(s). Under Existing role, select Basic Edge Lambda permissions
Click Create Function (bottom right corner of the page). After a few moments, you’ll see configuration options for your Lambda function.
You’ll need this page later on, so keep it open.
Zip the source files
Download the CloudWatch Logs Shipper - Lambda project from GitHub to your computer, and zip the Python files in the src/ folder.
mkdir dist; cp -r ../shipper dist/ && cp src/lambda_function.py dist/ && cd dist/ && zip logzio-cloudwatch shipper/* lambda_function.py
Upload the zip file and set environment variables
In the Function code section of Lambda find the Code entry type list. Choose Upload a .ZIP file from this list.
Click Upload, and choose the zip file you created earlier (
In the Environment variables section, set your Logz.io account token, URL, and log type, and any other variables that you need to use.
- The token of the account you want to ship to.
- Your region’s listener host. For more information on finding your account’s region, see Account region.
- The log type you’ll use with this Lambda.
This can be a built-in log type, or a custom log type.
Please note that you should create a new Lambda for each log type you use.
json, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields.
- Set to
trueto compress logs before sending them. Set to
falseto send uncompressed logs.
Configure the function’s basic settings
In Basic settings, we recommend starting with these settings:
- Memory: 512 MB
- Timeout: 1 min 0 sec
These default settings are just a starting point. Check your Lambda usage regularly, and adjust these values if you need to.
Set the CloudWatch Logs event trigger
Find the Add triggers list (left side of the Designer panel). Choose CloudWatch Logs from this list.
Below the Designer, you’ll see the Configure triggers panel. Choose the Log group that the Lambda function will watch.
Type a Filter name (required) and Filter pattern (optional).
Click Add, and then click Save at the top of the page.
Check Logz.io for your logs
Give your logs a few minutes to get from your system to ours, and then open Kibana.
If you still don’t see your logs, see log shipping troubleshooting.