Skip to main content

AWS EC2 Auto Scaling

Amazon EC2 Auto Scaling integration with allows you to monitor the reliability, availability, and performance of all your EC2 instances in one place.


This service integration is specifically designed to work with the destination bucket to which the service writes its logs.

It is based on the service's naming convention and path structure.

If you're looking to ship the service's logs from a different bucket, please use the S3 Bucket shipping method instead.

Before you begin:

  • If you plan on using an access key to authenticate your connection, you'll need to set the s3:ListBucket and s3:GetObject permissions for the required S3 bucket.

  • If you plan on using an IAM role to authenticate your connection, you can get the role policy by filling out the bucket information and clicking the "Get the role policy" button.

  • File names in ascending alphanumeric order. This is important because the S3 fetcher's offset is determined by the name of the last file fetched. We recommend using standard AWS naming conventions to determine the file name ordering and to avoid log duplication.

Send your logs to an S3 bucket fetches your CloudTrail logs from an S3 bucket.

For help with setting up a new trail, see Overview for Creating a Trail from AWS.

Verify bucket definition on AWS

Navigate to the location of your trail logs on AWS:

Trail location

And verify the definition of the bucket is under the CloudTrail path:

Trail definition

Region data must be created under the CloudTrain path BEFORE the S3 bucket is defined on Otherwise, you won't be able to proceed with sending CloudTrail data to Trail regions :::

Next, note the bucket's name and the way the prefix is constructed, for example:

Bucket name: aws-cloudtrail-logs-486140753397-9f0d7dbd.

Prefix name: AWSLogs/486140753397/CloudTrail/.

You'll need these values when adding your S3 bucket information.


Add your S3 bucket information

To use the S3 fetcher, log into your account, and go to the CloudTrail log shipping page.

  1. Click + Add a bucket
  2. Select your preferred method of authentication - an IAM role or access keys.

The configuration wizard will open.

  1. Provide the S3 bucket name
  2. Provide your Prefix. That is your CloudTrail path. See further details below.
  3. There is no Region selection box because it is not needed. will pull data from all regions in AWS for the specified bucket and account.
  4. Choose whether you want to include the source file path. This saves the path of the file as a field in your log.
  5. Save your information.

S3 bucket configuration wizard

Getting the information from your CloudTrail AWS path

You may need to fill in 2 parameters when creating the bucket - {BUCKET_NAME} and {PREFIX}. You can find them in your CloudTrail AWS path. The AWS path structure for CloudTrail looks like the examle below:

  • {BUCKET_NAME} is your S3 bucket name.

  • {PREFIX} is your CloudTrail path. The prefix is generate by default and represents the complete path inside the bucket up until the regions section. It should look like this: AWSLogs/{AWS_ACCOUNT_ID}/CloudTrail/.

Important fetches logs that are generated after configuring an S3 bucket. cannot fetch past logs retroactively.

Check for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don't see your logs, see log shipping troubleshooting.


Problem: Failed to save bucket configuration

The following error appears when you're trying to create a bucket:

AWS failed to create cloudtrail bucket. Exception AWS bucket is empty: 403.

Possible cause

The bucket's location is incorrect or might be missing the correct prefix.

Suggested remedy

  1. Head to CloudTrail console on AWS and check the relevant trail:

Dashboard trail

  1. Verify that the location of the trail is correct:

Trail location

And verify that the prefix contains all parts:

Prefix trail

In this case, the cause of the error is that the location is empty or that the prefix is wrong.

The bucket should be aws-cloudtrail-logs-486140753397-9f0d7dbd, and the prefix should be AWSLogs/486140753397/CloudTrail/. You can click on the prefix to verify that it is empty.

Once you fix these issues, you can return to to create the CloudTrail bucket.



For a much easier and more efficient way to collect and send metrics, consider using the telemetry collector.

Deploy this integration to send your Amazon EC2 Auto Scaling metrics to

This integration creates a Kinesis Data Firehose delivery stream that links to your Amazon EC2 Auto Scaling metrics stream and then sends the metrics to your account. It also creates a Lambda function that adds AWS namespaces to the metric stream, and a Lambda function that collects and ships the resources' tags.

Log in to your account and navigate to the current instructions page inside the app. Install the pre-built dashboard to enhance the observability of your metrics.

To view the metrics on the main dashboard, log in to your Metrics account, and open the Metrics tab.

Before you begin, you'll need:

  • An active account with

Configure AWS to forward metrics to

Set the required minimum IAM permissions

Make sure you have configured the minimum required IAM permissions as follows:

  • Amazon S3:
    • s3:CreateBucket
    • s3:DeleteBucket
    • s3:PutObject
    • s3:GetObject
    • s3:DeleteObject
    • s3:ListBucket
    • s3:AbortMultipartUpload
    • s3:GetBucketLocation
  • AWS Lambda:
    • lambda:CreateFunction
    • lambda:DeleteFunction
    • lambda:InvokeFunction
    • lambda:GetFunction
    • lambda:UpdateFunctionCode
    • lambda:UpdateFunctionConfiguration
    • lambda:AddPermission
    • lambda:RemovePermission
    • lambda:ListFunctions
  • Amazon CloudWatch:
    • cloudwatch:PutMetricData
    • cloudwatch:PutMetricStream
    • logs:CreateLogGroup
    • logs:CreateLogStream
    • logs:PutLogEvents
    • logs:DeleteLogGroup
    • logs:DeleteLogStream
  • AWS Kinesis Firehose:
    • firehose:CreateDeliveryStream
    • firehose:DeleteDeliveryStream
    • firehose:PutRecord
    • firehose:PutRecordBatch
  • IAM:
    • iam:PassRole
    • iam:CreateRole
    • iam:DeleteRole
    • iam:AttachRolePolicy
    • iam:DetachRolePolicy
    • iam:GetRole
    • iam:CreatePolicy
    • iam:DeletePolicy
    • iam:GetPolicy
  • Amazon CloudFormation:
    • cloudformation:CreateStack
    • cloudformation:DeleteStack
    • cloudformation:UpdateStack
    • cloudformation:DescribeStacks
    • cloudformation:DescribeStackEvents
    • cloudformation:ListStackResources

Create Stack in the relevant region

To deploy this project, click the button that matches the region you wish to deploy your Stack to:

us-east-1Deploy to AWS
us-east-2Deploy to AWS
us-west-1Deploy to AWS
us-west-2Deploy to AWS
eu-central-1Deploy to AWS
eu-central-2Deploy to AWS
eu-north-1Deploy to AWS
eu-west-1Deploy to AWS
eu-west-2Deploy to AWS
eu-west-3Deploy to AWS
eu-south-1Deploy to AWS
eu-south-2Deploy to AWS
sa-east-1Deploy to AWS
ap-northeast-1Deploy to AWS
ap-northeast-2Deploy to AWS
ap-northeast-3Deploy to AWS
ap-south-1Deploy to AWS
ap-south-2Deploy to AWS
ap-southeast-1Deploy to AWS
ap-southeast-2Deploy to AWS
ap-southeast-3Deploy to AWS
ap-southeast-4Deploy to AWS
ap-east-1Deploy to AWS
ca-central-1Deploy to AWS
ca-west-1Deploy to AWS
af-south-1Deploy to AWS
me-south-1Deploy to AWS
me-central-1Deploy to AWS
il-central-1Deploy to AWS

Specify stack details

Specify the stack details as per the table below, check the checkboxes and select Create stack.

logzioListenerThe listener URL for your region. (For more details, see the regions page. For example -
logzioTokenYour metrics shipping token.Required
awsNamespacesComma-separated list of the AWS namespaces you want to monitor. See this list of namespaces. If you want to automatically add all namespaces, use value all-namespaces.At least one of awsNamespaces or customNamespace is required
customNamespaceA custom namespace for CloudWatch metrics. This is used to specify a namespace unique to your setup, separate from the standard AWS namespaces.At least one of awsNamespaces or customNamespace is required
logzioDestinationYour destination URL.Required
httpEndpointDestinationIntervalInSecondsThe length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination.60
httpEndpointDestinationSizeInMBsThe size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination.5
debugModeEnable debug mode for detailed logging (true/false).false

Check for your metrics

Give your data some time to get from your system to ours, then log in to your Metrics account, and open the Metrics tab.

Log in to your account and navigate to the current instructions page inside the app. Install the pre-built dashboard to enhance the observability of your metrics.

To view the metrics on the main dashboard, log in to your Metrics account, and open the Metrics tab.