AWS Cost and Usage Reports
AWS Cost and Usage Reports function tracks your AWS usage and provides estimated charges associated with your account. This integration allows you to ship logs from your AWS Cost and Usage Reports to your Logz.io account.
Your Lambda function needs to run within the AWS Lambda limits, such as memory allocation and timeout. Make sure you understand these limits. If you can't adjust your settings to stay within the Lambda limits, you can use the AWS Support Center console to request an increase. Learn more about AWS Lambda Limits.
Automated CloudFormation deployment
This deployment will automatically create the following resources:
Login to your account
To begin, you need to login to your AWS account.
Create a new stack
Select the button below to create a new stack dedicated to sending the AWS cost and usage reports to Logz.io.
Keep the default setting in the Create stack screen and select Next.
Specify the stack details
Specify the stack details as per the table below and select Next.
Parameter | Description |
---|---|
Stack name | logzio-cur-auto-deployment |
CloudWatchEventScheduleExpression | Default: rate(10 hours) The scheduling expression that determines when and how often the Lambda function runs. Logz.io recommends to start with 10 hours rate. |
LambdaMemorySize | Default: 1024 (MB) The amount of memory available to the function at runtime. Logz.io recommends to start with 1024 MB. |
LambdaTimeout | Default: 300 (seconds) The amount of time that Lambda allows a function to run before stopping it. Logz.io recommends to start with 300 seconds (5 minutes). |
LogzioToken | Your Logz.io log shipping token:<<LOG-SHIPPING-TOKEN>> Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
LogzioURL | The Logz.io listener URL: https://<<LISTENER-HOST>>:8071 Replace <<LISTENER-HOST>> with the host for your region. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. |
ReportAdditionalSchemaElements | Choose INCLUDE if you want AWS to include additional details about individual resources IDs in the report (This might significantly increase the report size and might affect performance. AWS Lambda can run for up to 15 minutes with up to 10240 MB, and the process time for the whole file must end within this timeframe.) This is an optional parameter. |
ReportName | The name of report that you want to create. |
ReportPrefix | The prefix that AWS adds to the report name when AWS delivers the report. |
ReportTimeUnit | The granularity of the line items in the report. Can be Hourly, Daily or Monthly. (Enabling hourly reports does not mean that a new report is generated every hour. It means that data in the report is aggregated with a granularity of one hour.) |
S3BucketName | The name for the bucket which will contain the report files. |
Configure the stack options
Specify the Key and Value parameters for the Tags and select Next.
Review the deployment
Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select Create stack.
Check Logz.io for your logs
Give your logs the time specified in CloudWatchEventScheduleExpression
to get from your system to ours, and then open Open Search Dashboards.
All logs that were sent from the Lambda function will be under the type billing.
To get more out of this functionality, you can enable a dedicated AWS cost and usage dashboard in ELK Apps.
If you still don't see your logs, see log shipping troubleshooting.
Deployment using a module
Deploy this integration to add a module for AWS cost and usage report to your existing stack. This integration uses CloudWatch Public Registry.
Logz.io Public Registry extensions are currently only available on the AWS region us-east-1
.
At the moment, our AWS Lambda-based integrations do not support working with test events. This functionality is in development.
Before you begin, you'll need:
- A CloudFormation stack
- An S3 bucket to store the CloudFormation package
Select the Logz.io AWS Cost and Usage extension
- Navigate to CloudFormation > Registry > Public extensions.
- Set Extension type > Modules and Publisher > Third party.
- Select Logzio::awsCostAndUsage::cur::MODULE.
Activate the Logz.io AWS Cost and Usage extension
- On the Logzio::awsCostAndUsage::cur::MODULE select Activate.
- In the Extension details section, select Use default.
- In the Automatic updates section, select On.
- Select Activate extension.
Copy the configuration template
On the Logzio::awsCostAndUsage::cur::MODULE page, navigate to Example template and select Copy template.
Add your stack values to the configuration template
{
"Resources": {
"MyModule": {
"Type": "Logzio::awsCostAndUsage::cur::MODULE",
"Properties": {
"S3BucketName": "S3BucketName",
"ReportName": "ReportName",
"ReportPrefix": "ReportPrefix",
"ReportTimeUnit": "ReportTimeUnit",
"ReportAdditionalSchemaElements": "ReportAdditionalSchemaElements",
"LogzioURL": "https://<<LISTENER-HOST>>:8071",
"LogzioToken": "<<LOG-SHIPPING-TOKEN>>",
"LambdaMemorySize": "LambdaMemorySize",
"LambdaTimeout": "LambdaTimeout",
"CloudWatchEventScheduleExpression": "CloudWatchEventScheduleExpression"
}
}
}
}
Save the template as a yaml file and add the values of your stack to the as per the table below.
Parameter | Description |
---|---|
CloudWatchEventScheduleExpression | Default: rate(10 hours) The scheduling expression that determines when and how often the Lambda function runs. Logz.io recommends to start with 10 hours rate. |
LambdaMemorySize | Default: 1024 (MB) The amount of memory available to the function at runtime. Logz.io recommends to start with 1024 MB. |
LambdaTimeout | Default: 300 (seconds) The amount of time that Lambda allows a function to run before stopping it. Logz.io recommends to start with 300 seconds (5 minutes). |
LogzioToken | Your Logz.io log shipping token:<<LOG-SHIPPING-TOKEN>> Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. |
LogzioURL | The Logz.io listener URL: https://<<LISTENER-HOST>>:8071 Replace <<LISTENER-HOST>> with the host for your region. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. |
ReportAdditionalSchemaElements | Choose INCLUDE if you want AWS to include additional details about individual resources IDs in the report (This might significantly increase the report size and might affect performance. AWS Lambda can run for up to 15 minutes with up to 10240 MB, and the process time for the whole file must end within this timeframe.) This is an optional parameter. |
ReportName | The name of report that you want to create. |
ReportPrefix | The prefix that AWS adds to the report name when AWS delivers the report. |
ReportTimeUnit | The granularity of the line items in the report. Can be Hourly, Daily or Monthly. (Enabling hourly reports does not mean that a new report is generated every hour. It means that data in the report is aggregated with a granularity of one hour.) |
S3BucketName | The name for the bucket which will contain the report files. |
Add your stack values to the configuration template
If you are creating a new stack:
- In step 1 of the Create stack process, select Template is ready.
- Select Upload a template file.
If you are editing an existing stack:
Select the stack.
Select Update.
Select Edit template in designer.
Paste the content of the yaml file into the Resources section of the template as follows:
"MyModule": {
"Type": "Logzio::awsCostAndUsage::cur::MODULE",
"Properties": {
"S3BucketName": "S3BucketName",
"ReportName": "ReportName",
"ReportPrefix": "ReportPrefix",
"ReportTimeUnit": "ReportTimeUnit",
"ReportAdditionalSchemaElements": "ReportAdditionalSchemaElements",
"LogzioURL": "LogzioURL",
"LogzioToken": "<<LOG-SHIPPING-TOKEN>>",
"LambdaMemorySize": "LambdaMemorySize",
"LambdaTimeout": "LambdaTimeout",
"CloudWatchEventScheduleExpression": "CloudWatchEventScheduleExpression"
}
}If required, change the module name by editing the
"MyModule"
value.