AWS API Gateway

tip

For a much easier and more efficient way to collect and send telemetry, consider using the Logz.io telemetry collector.

Logs

Deploy this integration to forward your Amazon API Gateway logs to Logz.io using AWS Firehose

Auto-deploy the Stack in the relevant region

This integration will deploy a Firehose connection with your AWS services to forward logs to Logz.io To deploy this project, click the button that matches the region you wish to deploy your Stack to:

Region	Deployment
us-east-1
us-east-2
us-west-1
us-west-2
eu-central-1
eu-north-1
eu-west-1
eu-west-2
eu-west-3
sa-east-1
ap-northeast-1
ap-northeast-2
ap-northeast-3
ap-south-1
ap-southeast-1
ap-southeast-2
ca-central-1

Specify stack details

Specify the stack details as per the table below, check the checkboxes and select Create stack.

Parameter	Description	Required/Default
logzioToken	The token of the account you want to ship logs to.	Required
logzioListener	Listener host.	Required
logzioType	The log type you'll use with this Lambda. This can be a built-in log type, or a custom log type.	logzio_firehose
services	A comma-seperated list of services you want to collect logs from. Supported options are: apigateway, rds, cloudhsm, cloudtrail, codebuild, connect, elasticbeanstalk, ecs, eks, aws-glue, aws-iot, lambda, macie, amazon-mq.	-
customLogGroups	A comma-separated list of custom log groups to collect logs from, or the ARN of the Secret parameter (explanation below) storing the log groups list if it exceeds 4096 characters.	-
useCustomLogGroupsFromSecret	If you want to provide list of customLogGroups which exceeds 4096 characters, set to true and configure your customLogGroups as defined below.	false
triggerLambdaTimeout	The amount of seconds that Lambda allows a function to run before stopping it, for the trigger function.	60
triggerLambdaMemory	Trigger function's allocated CPU proportional to the memory configured, in MB.	512
triggerLambdaLogLevel	Log level for the Lambda function. Can be one of: debug, info, warn, error, fatal, panic	info
httpEndpointDestinationIntervalInSeconds	The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination	60
httpEndpointDestinationSizeInMBs	The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination	5

Important

AWS limits every log group to have up to 2 subscription filters. If your chosen log group already has 2 subscription filters, the trigger function won't be able to add another one.

Custom Log Group list exceeds 4096 characters limit

If your customLogGroups list exceeds the 4096 characters limit, follow the below steps:

1. Open AWS Secret Manager
2. Click Store a new secret
   • Choose Other type of secret
   • For key use logzioCustomLogGroups
   • In value store your comma-separated custom log groups list
   • Name your secret, for example as LogzioCustomLogGroups
   • Copy the new secret's ARN
3. In your stack, Set:
   • customLogGroups to your secret ARN that you copied in step 2
   • useCustomLogGroupsFromSecret to true

Send logs

Give the stack a few minutes to be deployed.

Once new logs are added to your chosen log group, they will be sent to your Logz.io account.

Important

If you've used the services field, you'll have to wait 6 minutes before creating new log groups for your chosen services. This is due to cold start and custom resource invocation, that can cause the Lambda to behave unexpectedly.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

If you still don't see your logs, see log shipping troubleshooting.

Metrics

note

For a much easier and more efficient way to collect and send metrics, consider using the Logz.io telemetry collector.

Deploy this integration to send your Amazon API Gateway metrics to Logz.io.

This integration creates a Kinesis Data Firehose delivery stream that links to your Amazon API Gateway metrics stream and then sends the metrics to your Logz.io account. It also creates a Lambda function that adds AWS namespaces to the metric stream, and a Lambda function that collects and ships the resources' tags.

Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your metrics.

To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.

Before you begin, you'll need:

• An active account with Logz.io

Configure AWS to forward metrics to Logz.io

Set the required minimum IAM permissions

Make sure you have configured the minimum required IAM permissions as follows:

• Amazon S3:
  • s3:CreateBucket
  • s3:DeleteBucket
  • s3:PutObject
  • s3:GetObject
  • s3:DeleteObject
  • s3:ListBucket
  • s3:AbortMultipartUpload
  • s3:GetBucketLocation
• AWS Lambda:
  • lambda:CreateFunction
  • lambda:DeleteFunction
  • lambda:InvokeFunction
  • lambda:GetFunction
  • lambda:UpdateFunctionCode
  • lambda:UpdateFunctionConfiguration
  • lambda:AddPermission
  • lambda:RemovePermission
  • lambda:ListFunctions
• Amazon CloudWatch:
  • cloudwatch:PutMetricData
  • cloudwatch:PutMetricStream
  • logs:CreateLogGroup
  • logs:CreateLogStream
  • logs:PutLogEvents
  • logs:DeleteLogGroup
  • logs:DeleteLogStream
• AWS Kinesis Firehose:
  • firehose:CreateDeliveryStream
  • firehose:DeleteDeliveryStream
  • firehose:PutRecord
  • firehose:PutRecordBatch
• IAM:
  • iam:PassRole
  • iam:CreateRole
  • iam:DeleteRole
  • iam:AttachRolePolicy
  • iam:DetachRolePolicy
  • iam:GetRole
  • iam:CreatePolicy
  • iam:DeletePolicy
  • iam:GetPolicy
• Amazon CloudFormation:
  • cloudformation:CreateStack
  • cloudformation:DeleteStack
  • cloudformation:UpdateStack
  • cloudformation:DescribeStacks
  • cloudformation:DescribeStackEvents
  • cloudformation:ListStackResources

Create Stack in the relevant region

To deploy this project, click the button that matches the region you wish to deploy your Stack to:

Region	Deployment
us-east-1
us-east-2
us-west-1
us-west-2
eu-central-1
eu-central-2
eu-north-1
eu-west-1
eu-west-2
eu-west-3
eu-south-1
eu-south-2
sa-east-1
ap-northeast-1
ap-northeast-2
ap-northeast-3
ap-south-1
ap-south-2
ap-southeast-1
ap-southeast-2
ap-southeast-3
ap-southeast-4
ap-east-1
ca-central-1
ca-west-1
af-south-1
me-south-1
me-central-1
il-central-1

Specify stack details

Specify the stack details as per the table below, check the checkboxes and select Create stack.

Parameter	Description	Required/Default
logzioListener	The Logz.io listener URL for your region. (For more details, see the regions page. For example - https://listener.logz.io:8053	Required
logzioToken	Your Logz.io metrics shipping token.	Required
awsNamespaces	Comma-separated list of the AWS namespaces you want to monitor. See this list of namespaces. If you want to automatically add all namespaces, use value all-namespaces.	At least one of awsNamespaces or customNamespace is required
customNamespace	A custom namespace for CloudWatch metrics. This is used to specify a namespace unique to your setup, separate from the standard AWS namespaces.	At least one of awsNamespaces or customNamespace is required
logzioDestination	Your Logz.io destination URL.	Required
httpEndpointDestinationIntervalInSeconds	The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination.	60
httpEndpointDestinationSizeInMBs	The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination.	5
debugMode	Enable debug mode for detailed logging (true/false).	false

Check Logz.io for your metrics

Give your data some time to get from your system to ours, then log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.

Log in to your Logz.io account and navigate to the current instructions page inside the Logz.io app. Install the pre-built dashboard to enhance the observability of your metrics.

To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.