Deploy this integration to forward logs from your Azure Blob Storage account to Logz.io using an automated deployment process via the trigger function. Each new log in the container path inside the storage account (including sub directories), will trigger the Logz.io function that will ship the file content to Logz.io.

Architecture overview

The following resources are needed for this integration:

  • Storage Account (general purpose v2) + Container
  • App Service Plan - Consumption Plan
  • Application Insights
  • Logz.io Function App + Logz.io Blob Trigger Function
  • Storage Account for Logz.io Function App logs

Integration-architecture

Supported data types

This Logz.io function supports the following data types:

  • Json
  • CSV (supported delimiters: ',' ';' '|')
  • Text (supports multiline text - MultilineRegex parameter)

The file name does not have to be with these extensions.

Supported file formats

The Logz.io function supports the following file formats:

  • Gzip

The file name does not have to be with these extensions.

Create a new blob storage account

If you don’t have a general purpose v2 storage account with a container for logs, or you want to create everything from scratch, this auto-deployment is for you.

  • Storage Account (general purpose v2) + Container
  • App Service Plan - Consumption Plan
  • Application Insights
  • Logz.io Function App and Logz.io Blob Trigger Function
  • Storage Account for Logz.io Function App logs
Launch an automated deployment

👇 Click this button:

Deploy to Azure

Fill in the deployment parameters

In the Custom deployment screen, fill in all the parameters as per table below and click Review + create.

Custom deployment

Parameter Description
Storage Account Name The storage account (general purpose v2) name.
Container Name The name of the container inside the storage account
Logs Path The path from where blob files will trigger the Logz.io function (including sub-directories in that path). Leave empty if you want every blob file in the container to trigger the Logz.io function.
Logzio URL Use the listener URL specific to the region where your Logz.io account is hosted. Click to look up your listener URL.
Logzio Token Your Logz.io log shipping token directs the data securely to your Logz.io Log Management account. The default token is auto-populated in the examples when you’re logged into the Logz.io app as an Admin. Manage your tokens.
Multiline Regex The regex that matches the multiline logs in text blob files. Leave empty if you do not use multiline logs in your text blob files.
FilterDate Every log with a date greater than or equal to this date will be shipped to Logz.io. For it to take effect, FilterDateJson must not be empty. The date must be in the same format as the date in the logs. Leave empty if you want all logs to be shipped to Logz.io.
FilterDateJsonPath The json path of the date inside each log (for example: metadata.datetime). If this value cannot be found inside a log, the log will be shipped to Logz.io. Leave empty if you are not using FilterDate.
Confirm the deployment parameters

In the Custom deployment: review + create screen, review the deployment and click Create.

Custom deployment

If all the parameters have been configured correctly, the following conformation screen will appear:

Custom deployment

Click Go to resource group to go to your resource group with all the created resources.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana. You can filter for logs of type blobStorage to see the incoming logs.

If you still don’t see your logs, see log shipping troubleshooting.

Connect to existing blob storage account

Before you begin, you’ll need: a blob storage account of the type StorageV2 (general purpose v2).

If your existing blob storage account is of any other kind, it will NOT work. Instead, follow the process to set up a new blob storage account.

Check your storage account for compatibility

Double-check your Storage accounts to make sure that they are compatible with this integration. They should be of the type StorageV2 (general purpose v2).

Launch an automated deployment

👇 Click this button:

Deploy to Azure

Fill in the deployment parameters

In the Custom deployment screen, fill in all the parameters as per table below and click Review + create.

Custom deployment

Parameter Description
Storage Account Name The storage account (general purpose v2) name.
Storage Account Resource Name The resource name that contains the storage account. (Needed only in Logz.io Function Auto-Deployment)
Container Name The name of the container inside the storage account
Logs Path The path from where blob files will trigger the Logz.io function (including sub-directories in that path). Leave empty if you want every blob file in the container to trigger the Logz.io function.
Logzio URL Use the listener URL specific to the region where your Logz.io account is hosted. Click to look up your listener URL.
Logzio Token Your Logz.io log shipping token directs the data securely to your Logz.io Log Management account. The default token is auto-populated in the examples when you’re logged into the Logz.io app as an Admin. Manage your tokens.
Multiline Regex The regex that matches the multiline logs in text blob files. Leave empty if you do not use multiline logs in your text blob files.
FilterDate Every log with a date greater than or equal to this date will be shipped to Logz.io. For it to take effect, FilterDateJson must not be empty. The date must be in the same format as the date in the logs. Leave empty if you want all logs to be shipped to Logz.io.
FilterDateJsonPath The json path of the date inside each log (for example: metadata.datetime). If this value cannot be found inside a log, the log will be shipped to Logz.io. Leave empty if you are not using FilterDate.

Logs that were in the container before the deployment will be shipped to Logz.io. If these logs have already been shipped to Logz.io, we recommend that you empty the container before the deployment or use the FilterDate and FilterDateJsonPath parameters.

Confirm the deployment parameters

In the Custom deployment: review + create screen, review the deployment and click Create.

Custom deployment

If all the parameters have been configured correctly, the following conformation screen will appear:

Custom deployment

Click Go to resource group to go to your resource group with all the created resources.

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana. You can filter for logs of type azure_blob_trigger to see the incoming logs.

If you still don’t see your logs, see log shipping troubleshooting.