If your account is nearing its daily quota, Logz.io sends an email alert to account administrators. That’s fine for most situations, but what if you want to be more proactive in managing your accounts?
For this, you have account utilization metrics (which saves log size and other metadata) and drop filters (which allow you to stop ingestion of some logs).
What are account utilization metrics?
Account utilization metrics capture a snapshot
of your account usage in regular increments—every 10, 30, or 60 minutes—depending on your settings.
Logz.io starts logging your usage after your account reaches 10 MB in size.
You can find these logs in Kibana
when you filter for the logzio_account_utilization log type.
| Field name | Description |
|---|---|
| account_id | Logz.io account ID |
| account_name | Name of this account |
| expected_utilization_EOD | Expected utilization by the end of the day, in percent |
| expected_volume_in_GB_EOD | The expected utilization by the end of the day, in GB |
| grace_capacity | The overage configured on the account, in percent. For instance, 120% means that you have an overage allowance of 20% of your plan volume. |
| number_of_fields | The total number of fields in the current index |
| plan_retention_in_days | The number of days data is kept in the account until it is deleted |
| plan_volume_in_GB | Data allocated to this account, in GB |
| utilization | Current utilization, in percent |
| volume_in_GB | Current utilization, in GB |
What happens when I save log size?
When you enable saving log size,
a new field is added to incoming logs.
This new field is called LogSize,
and it contains the size of the log line in bytes, taken as a single string.
Kibana doesn’t recognize LogSize as a number right away.
You can fix this by clicking (refresh mapping) for LogSize in Kibana.
You’ll need to do this for each account where you enabled LogSize.
Enabling account utilization metrics and log size
Account utilization metrics and log size are set per individual account. You can find these settings on the Manage accounts page below Advanced options. This page is available only from your main account—you won’t be able to make this change from a sub account.
Even though you need to be an account admin to enable this setting, everyone who has access to your Logz.io account can see the resulting logs.
Visualizing utilization with an ELK app
We offer a preconfigured dashboard for account utilization metrics. You can find this dashboard in the ELK Apps by searching for “data volume dashboard”.
If you recently enabled account utilization metrics, you may need to wait up to 60 minutes before this data shows up in Kibana.
Managing ingestion with drop filters
We recommend shipping only the logs you need so that you don’t end up paying for unnecessary data retention.
However, if you’re sending logs that you’ll need only sometimes (such as debug logs), you can set up a drop filter.
Logs caught by drop filters aren’t ingested, so they won’t count against your plan limits. However, the volume of logs you can filter is based on your plan’s quota. For more information, see Some important notes on drop filtering.
Dropped logs will never arrive at your Logz.io account, so they can’t be searched or trigger alerts. Always confirm you’re dropping the right logs when you apply a new filter.