The logzio-k8s image comes pre-configured for Fluentd to gather all logs from the Kubernetes node environment and append the proper metadata to the logs.

A DaemonSet ensures that some or all Kubernetes nodes run a copy of a pod. This implementation uses a Fluentd DaemonSet to collect Kubernetes logs and send them to Logz.io. Fluentd is a great option because it is flexible enough and has the right plugins to distribute logs to Logz.io and other third-party apps.

For most environments, deploying logzio-k8s with the default configuration is recommended. If your environment requires a custom configuration, follow the steps for deploying a custom configuration.

To deploy logzio-k8s

Store your Logz.io credentials

Save your Logz.io shipping credentials as a Kubernetes secret.

Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.

Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.

kubectl create secret generic logzio-logs-secret \
--from-literal=logzio-log-shipping-token='<<SHIPPING-TOKEN>>' \
--from-literal=logzio-log-listener='https://<<LISTENER-HOST>>:8071' \
-n kube-system
Deploy the DaemonSet

Run:

kubectl apply -f https://raw.githubusercontent.com/logzio/logzio-k8s/master/logzio-daemonset-containerd.yaml
Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana.

If you still don’t see your logs, see log shipping troubleshooting.

You can deploy logzio-k8s with a custom configuration by customizing the Fluentd container configuration. This is done using a ConfigMap that overwrites the default DaemonSet.

To deploy logzio-k8s

Store your Logz.io credentials

Save your Logz.io shipping credentials as a Kubernetes secret.

Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.

Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.

kubectl create secret generic logzio-logs-secret \
--from-literal=logzio-log-shipping-token='<<SHIPPING-TOKEN>>' \
--from-literal=logzio-log-listener='https://<<LISTENER-HOST>>:8071' \
-n kube-system
Configure Fluentd

Download Logz.io’s Containerd Daemonset.

Open the file in your text editor and customize the integration environment variables. The available parameters and their defaults are shown below.

Parameters

Parameter Description
output_include_time Default: true
To append a timestamp to your logs when they’re processed, true. Otherwise, false.
LOGZIO_BUFFER_TYPE Default: file
Specifies which plugin to use as the backend.
LOGZIO_BUFFER_PATH Default: /var/log/Fluentd-buffers/stackdriver.buffer
Path of the buffer.
LOGZIO_OVERFLOW_ACTION Default: block
Controls the behavior when the queue becomes full.
LOGZIO_CHUNK_LIMIT_SIZE Default: 2M
Maximum size of a chunk allowed
LOGZIO_QUEUE_LIMIT_LENGTH Default: 6
Maximum length of the output queue.
LOGZIO_FLUSH_INTERVAL Default: 5s
Interval, in seconds, to wait before invoking the next buffer flush.
LOGZIO_RETRY_MAX_INTERVAL Default: 30s
Maximum interval, in seconds, to wait between retries.
LOGZIO_FLUSH_THREAD_COUNT Default: 2
Number of threads to flush the buffer.
LOGZIO_LOG_LEVEL Default: info
The log level for this container.
Deploy the DaemonSet

Run:

kubectl apply -f /path/to/logzio-daemonset-containerd.yaml
Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Kibana.

If you still don’t see your logs, see log shipping troubleshooting.

Disabling systemd input

To suppress Fluentd system messages, set the environment variable FLUENTD_SYSTEMD_CONF to disable in your Kubernetes environment.

Disable Prometheus input plugins

By default, the latest images launch prometheus plugins to monitor Fluentd. If you’d like to disable the Prometheus input plugin, set the environment variable FLUENTD_PROMETHEUS_CONF to disable in your Kubernetes configuration.