Logz.io Python Handler sends logs in bulk over HTTPS to Logz.io. Logs are grouped into bulks based on their size.

If the main thread quits, the handler tries to consume the remaining logs and then exits. If the handler can’t send the remaining logs, they’re written to the local file system for later retrieval.

Set up Logz.io Python Handler

Supported versions: Python 3.5 or newer.

Add the dependency to your project

Navigate to your project’s folder in the command line, and run this command to install the dependency.

pip install logzio-python-handler
Configure Logz.io Python Handler for a standard Python project

Use the samples in the code block below as a starting point, and replace the sample with a configuration that matches your needs.

For a complete list of options, see the configuration parameters below the code block.👇

[handlers]
keys=LogzioHandler

[handler_LogzioHandler]
class=logzio.handler.LogzioHandler
formatter=logzioFormat

# Parameters must be set in order. Replace these parameters with your configuration.
args=('<<LOG-SHIPPING-TOKEN>>', '<<LOG-TYPE>>', <<TIMEOUT>>, 'https://<<LISTENER-HOST>>:8071', <<DEBUG-FLAG>>)

[formatters]
keys=logzioFormat

[loggers]
keys=root

[logger_root]
handlers=LogzioHandler
level=INFO

[formatter_logzioFormat]
format={"additional_field": "value"}
Dict Config

This is an alternative configuration option recommended if you are using Python 3.8.

See Python’s documentation regarding the logging.config.dictConfig method.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'logzioFormat': {
            'format': '{"additional_field": "value"}',
            'validate': False
        }
    },
    'handlers': {
        'logzio': {
            'class': 'logzio.handler.LogzioHandler',
            'level': 'INFO',
            'formatter': 'logzioFormat',
            'token': '<<LOG-SHIPPING-TOKEN>>',
            'logzio_type': '<<LOG-TYPE>>',
            'logs_drain_timeout': 5,
            'url': 'https://<<LISTENER-HOST>>:8071'
        }
    },
    'loggers': {
        '': {
            'level': 'DEBUG',
            'handlers': ['logzio'],
            'propagate': True
        }
    }
}

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to.

  • Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.

Parameters

Order matters. The arguments must be configured in the order shown here. For example, to set debug-flag to true, you need to set every argument that comes before it.

Parameter Description Required/Default
token Your Logz.io account token. Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to. Required
logzio_type The log type, shipped as type field. Used by Logz.io for consistent parsing. Can’t contain spaces. python
timeout Time to wait between log draining attempts, in seconds. 3
url Listener URL and port. Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. https://listener.logz.io:8071
debug Debug flag. To print debug messages to stdout, True. Otherwise, False. False
backup-logs If set to False, disables the local backup of logs in case of failure. True
network-timeout Timeout in seconds, int or float, for connecting to Logz.io. 10
logs_drain_timeout Timeout in seconds, int or float, for sending the logs to Logz.io. 5
add_context Set to True if you’re using OpenTelemetry instrumentation and wish to inject trace context into your logs. For more info, see the Trace context section. False

Serverless platforms

If you’re using a serverless function, you’ll need to import and add the LogzioFlusher annotation before your sender function. To do this, in the code sample below, uncomment the import statement and the @LogzioFlusher(logger) annotation line.

For the LogzioFlusher to work properly, you’ll need to make sure that the Logz.io. handler is added to the root logger. See the configuration above for an example.

Code Example

import logging
import logging.config
# If you're using a serverless function, uncomment.
# from logzio.flusher import LogzioFlusher

# Say I have saved my configuration as a dictionary in a variable named 'LOGGING' - see 'Dict Config' sample section
logging.config.dictConfig(LOGGING)
logger = logging.getLogger('superAwesomeLogzioLogger')

# If you're using a serverless function, uncomment.
# @LogzioFlusher(logger)
def my_func():
    logger.info('Test log')
    logger.warn('Warning')

    try:
        1/0
    except:
        logger.exception("Supporting exceptions too!")

To add dynamic metadata to your logger other than the constant metadata from the formatter, you can use the extra parameter. Key-value pairs passed in extra are shown as new fields in Logz.io. You can’t override default fields from the python logger, such as lineno or thread.

logger.info('Warning', extra={'extra_key':'extra_value'})

Trace context

If you’re sending traces with OpenTelemetry instrumentation (auto or manual), you can correlate your logs with the trace context. In this way, your logs will have traces data in it, such as service name, span id and trace id. To enable this feature, set the add_context param in your handler configuration to True, like in this example:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'logzioFormat': {
            'format': '{"additional_field": "value"}',
            'validate': False
        }
    },
    'handlers': {
        'logzio': {
            'class': 'logzio.handler.LogzioHandler',
            'level': 'INFO',
            'formatter': 'logzioFormat',
            'token': '<<LOG-SHIPPING-TOKEN>>',
            'logzio_type': 'python-handler',
            'logs_drain_timeout': 5,
            'url': 'https://<<LISTENER-HOST>>:8071',
            'retries_no': 4,
            'retry_timeout': 2,
            'add_context': True
        }
    },
    'loggers': {
        '': {
            'level': 'DEBUG',
            'handlers': ['logzio'],
            'propagate': True
        }
    }
}

Replace the placeholders to match your specifics. (They are indicated by the double angle brackets << >>):

  • Replace <<LOG-SHIPPING-TOKEN>> with the token of the account you want to ship to.

  • Replace <<LISTENER-HOST>> with the host for your region. For example, listener.logz.io if your account is hosted on AWS US East, or listener-nl.logz.io if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071.

Note that this feature is only available from version 4.0.0.

For troubleshooting this solution, see our Python troubleshooting guide.