Ship Java logs

Log4j 2
Logback

Log4j 2 setup

The Logz.io Log4j 2 appender sends logs using non-blocking threading, bulks, and HTTPS encryption to port 8071.

This appender uses LogzioSender. Logs queue in the buffer and are 100% non-blocking. A background task handles log shipping. To help manage dependencies, this .jar shades LogzioSender, BigQueue, Gson, and Guava.

You’ll need: Log4j 2.7 or higher, Java 8 or higher

Add the dependency to your project

Add a dependency to your project configuration file (for instance, pom.xml in a Maven project).

<dependencies>
  <dependency>
    <groupId>io.logz.log4j2</groupId>
    <artifactId>logzio-log4j2-appender</artifactId>
    <version>1.0.10</version>
  </dependency>
</dependencies>

Configure Log4j

Use the samples in the code block below as a starting point, and replace the sample with a configuration that matches your needs.

For a complete list of options, see the configuration parameters below the code block.👇

See the Log4j documentation for more information on the Log4j 2 configuration file.

<Appenders>

  <!-- Replace these parameters with your configuration -->
  <LogzioAppender name="Logzio">
    <logzioToken><<SHIPPING-TOKEN>></logzioToken>
    <logzioUrl>https://<<LISTENER-HOST>>:8071</logzioUrl>
    <logzioType>myAwesomeType</logzioType>
  </LogzioAppender>

</Appenders>

<Loggers>
  <Root level="info">
    <AppenderRef ref="Logzio"/>
  </Root>
</Loggers>

Parameters

logzioToken: Your Logz.io account token.
Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.
Begin with $ to use an environment variable or system property with the specified name. For example, $LOGZIO_TOKEN uses the LOGZIO_TOKEN environment variable.
logzioUrl https://listener.logz.io:8071: Listener URL and port.
Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.
logzioType java: The log type, shipped as type field. Used by Logz.io for consistent parsing. Can’t contain spaces.
addHostname false: Boolean. Indicates whether to add hostname field to logs. This field holds the machine’s host name.
Set to true to include hostname. Set to false to leave it off. If a host name can’t be found, this field is not added.
additionalFields: Adds fields to the JSON message output, formatted as field1=value1;field2=value2.
Use $ to inject an environment variable value, such as field2=$VAR_NAME. The environment variable should be the only value in the key-value pair. If the environment variable can’t be resolved, the field is omitted.
bufferDir System.getProperty("java.io.tmpdir"): Filepath where the appender stores the buffer
compressRequests false: Boolean. Set to true if you’re sending gzip-compressed logs. Set to false if sending uncompressed logs.
connectTimeoutMs 10 * 1000: Connection timeout during log shipment, in milliseconds.
debug false: Set to true to print debug messages to stdout.
drainTimeoutSec 5: How often the appender drains the buffer, in seconds.
fileSystemFullPercentThreshold 98: Identifies a maximum file system usage, in percent. Set to -1 to disable.
If the file system storage exceeds this threshold, the appender stops buffering and drops all new logs. Buffering resumes if used space drops below the threshold.
socketTimeoutMs 10 * 1000: Socket timeout during log shipment, in milliseconds.

Code sample

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class LogzioLog4j2Example {
  public static void main(String[] args) {
    Logger logger = LogManager.getLogger(LogzioLog4j2Example.class);

    logger.info("Testing logz.io!");
    logger.warn("Winter is coming");
  }
}

MDC

You can add mapped diagnostic context (MDC) to your logs. Each key-value pair you define is added to every log line while the thread is alive.

So this code sample...

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.ThreadContext;

public class LogzioLog4j2Example {
  public static void main(String[] args) {
    Logger logger = LogManager.getLogger(LogzioLog4j2Example.class);
    ThreadContext.put("Key", "Value");
    logger.info("This log will hold the MDC data as well");
  }
}

...produces this log output.

{
  "message": "This log will hold the MDC data as well",
  "Key": "Value",
  "Your log message follows": "..."
}

Markers

Markers are values you can use to tag and enrich log statements.

This code...

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.Marker;
import org.apache.logging.log4j.MarkerManager;

public class LogzioLog4j2Example {
  public static void main(String[] args) {
    Logger logger = LogManager.getLogger(LogzioLog4j2Example.class);
    Marker marker = MarkerManager.getMarker("Fatal");
    logger.error(marker, "This line has a fatal error");
  }
}

...produces this log output.

{
  "message": "This line has a fatal error",
  "Marker": "Fatal",
  "Your log message follows": "..."
}

Logback setup

Logback sends logs to your Logz.io account using non-blocking threading, bulks, and HTTPS encryption to port 8071.

This appender uses BigQueue implementation of persistent queue, so all logs are backed up to a local file system before being sent. Once you send a log, it will be enqueued in the buffer and 100% non-blocking. A background task handles the log shipment. To help manage dependencies, this .jar shades BigQueue, Gson, and Guava.

You’ll need: Logback 1.1.7 or higher, Java 8 or higher

Add the dependency to your project

Add a dependency to your project configuration file (for instance, pom.xml in a Maven project).

<dependencies>
  <dependency>
    <groupId>io.logz.logback</groupId>
    <artifactId>logzio-logback-appender</artifactId>
    <version>1.0.20</version>
  </dependency>
</dependencies>

Configure Logback

Use the samples in the code block below as a starting point, and replace the sample with a configuration that matches your needs.

For a complete list of options, see the configuration parameters below the code block.👇

See the Logback documentation for more information on the Logback configuration file.

<configuration>
  <!-- Closes gracefully and finishes the log drain -->
  <shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>

  <appender name="LogzioLogbackAppender" class="io.logz.logback.LogzioLogbackAppender">
    <!-- Replace these parameters with your configuration -->
    <token><<SHIPPING-TOKEN>></token>
    <logzioUrl><<LISTENER-HOST>>:8071</logzioUrl>
    <logzioType>myType</logzioType>

    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>INFO</level>
    </filter>
  </appender>

  <root level="debug">
    <!-- IMPORTANT: This line is required -->
    <appender-ref ref="LogzioLogbackAppender"/>
  </root>
</configuration>

Parameters

token: Your Logz.io account token.
Replace <<SHIPPING-TOKEN>> with the token of the account you want to ship to.
Begin with $ to use an environment variable or system property with the specified name. For example, $LOGZIO_TOKEN uses the LOGZIO_TOKEN environment variable.
logzioUrl https://listener.logz.io:8071: Listener URL and port.
Replace <<LISTENER-HOST>> with your region’s listener host (for example, listener.logz.io). For more information on finding your account’s region, see Account region.
logzioType java: The log type, shipped as type field. Used by Logz.io for consistent parsing. Can’t contain spaces.
addHostname false: Indicates whether to add hostname field to logs. This field holds the machine’s host name.
Set to true to include hostname. Set to false to leave it off. If a host name can’t be found, this field is not added.
additionalFields: Adds fields to the JSON message output, formatted as field1=value1;field2=value2.
Use $ to inject an environment variable value, such as field2=$VAR_NAME. The environment variable should be the only value in the key-value pair. If the environment variable can’t be resolved, the field is omitted.
bufferDir System.getProperty("java.io.tmpdir"): Filepath where the appender stores the buffer.
compressRequests false: Boolean. Set to true if you’re sending gzip-compressed logs. Set to false if sending uncompressed logs.
connectTimeout 10 * 1000: Connection timeout during log shipment, in milliseconds.
debug false: Boolean. Set to true to print debug messages to stdout.
drainTimeoutSec 5: How often the appender drains the buffer, in seconds.
fileSystemFullPercentThreshold 98: Integer. Identifies a maximum file system usage, in percent. Set to -1 to disable.
If the file system storage exceeds this threshold, the appender stops buffering and drops all new logs. Buffering resumes if used space drops below the threshold.
format text: Set to json if the log message is to be sent as JSON, so that each JSON node is a field in Logz.io. Set to text to send the log message as plain text.
line false: Boolean. Set to true to print the line number of the code that generated this log message. Set to false to leave the line number out.
socketTimeout 10 * 1000: Socket timeout during log shipment, in milliseconds.

Code sample

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LogzioLogbackExample {
  public static void main(String[] args) {
    Logger logger = LoggerFactory.getLogger(LogzioLogbackExample.class);

      logger.info("Testing logz.io!");
      logger.warn("Winter is coming");
  }
}

MDC

You can add mapped diagnostic context (MDC) to your logs. Each key-value pair you define is added to every log line while the thread is alive.

So this code sample...

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;

public class LogzioLogbackExample {
  public static void main(String[] args) {
    Logger logger = LoggerFactory.getLogger(LogzioLogbackExample.class);

    MDC.put("Key", "Value");
    logger.info("This log will hold the MDC data as well");
  }
}

...produces this log output.

{
  "message": "This log will hold the MDC data as well",
  "Key": "Value",
  "Your log message follows": "..."
}

Markers

Markers are values you can use to tag and enrich log statements.

This code...

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.Marker;

public class LogzioLogbackExample {

  public static void main(String[] args) {
    Logger logger = LoggerFactory.getLogger(LogzioLogbackExample.class);

    Marker marker = MarkerFactory.getMarker("Fatal");
    logger.error(marker, "This line has a fatal error");
  }
}

...produces this log output.

{
  "message": "This line has a fatal error",
  "Marker": "Fatal",
  "Your log message follows": "..."
}