How to configure Log4j for log compression

Configuring Log4j for Log Compression in New Versions

Introduction

In the world of software development, managing and storing logs is of utmost importance. The Log4j framework is one of the most popular logging libraries in Java, providing capabilities for storing, formatting, and even compressing logs. In this article from Radib, we will explain in detail how to configure Log4j 2 for log compression. We will also provide practical examples to show you how to automatically compress your log files and avoid excessive disk space usage.

1. Why Should We Use Log Compression?

In large projects and high-traffic services, logs grow rapidly and occupy significant disk space. Some benefits of log compression include:

Buy a virtual server at the best price from Radib, click here

• Reducing Log File Size: Text files usually have a large size, but compression can reduce their size by up to 90%.
• Optimizing Storage Management: It reduces the need for disk space and improves system performance.
• Faster Log Transfer and Processing: Compressed files are transferred and processed faster.

2. Introduction to RollingFileAppender in Log4j 2

To manage log size in Log4j 2, RollingFileAppender is used. This Appender allows log files to be archived and automatically compressed after reaching a specific size or time period.

Note: Common compression formats such as ZIP and GZ are supported in Log4j.

3. Log4j 2 Configuration for Log Compression

To compress logs, you need to configure RollingFileAppender in the log4j2.xml file. Below is a complete configuration example:

Register your domain instantly with three simple clicks at Radib, click here

Sample Configuration in log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
    <Appenders>
        <!-- Define RollingFileAppender for managing and compressing logs -->
        <RollingFile name="RollingFileAppender"
                     fileName="logs/app.log"
                     filePattern="logs/archive/app-%d{yyyy-MM-dd}.log.gz">
            <PatternLayout>
                <Pattern>%d{yyyy-MM-dd HH:mm:ss} [%t] %-5level %logger{36} - %msg%n</Pattern>
            </PatternLayout>
            <Policies>
                <!-- Compress logs after reaching 10MB -->
                <SizeBasedTriggeringPolicy size="10MB"/>
                <!-- Compress logs daily -->
                <TimeBasedTriggeringPolicy interval="1" modulate="true"/>
            </Policies>
        </RollingFile>
    </Appenders>

    <Loggers>
        <Root level="info">
            <AppenderRef ref="RollingFileAppender"/>
        </Root>
    </Loggers>
</Configuration>
XML
Copy

Configuration Explanation:

1. RollingFileAppender: This Appender creates the main log file and archives/compresses it when necessary.
2. filePattern: Specifies that archived files should be saved in the format app-YYYY-MM-DD.log.gz.
3. SizeBasedTriggeringPolicy: Determines that logs should be archived after reaching 10MB.
4. TimeBasedTriggeringPolicy: Specifies that a new log file should be created daily, and the previous file should be archived.

4. Testing the Configuration with a Java Program

Now that the configuration is done, let’s run a Java program to generate logs and test the compression:

Buy cPanel and DirectAdmin hosting at the best price from Radib, click here

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class LogCompressionTest {
    private static final Logger logger = LogManager.getLogger(LogCompressionTest.class);

    public static void main(String[] args) {
        for (int i = 0; i < 10000; i++) {
            logger.info("This is a test log number " + i + ".");
        }
        System.out.println("Logs have been generated. Please check the logs folder.");
    }
}
Java
Copy

Execution Steps:

1. Run the above code in a Java project.
2. Observe that the logs/app.log file is created and data is written to it.
3. After the logs reach 10MB or at the end of the day, a new log file is created, and the previous file is compressed in GZ format.

5. Advanced Settings (Optional)

5.1. Compression in ZIP Format

If you want to use ZIP instead of GZ, you can change the filePattern value:

filePattern="logs/archive/app-%d{yyyy-MM-dd}.zip"
XML
Copy

5.2. Automatic Deletion of Old Logs

If you want logs older than 30 days to be automatically deleted, you can use DefaultRolloverStrategy:

<DefaultRolloverStrategy max="30"/>
XML
Copy

6. Conclusion

In this article, we thoroughly explored how to configure Log4j 2 for log compression. You learned how to set up RollingFileAppender to automatically compress and archive log files. We also provided several examples of XML configuration and a Java program to help you better understand these concepts.

Our Recommendation:

If your project generates a large number of logs, be sure to use log compression to optimize storage space.
You can also add advanced policies such as automatic deletion of old logs.

Have a question or comment? Share it with our experts in the Radib ticket system! Get detailed guidance from Radib.