Using Log4j With Logstash

Logstash Log4j
Better Stack Team
Updated on October 26, 2024

Integrating Log4j with Logstash

Log4j and Logstash together enable centralized logging for Java applications, helping with real-time log analysis, troubleshooting, and monitoring. Here's a concise guide to get you started:

Prerequisites:

  • A Java application using Log4j or Log4j2.
  • Logstash installed and running.
  • (Optional) Elasticsearch and Kibana for storing and visualizing logs.

Integration Methods:

  1. Log4j1 with Logstash (Deprecated):

    • Use the logstash-log4j appender to send logs over TCP or UDP.
    • Add dependencies and configure Log4j to forward logs to Logstash.
     
    log4j.appender.LOGSTASH=net.logstash.log4j.LogstashTcpSocketAppender
log4j.appender.LOGSTASH.remoteHost=localhost
log4j.appender.LOGSTASH.port=5000

  2. Log4j2 with Logstash:

    • For Log4j2, use the logstash-log4j2 appender.
    • Update your log4j2.xml configuration to send logs over TCP using JSON layout.
     
    <Logstash name="Logstash" host="localhost" port="5000">
  <JacksonJsonLayout/>
</Logstash>

  3. Filebeat for Scalability:

    • Write logs to a file using Log4j and have Filebeat forward them to Logstash.
    • Configure Log4j2 to log to a file, then configure Filebeat to ship the logs.
     
    filebeat.inputs:
  - type: log
    paths: ["/var/log/myapp/*.log"]

Logstash Configuration:

In your Logstash configuration, use a TCP input to receive the logs and forward them to Elasticsearch or other outputs.

 
input {
  tcp { port => 5000 codec => json }
}
output {
  elasticsearch { hosts => ["localhost:9200"] }
}

Security:

  • Use TLS/SSL to encrypt logs during transmission.
  • Ensure you don't log sensitive data like passwords.

Conclusion:

By integrating Log4j with Logstash, you centralize log management, enabling better monitoring, troubleshooting, and data analysis for your Java applications.

Got an article suggestion? Let us know
Explore more
Logstash Log4j
Licensed under CC-BY-NC-SA

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.