Log4J Custom Memory Appender

Hi all,

This post will describe how to write a custom appenders for Log4J, and about the Memory Appender, which I implemented.

Log4j is a project of Apache Software Foundation, which provides Java based logging Utility. It is quite efficient do logging with log4j, than having common System.out.println s all around your code.

Ok, lets get back to our custom appender.

Writing a custom appender for log4j is pretty easy. All what you have to do is to extend the “AppenderSkeleton” class in Log4j.

package org.adroitlogic.ultraesb.core.helper.memoryappender;

import org.adroitlogic.ultraesb.Util;
import org.adroitlogic.ultraesb.jmx.JMXConstants;
import org.adroitlogic.ultraesb.jmx.core.LogManagementMXBean;
import org.adroitlogic.ultraesb.jmx.view.LogEntry;
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.helpers.LogLog;
import org.apache.log4j.spi.LoggingEvent;

import java.util.List;

/**
* InMemoryAppender appends log messages to a LogList, and implements methods in the MXBean interface
*/

public class InMemoryAppender extends AppenderSkeleton implements LogManagementMXBean {

private volatile long id = 0;
private final LogList logList = new LogList();

public synchronized void close() {
    if (this.closed) {
    return;
}
    this.closed = true;
}

public boolean requiresLayout() {
    return true;
}

protected boolean checkEntryConditions() {
    if (this.closed) {
        LogLog.warn("Not allowed to write to a closed appender.");
        return false;
    }

    if (this.layout == null) {
        errorHandler.error("No layout set for the appender named [" + name + "].");
        return false;
    }
    return true;
}

public void setSize(int size) {
    logList.setSize(size);
}

public void append(LoggingEvent event) {
    if (!checkEntryConditions()) {
       return;
    }
    subAppend(event);
}

protected void subAppend(LoggingEvent event) {
    int index = event.getLoggerName().lastIndexOf('.');
    String loggerName;

    if (index > -1) {
        loggerName = event.getLoggerName().substring(index + 1);
    } else {
        loggerName = event.getLoggerName();
    }

    LogEntry log = new LogEntry();
    log.setId(id++);
    log.setHost(event.getProperty("host"));
    log.setIp(event.getProperty("ip"));
    log.setLoggerName(loggerName);
    log.setMessage((String) event.getMessage());
    log.setThreadName(event.getThreadName());
    log.setTimeStamp(event.getTimeStamp());
    log.setLogLevel(event.getLevel().toString());
    logList.insert(log);
}

}

The method void close(), boolean requiresLayout() and void append(LoggingEvent event) are extended from  AppendercSkeleton class. Out of these, most important method is void append(LoggingEvent event) method. Each time a log message occurs, it is passed to this method as a LoggingEvent. Once  the LoggingEvent  is captured, you are almost done with writing the custom appender.

Then the LogginEvent is passed to  “subAppend(LogginEvent event)”. In there,  the “LoggingEvent” can be processed according to one’s requirements.

My requirement was to retrieve the tail end of UltraESB log messages and to display them on the web management console.

The following class defines the data structures I used and its insertion and retrieval methods.

package org.adroitlogic.ultraesb.core.helper.memoryappender;

import org.adroitlogic.ultraesb.jmx.view.LogEntry;

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.*;

public class LogList {

/**
* Specifies the number of log messages appended. Is incremented each time, a log is added to the linked list
*/
private int usedSize = 0;
/**
* Size of the hash map
*/
private int capacity;
private Map map;
private LinkedList linkedList = new LinkedList();
private static final DateFormat dfm = new SimpleDateFormat("yyyy-MM-dd' 'HH:mm:ss");

public LogList() {
}

public void setSize(int hashMapSize) {
    this.capacity = hashMapSize;
    map = new HashMap(hashMapSize);
}

/**
* Inserts log messages to the linked list and the hash map.
* If the number of log messages exceed the hashMapSize, new logs will be added after removing, the oldest log from the linked list and the hash map.
* @param log the log entries
*/
public synchronized void insert(LogEntry log) {
    if (usedSize > capacity) {
        map.remove(linkedList.removeFirst().getId());
        map.put(log.getId(), log);
        linkedList.add(log);
    } else {
        linkedList.add(log);
        map.put(log.getId(), log);
        usedSize++;
    }
}

public List formatFields(List list) {
int i = 0;
LogEntry entry;
String formattedString;

while (i < list.size()) {
    entry = list.get(i);
    if (entry.getFormattedTime() == null) {
        synchronized (dfm) {
        formattedString = dfm.format(entry.getTimeStamp());
        entry.setFormattedTime(formattedString);
        }
    }

    formattedString = entry.getMessage().replaceAll("( ){2}?", "  ");
    entry.setMessage(formattedString);
    i++;
}
return list;
}

/**
* When called returns the entire list of logs
* @return linkedList the list containing the logs
*/
public List getAll() {
    if (linkedList != null) {
        return formatFields(linkedList);
    } else {
    return Collections.emptyList();
    }
}

/**
* When called returns a sub list of the logs starting from the specified log id
* @param start starting id
* @return list the list containing the sub list
*/
public List getListFrom(long start) {
    if (map.get(start) != null) {
        List list = new ArrayList();
        long i = start;
        while (map.get(i) != null) {
            list.add(map.get(i));
            i++;
        }
        return formatFields(list);
    } else {
        return Collections.emptyList();
    }
}
}

I suppose that you guys noticed the “void setSize(int size)” method in “InMemoryappender” class. This is where I set the size of the LogList HashMap.
This method is directly called from the Log4j properties file and anyone can change the size of the hash map by editing this file. This is how it looks like.

log4j.rootCategory=ERROR, MEMORY_APPENDER

log4j.appender.MEMORY_APPENDER=org.adroitlogic.ultraesb.core.helper.memoryappender.InMemoryAppender
log4j.appender.MEMORY_APPENDER.Size=500
log4j.appender.MEMORY_APPENDER.layout=org.apache.log4j.PatternLayout

Now I’ll try to provide a more detailed view of the LogViewer. The “void insert(LogEntry log)” method is called by the subAppend method. Each log message is inserted to a LinkedList and a HashMap. The HashMap is used to retrieve a log message by its log id. An ESB can generate thousands of log messages, but users may be interested on reading the latest logs. So only the latest 500 (this number can be configured by the user as mentioned above) messages are kept in the memory and presented to the user.

JMX capabilities are used for the interactions between the console and the back end. Communication is done via JSON messages and “Apache Wink” is used to map Json to Java and vice versa.

When a user navigates to the LogViwer tab, the “List getAll()” method is invoked, displaying all existing log messages in the linked list. The console app checks for any latest log messages with an interval of 5 seconds. “List getListFrom(long start)” method is invoked for this purpose.

So that’s all about writing a custom appender for log4j. Hope it was useful…:)