Hi, I'm doing some testing on how to use Splunk the best possible way. I have fallen in love with the method of using the Rest API and log4j, since it doesn't require any forwarder or anything to be installed. Anyway, I have written a test application. It consists of a class and a log4j-config file:
AutoLogger:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import org.slf4j.*;
import com.splunk.logging.SplunkLogEvent;
public class AutoLogger {
public static void main(String[] args) throws IOException {
Logger logger = LoggerFactory.getLogger("splunk.logger");
InputStreamReader converter = new InputStreamReader(System.in);
BufferedReader reader = new BufferedReader(converter);
String in = "";
while(!in.equals("exit")) {
System.out.println("Log something:");
in = reader.readLine();
SplunkLogEvent event = new SplunkLogEvent("User logged something", "user_log");
event.addPair("input", in);
logger.error(event.toString());
}
}
}
log4j.properties: (stolen from https://github.com/splunk/splunk-library-javalogging)
#Example log4j configuration using a Splunk REST Appender or Splunk Raw TCP Appender
# Root logger option
log4j.rootLogger=DEBUG,stdout
#Splunk logger
log4j.logger.splunk.logger=INFO, splunkrest
#log4j.logger.splunk.logger=INFO, splunkrawtcp
log4j.additivity.splunk.logger=false
# Send log events to a Splunk REST endpoint
log4j.appender.splunkrest=com.splunk.logging.log4j.appender.SplunkRestAppender
log4j.appender.splunkrest.user=admin
log4j.appender.splunkrest.pass=splunk
log4j.appender.splunkrest.host=splunkserver
log4j.appender.splunkrest.port=8089
log4j.appender.splunkrest.delivery=stream
log4j.appender.splunkrest.metaSource=rest
log4j.appender.splunkrest.metaSourcetype=testing
log4j.appender.splunkrest.metaIndex=main
log4j.appender.splunkrest.maxQueueSize=5MB
log4j.appender.splunkrest.dropEventsOnQueueFull=false
log4j.appender.splunkrest.layout=org.apache.log4j.PatternLayout
log4j.appender.splunkrest.layout.ConversionPattern=%m%n
# optionally you can enrich the messages with formatting tokens from the logging framework
#log4j.appender.splunkrest.layout.ConversionPattern=%d{ABSOLUTE} %m%n
#log4j.appender.splunkrest.layout.ConversionPattern=%m loglevel="%p"%n
# Send log events to a Splunk Raw TCP server socket
log4j.appender.splunkrawtcp=com.splunk.logging.log4j.appender.SplunkRawTCPAppender
log4j.appender.splunkrawtcp.host=splunkserver
log4j.appender.splunkrawtcp.port=5150
log4j.appender.splunkrawtcp.maxQueueSize=5MB
log4j.appender.splunkrawtcp.dropEventsOnQueueFull=false
log4j.appender.splunkrawtcp.layout=org.apache.log4j.PatternLayout
log4j.appender.splunkrawtcp.layout.ConversionPattern=%m%n
# optionally you can enrich the messages with formatting tokens from the logging framework
#log4j.appender.splunkrawtcp.layout.ConversionPattern=%d{ABSOLUTE} %m%n
#log4j.appender.splunkrawtcp.layout.ConversionPattern=%m loglevel="%p"%n
#Console appender
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
Running this application should allow the user to type in a line in the console, which should then be logged via log4j, and show up in Splunk. And this works. Sometimes! But only sometimes. Whether the logs arrive at Splunk or not seems completely random!
Does anyone have even the slightest idea about what can possibly be going wrong?