Quantcast
Channel: Latest Questions on Splunk Answers
Viewing all articles
Browse latest Browse all 13053

Transforms.conf SOURCE_KEY Questions

$
0
0

I run HAProxy and grab it via a universal forwarder and send it to our receiver/indexer (all on same host). I modified my props.conf as follows.

props.conf
[source::/var/log/*haproxy.log]
TRANSFORMS-syslogstripper = haproxy_syslog_stripper, haproxyfields, clientinfofields, backendfields, requestinfo, connectioninfo, queueinfo, uriinfo

[sourcetype::HAProxy]
MAX_TIMESTAMP_LOOKAHEAD=40
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TZ = US/Mountain
REPORT-haproxyfieldextract = haproxyfields, clientinfofields, backendfields, requestinfo, connectioninfo, queueinfo, uriinfo
TRANSFORMS-haproxystuff = haproxyfields

Here is my transforms.conf where I listed pertinent HAProxy info

transforms.conf

# This will strip the syslog header (date stamp and host) from a syslog event
[haproxy_syslog_stripper]
REGEX         = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s[^\s]*\s(.*)$
FORMAT        = $1
DEST_KEY      = _raw

# Transform for HAProxy

[haproxyfields]
DELIMS = " "
FIELDS = haproxy_id,client_info, date_time,frontend_name,backend,request_info,status_code,response_size,val1,val2,flags,connection_info,queue_info,req_header,resp_header,method,uri_info
CLEAN_KEYS=true

#the following is used to extract values from the previous extraction
[clientinfofields]
SOURCE_KEY=client_info
DELIM = ":"
FIELDS = client_ip,client_port
[backendfields]
SOURCE_KEY=backend
DELIM = "/"
FIELDS = backend_name,server_name
[requestinfo]
SOURCE_KEY=request_info
DELIM= "/"
FIELDS=request_time,queue_time,connection_time,response_time,total_time
[connectioninfo]
SOURCE_KEY=connection_info
DELIM= "/"
FIELDS=process_connections,frontend_connections,backend_connections,server_connections,retries
[queueinfo]
SOURCE_KEY=queue_info
DELIM= "/"
FIELDS=server_queue_size,backend_queue_size

#You can still use regex on those extraction that still need it.
[uriinfo]
SOURCE_KEY=uri_info
REGEX=(?<uri>[^"]+?)

I am able to get the fields listed in haproxyfields stanza to extract using this search term:

sourcetype="HAProxy" | extract haproxyfields
That works great and I am super excited by that. The problem is that I have no idea how I would then get the fields that depend on the previously extracted stuff on my haproxyfields stanza to display as well (i.e. client_ip, client_port, backend_name, server_name, etc.) Any ideas why those fields wouldn't just be extracted along with the "haproxyfields"?


Viewing all articles
Browse latest Browse all 13053

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>