The following search removes usernames, host names, all time information, any digits, and turns all strings of white space into a single "_" for the _raw message.
.... |rex mode=sed "s/[a-z]+\d{1,4}//" |rex mode=sed "s/user\s[a-z]+/user /"
|rex mode=sed "s/(user|USER)=[a-z]+/user=/" |rex mode=sed "s/\d+//g"
|rex mode=sed "s/(Jan|January|Feb|Febuary|Mar|March|Apr|April|May|Jun|June|Jul|July|Aug|August|Sep|September|Oct|October|Nov|November|Dec|December|Mon|Tue|Wed|Thu|Fri|Sat|Sun|PM|AM|PDT|PST)//g"
|rex mode=sed "s/\s+/_/g"| rename _raw AS msgdigest |stats count by msgdigest
I would like to be able to have this "digested" message available as a field does anyone know how to turn this into a field. Preferably with the transforms and extraction pages in manager as I'm currently having unrelated problems with props.conf and transforms.conf files. Please help!