I haven't seen an example of this so far so I'm going to ask.
I have Backup Exec 10. There is a daily job and and then a WEEKLY job I have scheduled. The thing is, doesn't matter what the job is, the file name is always the same, which is something like BEX_HOSTNAME_00683.XML. Of course every time a job runs (whether weekly or daily) and a log is written to, the number changes to the next in line, and a new log file is written. A new log is made every time a job runs.
I have a batch file that runs and converts these XML log files to text. The contents of the files are dumped to one log called "backup.txt" which is indexed by splunk. It's incremental, so it just keeps dumping the converted text to this file.
So here's my problem. I already have a props.conf entry for the daily backup to make all the contents from this "event" into one event using line breaks. This works great for daily. However for weekly, well that's a different story. Splunk chopped it up into 9 separate events.
Can I use props.conf entries more then once on the same sourcetype? Not sure the best way to go about this.
Here's some info: This is what a daily backup log looks like:
(04/26/13 06:00:00):bemcmd -o31 -la"C:\Users\Administrator\Desktop\Backup Logs\backuplog.txt" s0 -f"C:\Program Files\Symantec\Backup Exec\Data\BEX_TAPEBACKUP_00106.xml"
Server name : TAPEBACKUP
Job name : Database Maintenance
Job log : C:\Program Files\Symantec\Backup Exec\Data\BEX_TAPEBACKUP_00106.xml
Device name : BEDB
Summary of database maintenance activity:
* Saved contents of BEDB database
* Deleted expired data for BEDB database:
0 expired audit logs were deleted
0 expired reports were deleted
0 expired job histories were deleted
2 expired alert histories were deleted
2 expired job logs were deleted
Job started : Friday, April 26, 2013 4:00:00 AM
Job ended : Friday, April 26, 2013 4:00:01 AM
Completed status: Successful
RETURN VALUE: 1
And here's the props.conf entry that makes this look perfect in splunk as 1 event. It originally was splitting it up into about 3 events:
[sym_backup]
MUST_NOT_BREAK_AFTER = \d\sexpired\sjob\slogs\swere\sdeleted
MUST_BREAK_AFTER = RETURN\sVALUE:\s\d
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
The weekly backup text is a lot more and splunk has split this up every time it see's a timestamp.
Can I have multiple MUST_NOT_BREAK_AFTER entries under the same sourcetype in props.conf? The lines that I would use for the weekly backup are not present at all in the daily backup log.
For example, I tried adding another line to props.conf for the weekly backup data that's not breaking up, but this didn't work. One of the breaks is right after a line in the logs that says "Job name: Incremental Backup".
MUST_NOT_BREAK_AFTER = Job\sname:\sIncremental\sBackup
I'm not sure how to go about this or if this makes sense. Any ideas here? If all backup data, whether weekly or daily, is being indexed, but the formatting of each type is different, how can I have splunk know the difference? Can you use multiple props.conf entries perhaps?
I saw LINE_BREAKER had patterns you could match but I don't even know if that's possible with this situation.
Only other thing I can research is having the weekly backup logs send their logs somewhere else using a different name.