I've been all over related questions in Splunk base, but I have not found out why exactly Splunk will sometime index duplicate events. A simple dedup will help mitigate this issue but does not get to the core of the problem.
My Scenario: I'm indexing mutiple logs from a global file system so my input.conf would look like this.
[monitor://global/file/system/apache/log/nodes*/access_log]
index = log_index
The duplicate number of events is not consistent. The number is usually between 2 an 12. Should I add crcSalt option? The Other option im using is setting the maxKBps = 56 on the forwarder, will this have any impact on the main indexer?