Greetings, I apologize in advance for the long post.
Problem abstract: field discovery and extract work great, but searching on extracted fields gives weird results.
Input stream: single-line events made up of unordered keyword/value pairs. Each pair has format KEYWORD^VALUE, where "^" is the K=V separator. Pairs are delimited by 0x1F. One example event is below:
action_forunit^n/ax1Faction_type^action_executex1Fresource_id^ELFVIEWx1FappId^ERDx1Fresource_currency^USDx1FcorrId^0x1FtimeStamp^1378492210757x1FeventType^3000x1Faction_foruser^InformDeveloperx1Fhostname^rsomdavecs01x1Fresource_amount^0.0x1Faction_forcustomer^n/ax1Faction_foraccount^n/ax1Faction_forregion^n/ax1Faction_forgroup^unknownx1Fresource_info^n/ax1Fresult_info^n/ax1Faudit_level^1x1Faction_info^Performing doGet() of the MainServletx1FcomponentId^ELFViewx1Fresource_name^ELF View Web Applicationx1FsessionId^N/Ax1Fresource_idtype^product_codex1Fresult_type^result_successx1Fresource_type^resource_productx1F
Note: The sequence "x1F" above represents a single byte 0x1F, as represented in Splunk search, but have verified actual 0x0f using hex dump.
Event breaking and time stamping as follows:
default/props.conf
[ELFDATA] NO_BINARY_CHECK=1 SHOULD_LINEMERGE=false TIME_PREFIX=timeStamp^ pulldown_type=1
Field extractions as follows:
local/props.conf
[ELFDATA] REPORT-ELFKV = ELFKV
local/transforms.conf
[ELFKV] CLEAN_KEYS = 1 FORMAT = $1::$2 MV_ADD = 0 REGEX = ([^^]+?)^([^x1f]+?)[x1f]
For the sample data above, field "appId" is extracted and assigned value “ERD”. But a search using appId="ERD" returns no results. Also, certain wildcard searches (appId=, appId=ERD, appId="E") work, while others (appId="ERD", appId="ER*") don't.
Makes no sense that I can see.
Partial workaround by piping search results to a subsequent search, as:
<first search=""> | search appId="ERD"
Works (though I wish someobody would tell me why), but fails when a new search is generated, as in auto-drilldown.
Like I said, sorry for the length of the post. Brevity was never my strong suit.