Hello
I am trying to get a stream of twitter data for a string to analyze some results. However I have not been able to do that successfully. Here are the two inputs which I tried and issues with each one of them.
[rest://resttwitter]
auth_type = oauth1
endpoint = https://stream.twitter.com/1.1/statuses/filter.json
http_method = GET
index = twitter
index_error_response_codes = 0
oauth1_access_token = 159023053-ZHaAF0KIVFoCfDvQvCvniw3oU3R4s3txxxxxx
oauth1_access_token_secret = E6AeYZTQnPk8s7C2UN4whxE1rKRozRugrvRdxxxxx
oauth1_client_key = VRyu5GvIPkhbC71xxxxx
oauth1_client_secret = F3THGsofgsV7xifw5FS78YMQ1t4jzderxxxxxxx
response_type = json
sourcetype = rest_twitter1
streaming_request = 1
url_args = track=twitter
With this endpoint I always get the error that its 401. I have confirmed the token again and again and still dont understand on why this error happens.
When I try this input with a different endpoint
[rest://resttwitter]
auth_type = oauth1
endpoint = https://api.twitter.com/1.1/search/tweets.json
http_method = GET
index = twitter
index_error_response_codes = 0
oauth1_access_token = 159023053-ZHaAF0KIVFoCfDvQvCvniw3oU3R4s3txxxxxx
oauth1_access_token_secret = E6AeYZTQnPk8s7C2UN4whxE1rKRozRugrvRdxxxxx
oauth1_client_key = VRyu5GvIPkhbC71xxxxx
oauth1_client_secret = F3THGsofgsV7xifw5FS78YMQ1t4jzderxxxxxxx
response_type = json
sourcetype = rest_twitter1
streaming_request = 1
url_args = q=PSEC
I get duplicate events generated. Also these events are not exactly breaking up as json events.
Here is the props.conf.
[rest_twitter1]
TIME_PREFIX = "created_at":"
TIME_FORMAT = %a %b %d %H:%M:%S %z %Y
MAX_TIMESTAMP_LOOKAHEAD = 40
Any help please?