Quantcast
Channel: Latest Questions on Splunk Answers
Viewing all articles
Browse latest Browse all 13053

Issue with REST API Modular Input and Streaming Twitter API

$
0
0

I have recently been playing around with the REST API application and the streaming twitter feed and have come across an odd issue. After a lot of troubleshooting, it appears that everything works fine, only when you have a search term that brings back a high volume of events continuously. However, if you use a search term that looks for the odd event here or there, it seems there are python errors in splunkd with SSL timeouts etc.

This is my base configuration (minus the oauth stuff):

 [rest://Twitter]
    auth_type = oauth1
    endpoint = https://stream.twitter.com/1.1/statuses/filter.json
    host = TwitterAPI
    http_method = GET
    index = main
    index_error_response_codes = 1
    response_type = json
    sourcetype = tweets
    streaming_request = 1
    url_args = track=cold,splunk^stall_warnings=true
    delimiter = ^
    disabled = 0

So the above configuration works fine as tracking the word 'cold' brings back a pretty hefty number of events. However, if i remove the word cold and just use 'splunk' which has a far lower tweet rate than the word cold, I get java errors in Splunkd after about 30 secs as follows.

All of the below errors are preceded by:

ERROR ExecProcessor - message from "python "C:\Program Files\Splunk\etc\apps\rest_ta\bin\rest.py"":

ssl.SSLError: The read operation timed out
return self._sslobj.read(len)
File "C:\Program Files\Splunk\Python-2.7\Lib\ssl.py", line 162, in read
return self.read(buflen)
File "C:\Program Files\Splunk\Python-2.7\Lib\ssl.py", line 243, in recv
data = self._sock.recv(self._rbufsize)
File "C:\Program Files\Splunk\Python-2.7\Lib\socket.py", line 476, in readline
line = self.fp.readline(_MAXLINE + 1)
File "C:\Program Files\Splunk\Python-2.7\Lib\httplib.py", line 585, in _read_chunked
return self._read_chunked(amt)
File "C:\Program Files\Splunk\Python-2.7\Lib\httplib.py", line 543, in read
File "C:\Program Files\Splunk\etc\apps\rest_ta\bin\requests-2.0.0-py2.7.egg\requests\packages\urllib3\response.py", line 174, in read
File "C:\Program Files\Splunk\etc\apps\rest_ta\bin\requests-2.0.0-py2.7.egg\requests\packages\urllib3\response.py", line 225, in stream
File "C:\Program Files\Splunk\etc\apps\rest_ta\bin\requests-2.0.0-py2.7.egg\requests\models.py", line 572, in generate
File "C:\Program Files\Splunk\etc\apps\rest_ta\bin\requests-2.0.0-py2.7.egg\requests\models.py", line 602, in iter_lines
for line in r.iter_lines():
File "C:\Program Files\Splunk\etc\apps\rest_ta\bin\rest.py", line 465, in do_run
do_run()

The message at the bottom is where I hit save to update the url_args with just track=splunk. The error messages all appear after about 30 secs of hitting save on the rest inputs. After the SSL error, i think it just bugs out and does nothing going forward.

Any ideas?


Viewing all articles
Browse latest Browse all 13053

Trending Articles