Popcorn Sutton Posted May 6, 2014 Posted May 6, 2014 I'm trying to get access to Twitter feed about bail bond information. We want to get all the info we possibly can by scraping the "firehose" as they call it. My boss told me that this should be a pretty simple project and take a day to work out at most. I've already spent 3 and a half hours working on it, so I'm wondering if anyone else knows anything about this that may help me finish this project. Thanks in advance
AtomicMaster Posted May 6, 2014 Posted May 6, 2014 Twitter has an API, please consult it https://dev.twitter.com/docs
Popcorn Sutton Posted May 6, 2014 Author Posted May 6, 2014 (edited) It's extremely complicated. I feel like this task was wayyyy underestimated I pulled it off. I'm not quite done making it pretty yet, but I pulled it off. If anyone else is looking for help, here is the code and here is where I got the answer. import twitter from twitter import * from HTMLParser import HTMLParser class MLStripper(HTMLParser): def __init__(self): self.reset() self.fed = [] def handle_data(self, d): self.fed.append(d) def get_data(self): return ''.join(self.fed) def strip_tags(html): s = MLStripper() s.feed(html) return s.get_data() t = Twitter( auth = OAuth(OAUTH_TOKEN, OAUTH_SECRET, CONSUMER_KEY, CONSUMER_SECRET) ) makealist = t.search.tweets(q="#put your search hashtag here") makeitastring = str(makealist) strip_tags(makeitastring) makingalist = makeitastring.split(',') Edited May 6, 2014 by Popcorn Sutton
Popcorn Sutton Posted May 13, 2014 Author Posted May 13, 2014 (edited) You can remove this post if you like. I asked how to get the coordinates of a particular location, what I found was to search the coordinates, and since I had an example of New York City's coordinates, I went off that template. https://www.google.com/search?q=new+york+citys+coordinates&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a&channel=sb Here you will see that it's approximately 40,74. In the twitter API, you put {'locations':'-74,40,-73,41'} and as you can see, the coordinates are reverse. Edited May 13, 2014 by Popcorn Sutton
Popcorn Sutton Posted May 13, 2014 Author Posted May 13, 2014 Also, in order to get streaming data from the 'firehose', TwitterAPI is your best bet. in the command line type this- pip install TwitterAPI This API is by far the easiest and most useful. If you monitor your target location, you will get the stream of data. This is not entirely explanatory on how to use the API, but it's sufficient to get you started.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now