Inspired by a post over at Silicon Florist about following local people on Twitter, I investigated whether it was possible to follow someone in a geographic area. The closest I found was TwitterVision, which is a great idea, but doesn’t exactly do what I want. So I ended up taking a few hours and developed TwitterWhere, an aggregator that watches the Twitter public timeline and geocodes what it finds. From there anyone can generate an RSS or XML feed that will pull only Tweets within a set radius around an area. Now you can have a feed of Tweets within 50 miles or Portland, or 5 miles of Manhattan, New York.
For anyone who is interested, here’s how I pulled it off…
I wrote the app in Ruby on Rails, utilizing my favorite gem of all time, Graticule (geocoding and spatial search), Hpricot (parsing the public timeline on Twitter), and a couple custom geocoding libraries I already had cooked up for Unthirsty and Knitmap.
Every minute I query the Twitter public timeline XML file and the results. If a new user is entered into the system it is geocoded and any Tweets added by that user from that point inherit the coordinates. Any users with an invalid location are thrown out and not cached. I also cache geocode queries locally. Both of these measures are in place so I don’t overload the geocoding services I use (especially considering I have two other fairly heavy traffic sites using geocoding services).
On the front end, feeds are not exactly ‘generated’, it’s more of a verification that the location someone has entered is geocodable. Once the location is verified a link is presented to the user that points to RSS or XML feeds. If this ends up getting more traffic, I’ll consider using caching for each unique feed, but right now when you visit a feed URL, it is generated from the database in real time, every time.
TwitterWhere runs on my good old Dual 1.8GHz Powermac G5 at our co-location facility, using a custom compiled Apache 2.2, mod_proxy_balancer and 5 mongrels.
Okay, I’m done geeking. Here are few mentions of TwitterWhere from around the web. Thanks for the write-ups!
- Silicon Florist - TwitterWhere: Get tweets by geographic location
- ReadWriteWeb - TwitterWhere Tracks Tweets from Any Location, Like San Diego
- Wired Blog - TwitterWhere Tracks Tweets By Location
P.S. Apologies to TwitterWhere.com, I didn’t even think to check if the domain existed, let alone to see if there was something there! We don’t exactly do the same thing (TwitterWhere.com allows you to easily update your current location on Twitter), so hopefully there isn’t too much confusion.
Hey, that’s pretty awesome - I found 3 new local peeps just reading the feed, and now I’ve set-up a Twitter account and am piping the feed from TwitterWhere into TwitterFeed into a Twitter account and have subscribed to it.
http://www.twitter.com/CanberraTweets
Surprised no comments yet - we’ve all been talking about it / poking at it all day.
Thanks!
Thanks so much for doing this. I’m able to find more people closer to me now.
I did the same thing as nathanaelB, but for central NM. Is there a more elegant method for piping the output back into twitter? Twitterfeed and RSS2Twitter are nice services, but they only update every 30 mins–not exactly the best way to get a feel for whats going on in the local area.
It sounds like Matt’s next project!
er, for anyone interested, the twitteruser is central_nm
now trying using twitterbot (with mono) for faster updates. It’s *much* better now, but I’m getting some wierd results, like people in new jersey and japan showing up in NM. (AFAIK the public timeline lists them *not* in NM)
hi, i am thinking to use twitterwhere for a website i have in mind since some time. as i couldn’t find an emailadress at first glance, i’m just leaving this comment to give you mine. would be great to have a short chat about my idea. first question: do you use the location in the userprofile and/or tweezts that include L:foobar? tia. ma
Brilliant application. Kudos for the work. How about TwitterWhere mobile — GPS tracking to your mobile device so it shows where you are, not where you’re from? Does that exist? Just a thought.
Hi Matt - I did this with RSS feeds from your great little app. It’s brilliant. Love. Please have a look and let me know if this is ok. As I say I don’t expect anyone will look but it was fun putting it together (it’s just wordpress with nine feeds)and I use it to find people to follow in New Zealand. http://wherewherewhere.wordpress.com/
Again great app, well done.
Hey Matt
I have some ideas for a website.
I do front end stuff but need someone to set up a database for a zip finder.
Please contact me if you can.
trevor
I’m getting a *lot* of false positives, mainly from new jersey and japan (matching against zip 87120/50mi)
@adin: I did a little research and it looks like the user’s location that showed up in your feed was Sasebo. I’m sure it meant Sasebo, Nagasaki, Japan, but since it was just Sasebo, our geocoders somehow figured out that there is a Sasebo Japanese Garden in Albequerque, and thought that’s what the real location was. Looks like our geocoders are a little to smart for their own good!
Thanks for bringing it up, I’m sure there are a bunch of little discrepancies like this in other places we’ll catch as TwitterWhere moves forward.
@adin: Also, fixed Chi-town…
Matt,
Thanks for sharing the story behind a remarkable tool that we’re using to improve life safety in the City of Los Angeles.
Respectfully Yours in Safety and Service,
Brian Humphrey
Firefighter/Specialist
Public Service Officer
Los Angeles Fire Department
Hey Matt-
Really cool tool but I am having trouble getting on the feed; maybe a bug?
Feed that I never seem to show up on: http://twitterwhere.mattking.org/feed/97214/3
My twittervision URL: http://twittervision.com/philips
As you can see twittervision seems to recognize me.
Thanks,
Brandon
Matt, I haven’t received my twitterwhere feeds for the last week or so. The site page itself seems to be down. Did something get changed recently? I love the service and hope you are still maintaining it. Thanks!
Thanks for this tool. Have checked web version. FYI, etting an error message unzipping AIR version zip file. Screen shot of dialog box
http://i101.photobucket.com/albums/m72/gregflynn/TwitterLocalWinZipScreen.jpg
Matt,
Would it be possible (in your “spare” time) to filter out your own (ie: my) posts when looking for people within x miles of my zip?
Also - could you add more distance to the drop down? Some of us (ok, ok, ME) live in rural places where 20 miles is a trip to the nearest town (if you’re lucky).
50, 100, 150, 200, 250 miles might work:)
Thanks!
Mark