The End of TwapperKeeper? (And What to Do about It)

TwapperKeeper LogoWe’ve long been fans of TwapperKeeper—the web service that tracks and archives Twitter hashtags. We’ve used TwapperKeeper to record student discussions on Twitter, and we’ve used TwapperKeeper to archive conference backchannels. The service was easy to set up (just enter a hashtag and forget about it while tweets were collected in the background) and easy to use (simply download the entire archive as a spreadsheet or XML file). TwapperKeeper was especially useful for post-event analyses of tweets from conferences, both real and imagined.

Alas, however, TwapperKeeper is ending as we know it. The service will continue to exist, but Twitter is forcing TwapperKeeper to discontinue its killer features—exporting and downloading archives—because they violate Twitter’s API Terms of Service. So, TwapperKeeper can still capture your tweets, but you won’t be able to get them from TwapperKeeper. Watch TwapperKeeper’s founder, John O’Brien, explain the situation and offer a few recommendations:

As O’Brien notes, users have until March 20, 2011 to download their archives. Do it now, before they disappear! If you have multiple archives (I have 29!), you’ll have to download each one individually. Again, do it now!

Audrey Watters notes on ReadWriteWeb that Twitter’s strict enforcement of its already strict Terms of Service will seriously hinder academic research about Twitter and social networking—not to mention complicating Twitter’s usefulness for our teaching and scholarship more generally.

So what’s a poor Twitterer to do?

We enthusiastically endorse O’Brien’s advice: take archiving into your own hands. There’s an open-source version of TwapperKeeper, called YourTwapperKeeper that users can install on their own web server (though I’ve had mixed results, getting it working perfectly on one server and not at all on another). YourTwapperKeeper exports hashtag archives into Excel files, preserving TwapperKeeper’s killer feature. And there’s also ThinkUp, which archives personal tweets but not hashtags (though that feature is on ThinkUp’s long-term roadmap). You can also create an RSS feed for a hashtag, though the resulting data is much less pliable for the purposes of data analysis. Finally, I personally recommend The Archivist Desktop, which runs on your Windows machine and can archive hashtags in the background. It generates exportable, structured XML data, but since The Archivist resides on your own machine, it’s not public—and not violating any Terms of Service.

Do you have other recommendations? How are you going to archive your classroom tweets or conference backchannel now? Have you gotten YourTwapperKeeper to work on your own server? Do you have any tips for users getting their existing archives out of TwapperKeeper?

Return to Top