61Fresh: Crossing Journalism with Technology

How do we make sense of the tsunami of information that surrounds us when even professional news editors find it challenging?

One answer could be 61Fresh, a crowd-driven news aggregator that searches tweets linking to 500 Boston-area websites every 10 minutes, identifies trending news bits, and then passes them on to Twitter followers or website readers.

61Fresh is among the many projects developed in a collaboration between GlobeLab, the Boston Globe’s R&D group, and the MIT Center for Civic Media (CFCM). Ali Hashmi, a civic media researcher and MIT Media Lab graduate student, contributed to 61Fresh as a Knight Fellow last summer by developing software that helped parse the universe of tweets. Among his projects was building natural language processing system that classified sports and non-sports news, so readers could modify their information streams.

For 61Fresh to work, you have to get rid of human decision makers, Hashmi says, they are just too slow. Algorithms churn through the tweets, identify what texts and URLs are retweeted, and determine—and share—the rising news.

“We can actually extract all these rising news items in real time and give you an accurate snapshot of how people are sharing news and what pieces of news are rising,” Hashmi says. And it’s more than just the news of the day, he says. “It is a very real time account of history in progress.”

Hashmi and CFCM are not just news hounds; they want to understand the impact of media tools and practices on communities worldwide. Hashmi, an expert in machine learning and natural language processing, is also studying the interactions between information published by the mainstream media in Pakistan and local bloggers.  Follow Hashmi on Twitter for more on his research.

Learn how the Nieman Journalism Lab described 61Fresh, which was funded by a grant from the Knight Foundation.

Ongoing collaborations bring authentic journalism challenges into the classroom and CFCM researchers are embedded at the GlobeLab, working on the technical issues facing newsrooms.