Streaming First Story Detection with application to Twitter

With the recent rise in popularity and size of social media, there is a growing need for systems that can extract useful information from this amount of data. We address the problem of detecting new events from a stream of Twitter posts. To make event detection feasible on web-scale corpora, we present an algorithm based on locality-sensitive hashing which is able overcome the limitations of traditional approaches, while maintaining competitive results. In particular, a comparison with a stateof-the-art system on the first story detection task shows that we achieve over an order of magnitude speedup in processing time, while retaining comparable performance. Event detection experiments on a collection of 160 million Twitter posts show that celebrity deaths are the fastest spreading news on Twitter.

• Type: Journal
• Year: 2010
• Where: NAACL
• Authors: Sasa Petrovic, Miles Osborne, Victor Lavrenko
• Keywords: Computational Linguistics, Event Detection, NAACL 2010, Story Detection Task, Twitter Posts