A Comparison of Language Identification Approaches on Short, Query-Style Texts

In a multi-language Information Retrieval setting, the knowledge about the language of a user query is important for further processing. Hence, we compare the performance of some typical approaches for language detection on very short, query-style texts. The results show that already for single words an accuracy of more than 80% can be achieved, for slightly longer texts we even observed accuracy values close to 100%.

• Type: Conference
• Year: 2010
• Where: ECIR
• Authors: Thomas Gottron, Nedim Lipka
• Keywords: ECIR 2010, Information Technology, Multi-language Information Retrieval, Typical Approaches, User Query