Sciweavers

ECIR
2009
Springer

Using Multiple Query Aspects to Build Test Collections without Human Relevance Judgments

14 years 24 days ago
Using Multiple Query Aspects to Build Test Collections without Human Relevance Judgments
Abstract. Collecting relevance judgments (qrels) is an especially challenging part of building an information retrieval test collection. This paper presents a novel method for creating test collections by offering a substitute for relevance judgments. Our method is based on an old idea in IR: a single information need can be represented by many query articulations. We call different articulations of a particular need query aspects. By combining the top k documents retrieved by a single system for multiple query aspects, we build judgment-free qrels whose rank ordering of IR systems correlates highly with rankings based on human relevance judgments.
Miles Efron
Added 08 Mar 2010
Updated 08 Mar 2010
Type Conference
Year 2009
Where ECIR
Authors Miles Efron
Comments (0)