Sciweavers

SIGIR
2010
ACM

PRES: a score metric for evaluating recall-oriented information retrieval applications

12 years 10 months ago
PRES: a score metric for evaluating recall-oriented information retrieval applications
Information retrieval (IR) evaluation scores are generally designed to measure the effectiveness with which relevant documents are identified and retrieved. Many scores have been proposed for this purpose over the years. These have primarily focused on aspects of precision and recall, and while these are often discussed with equal importance, in practice most attention has been given to precision focused metrics. Even for recalloriented IR tasks of growing importance, such as patent retrieval, these precision based scores remain the primary evaluation measures. Our study examines different evaluation measures for a recall-oriented patent retrieval task and demonstrates the limitations of the current scores in comparing different IR systems for this task. We introduce PRES, a novel evaluation metric for this type of application taking account of recall and the user's search effort. The behaviour of PRES is demonstrated on 48 runs from the CLEF-IP 2009 patent retrieval track. A ful...
Walid Magdy, Gareth J. F. Jones
Added 21 May 2011
Updated 21 May 2011
Type Journal
Year 2010
Where SIGIR
Authors Walid Magdy, Gareth J. F. Jones
Comments (0)