Sciweavers

Share
CLEF
2010
Springer

Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements

8 years 11 months ago
Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements
Recent years have seen a growing interest in research into patent retrieval. One of the key issues in conducting information retrieval (IR) research is meaningful evaluation of the effectiveness of the retrieval techniques applied to task under investigation. Unlike many existing well explored IR tasks where the focus is on achieving high retrieval precision, patent retrieval is to a significant degree a recall focused task. The standard evaluation metric used for patent retrieval evaluation tasks is currently mean average precision (MAP). However this does not reflect system recall well. Meanwhile, the alternative of using the standard recall measure does not reflect user search effort, which is a significant factor in practical patent search environments. In recent work we introduce a novel evaluation metric for patent retrieval evaluation (PRES) [13]. This is designed to reflect both system recall and user effort. Analysis of PRES demonstrated its greater effectiveness in evaluating...
Walid Magdy, Gareth J. F. Jones
Added 21 Mar 2011
Updated 21 Mar 2011
Type Journal
Year 2010
Where CLEF
Authors Walid Magdy, Gareth J. F. Jones
Comments (0)
books