Sciweavers

Share
ECIR
2008
Springer

Computing Information Retrieval Performance Measures Efficiently in the Presence of Tied Scores

8 years 12 months ago
Computing Information Retrieval Performance Measures Efficiently in the Presence of Tied Scores
Abstract. The Information Retrieval community uses a variety of performance measures to evaluate the effectiveness of scoring functions. In this paper, we show how to adapt six popular measures -- precision, recall, F1, average precision, reciprocal rank, and normalized discounted cumulative gain -- to cope with scoring functions that are likely to assign many tied scores to the results of a search. Tied scores impose only a partial ordering on the results, meaning that there are multiple possible orderings of the result set, each one performing differently. One approach to cope with ties would be to average the performance values across all possible result orderings; but unfortunately, generating result permutations requires super-exponential time. The approach presented in this paper computes precisely the same performance value as the approach of averaging over all permutations, but does so as efficiently as the original, tie-oblivious measures.
Frank McSherry, Marc Najork
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2008
Where ECIR
Authors Frank McSherry, Marc Najork
Comments (0)
books