Sciweavers

Share
SIGIR
2010
ACM

On statistical analysis and optimization of information retrieval effectiveness metrics

11 years 3 months ago
On statistical analysis and optimization of information retrieval effectiveness metrics
This paper presents a new way of thinking for IR metric optimization. It is argued that the optimal ranking problem should be factorized into two distinct yet interrelated stages: the relevance prediction stage and ranking decision stage. During retrieval the relevance of documents is not known a priori, and the joint probability of relevance is used to measure the uncertainty of documents’ relevance in the collection as a whole. The resulting optimization objective function in the latter stage is, thus, the expected value of the IR metric with respect to this probability measure of relevance. Through statistically analyzing the expected values of IR metrics under such uncertainty, we discover and explain some interesting properties of IR metrics that have not been known before. Our analysis and optimization framework do not assume a particular (relevance) retrieval model and metric, making it applicable to many existing IR models and metrics. The experiments on one of resulting app...
Jun Wang, Jianhan Zhu
Added 16 Aug 2010
Updated 16 Aug 2010
Type Conference
Year 2010
Where SIGIR
Authors Jun Wang, Jianhan Zhu
Comments (0)
books