Sciweavers

Share
SIGIR
2012
ACM

Top-k learning to rank: labeling, ranking and evaluation

8 years 4 months ago
Top-k learning to rank: labeling, ranking and evaluation
In this paper, we propose a novel top-k learning to rank framework, which involves labeling strategy, ranking model and evaluation measure. The motivation comes from the difficulty in obtaining reliable relevance judgments from human assessors when applying learning to rank in real search systems. The traditional absolute relevance judgment method is difficult in both gradation specification and human assessing, resulting in high level of disagreement on judgments. While the pairwise preference judgment, as a good alternative, is often criticized for increasing the complexity of judgment from O(n) to O(n log n). Considering the fact that users mainly care about top ranked search results, we propose a novel top-k labeling strategy which adopts the pairwise preference judgment to generate the top k ordering items from n documents (i.e. top-k ground-truth) in a manner similar to that of HeapSort. As a result, the complexity of judgment is reduced to O(n log k). With the topk ground-tru...
Shuzi Niu, Jiafeng Guo, Yanyan Lan, Xueqi Cheng
Added 28 Sep 2012
Updated 28 Sep 2012
Type Journal
Year 2012
Where SIGIR
Authors Shuzi Niu, Jiafeng Guo, Yanyan Lan, Xueqi Cheng
Comments (0)
books