Generalized distances between rankings

Spearman’s footrule and Kendall’s tau are two well established distances between rankings. They, however, fail to take into account concepts crucial to evaluating a result set in information retrieval: element relevance and positional information. That is, changing the rank of a highly-relevant document should result in a higher penalty than changing the rank of an irrelevant document; a similar logic holds for the top versus the bottom of the result ordering. In this work, we extend both of these metrics to those with position and element weights, and show that a variant of the Diaconis–Graham inequality still holds — the generalized two measures remain within a constant factor of each other for all permutations. We continue by extending the element weights into a distance metric between elements. For example, in search evaluation, swapping the order of two nearly duplicate results should result in little penalty, even if these two are highly relevant and appear at the top of...

• Type: Conference
• Year: 2010
• Where: WWW
• Authors: Ravi Kumar, Sergei Vassilvitskii
• Keywords: Constant Factor, Element Weights, Internet Technology, Kendall’s Tau, WWW 2010