Risky business: modeling and exploiting uncertainty in information retrieval

Most retrieval models estimate the relevance of each document to a query and rank the documents accordingly. However, such an approach ignores the uncertainty associated with the estimates of relevancy. If a high estimate of relevancy also has a high uncertainty, then the document may be very relevant or not relevant at all. Another document may have a slightly lower estimate of relevancy but the corresponding uncertainty may be much less. In such a circumstance, should the retrieval engine risk ranking the first document highest, or should it choose a more conservative (safer) strategy that gives preference to the second document? There is no definitive answer to this question, as it depends on the risk preferences of the user and the information retrieval system. In this paper we present a general framework for modeling uncertainty and introduce an asymmetric loss function with a single parameter that can model the level of risk the system is willing to accept. By adjusting the ri...

• Type: Conference
• Year: 2009
• Where: SIGIR
• Authors: Jianhan Zhu, Jun Wang, Ingemar J. Cox, Michael J. Taylor
• Keywords: Asymmetric Loss Function, Dirichlet Smoothing Methods, Document, Information Retrieval, SIGIR 2009