Sciweavers

Share
SIGIR
2010
ACM

Hierarchical pitman-yor language model for information retrieval

10 years 3 months ago
Hierarchical pitman-yor language model for information retrieval
In this paper, we propose a new application of Bayesian language model based on Pitman-Yor process for information retrieval. This model is a generalization of the Dirichlet distribution. The Pitman-Yor process creates a power-law distribution which is one of the statistical properties of word frequency in natural language. Our experiments on Robust04 indicate that this model improves the document retrieval performance compared to the commonly used Dirichlet prior and absolute discounting smoothing techniques. Categories and Subject Descriptors: H.3.3 [Information Storage and Retrieval]:Information Search and Retrieval General Terms: Theory, Algorithm, Experimentation
Saeedeh Momtazi, Dietrich Klakow
Added 16 Aug 2010
Updated 16 Aug 2010
Type Conference
Year 2010
Where SIGIR
Authors Saeedeh Momtazi, Dietrich Klakow
Comments (0)
books