Sciweavers

AIR
2005

Evolving General Term-Weighting Schemes for Information Retrieval: Tests on Larger Collections

13 years 3 months ago
Evolving General Term-Weighting Schemes for Information Retrieval: Tests on Larger Collections
Term-weighting schemes are vital to the performance of Information Retrieval models that use term frequency characteristics to determine the relevance of a document. The vector space model is one such model in which the weights assigned to the document terms are of crucial importance to the accuracy of the retrieval system. This paper describes a genetic programming framework used to automatically determine term-weighting schemes that achieve a high average precision. These schemes are tested on standard test collections and are shown to perform as well as, and often better than, the modern BM25 weighting scheme. We present an analysis of the schemes evolved to explain the increase in performance. Furthermore, we show that the global (collection wide) part of the evolved weighting schemes also increases average precision over idf on larger TREC data. These global weighting schemes are shown to adhere to Luhn's resolving power as middle frequency terms are assigned the highest weig...
Ronan Cummins, Colm O'Riordan
Added 15 Dec 2010
Updated 15 Dec 2010
Type Journal
Year 2005
Where AIR
Authors Ronan Cummins, Colm O'Riordan
Comments (0)