Natural Language Processing in Information Retrieval

13 years 4 months ago
Natural Language Processing in Information Retrieval
Many Natural Language Processing (NLP) techniques have been used in Information Retrieval. The results are not encouraging. Simple methods (stopwording, porter-style stemming, etc.) usually yield significant improvements, while higher-level processing (chunking, parsing, word sense disambiguation, etc.) only yield very small improvements or even a decrease in accuracy. At the same time, higher-level methods increase the processing and storage cost dramatically. This makes them hard to use on large collections. We review NLP techniques and come to the conclusion that (a) NLP needs to be optimized for IR in order to be effective and (b) document retrieval is not an ideal application for NLP, at least given the current state-of-the-art in NLP. Other IR-related tasks, e.g., question answering and information extraction, seem to be better suited.
Thorsten Brants
Added 31 Oct 2010
Updated 31 Oct 2010
Type Conference
Year 2003
Where CLIN
Authors Thorsten Brants
Comments (0)