Sciweavers

Share
IR
2000

A Task-Oriented Non-Interactive Evaluation Methodology for Information Retrieval Systems

11 years 5 months ago
A Task-Oriented Non-Interactive Evaluation Methodology for Information Retrieval Systems
Past research has identified many different types of relevance in information retrieval (IR). So far, however, most evaluation of IR systems has been through batch experiments conducted with test collections containing only expert, topical relevance judgements. Recently, there has been some movement away from this traditional approach towards interactive, more user-centred methods of evaluation. However, these are expensive for evaluators in terms both of time and of resources. This paper describes a new evaluation methodology, using a task-oriented test collection, which combines the advantages of traditional non-interactive testing with a more user-centred emphasis. The main features of a task-oriented test collection are the adoption of the task, rather than the query, as the primary unit of evaluation and the naturalistic character of the relevance judgements.
Jane Reid
Added 18 Dec 2010
Updated 18 Dec 2010
Type Journal
Year 2000
Where IR
Authors Jane Reid
Comments (0)
books