The focus of information retrieval evaluations, such as NIST's TREC evaluations (e.g. Voorhees 2003), is on evaluation of the information content of system responses. On the ...
Olga Babko-Malaya, Dan Hunter, Connie Fournelle, J...
While test collection construction is a time-consuming and expensive process, the true cost is amortized by reusing the collection over hundreds or thousands of experiments. Some ...
Ben Carterette, Evgeniy Gabrilovich, Vanja Josifov...
Definition questions represent a largely unexplored area of question answering--they are different from factoid questions in that the goal is to return as many relevant "nugg...
The paper reviews the TREC Programme up to TREC-6 (1997), considering the test results, the substantive findings for IR that follow, and the lessons TREC offers for IR evaluation....
This poster reports upon the ongoing efforts being made to establish TREC-like and other comprehensive evaluation paradigms within the Music IR (MIR) and Music Digital Library (MD...