Sciweavers

ISMIR
2003
Springer

Toward the scientific evaluation of music information retrieval systems

13 years 11 months ago
Toward the scientific evaluation of music information retrieval systems
This paper outlines the findings-to-date of a project to assist in the efforts being made to establish a TREC-like evaluation paradigm within the Music Information Retrieval (MIR) research community. The findings and recommendations are based upon expert opinion garnered from members of the Information Retrieval (IR), Music Digital Library (MDL) and MIR communities with regard to the construction and implementation of scientifically valid evaluation frameworks. Proposed recommendations include the creation of data-rich query records that are both grounded in real-world requirements and neutral with respect to retrieval technique(s) being examined; adoption, and subsequent validation, of a “reasonable person” approach to “relevance” assessment; and, the development of a secure, yet accessible, research environment that allows researchers to remotely access the large-scale testbed collection.
J. Stephen Downie
Added 07 Jul 2010
Updated 07 Jul 2010
Type Conference
Year 2003
Where ISMIR
Authors J. Stephen Downie
Comments (0)