The importance of manual assessment in link discovery

Using a ground truth extracted from the Wikipedia, and a ground truth created through manual assessment, we show that the apparent performance advantage seen in machine learning approaches to link discovery are an artifact of trivial links that are actively rejected by manual assessors. Categories and Subject Descriptors H.3.4 [Information Storage and Retrieval]: System and Software – Information networks, Performance evaluation (efficiency and effectiveness). General Terms Documentation, Performance, Experimentation, Human Factors Keywords Wikipedia, Link Discovery, Assessment, Evaluation, INEX.

• Type: Conference
• Year: 2009
• Where: SIGIR
• Authors: Darren Wei Che Huang, Andrew Trotman, Shlomo Geva
• Keywords: Apparent Performance Advantage, Ground Truth, Information Retrieval, Machine Learning Approaches, SIGIR 2009