Sciweavers

Share
ACL
2008

Collecting a Why-Question Corpus for Development and Evaluation of an Automatic QA-System

9 years 22 days ago
Collecting a Why-Question Corpus for Development and Evaluation of an Automatic QA-System
Question answering research has only recently started to spread from short factoid questions to more complex ones. One significant challenge is the evaluation: manual evaluation is a difficult, time-consuming process and not applicable within efficient development of systems. Automatic evaluation requires a corpus of questions and answers, a definition of what is a correct answer, and a way to compare the correct answers to automatic answers produced by a system. For this purpose we present a Wikipedia-based corpus of Whyquestions and corresponding answers and articles. The corpus was built by a novel method: paid participants were contacted through a Web-interface, a procedure which allowed dynamic, fast and inexpensive development of data collection methods. Each question in the corpus has several corresponding, partly overlapping answers, which is an asset when estimating the correctness of answers. In addition, the corpus contains information related to the corpus collection proce...
Joanna Mrozinski, Edward W. D. Whittaker, Sadaoki
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2008
Where ACL
Authors Joanna Mrozinski, Edward W. D. Whittaker, Sadaoki Furui
Comments (0)
books