Sciweavers

LREC
2008

Evaluating Robustness Of A QA System Through A Corpus Of Real-Life Questions

13 years 5 months ago
Evaluating Robustness Of A QA System Through A Corpus Of Real-Life Questions
This paper presents the sequential evaluation of the question answering system SQuaLIA. This system is based on the same sequential process as most statistical question answering systems, involving 4 main steps from question analysis to answer extraction.The evaluation is based on a corpus made from 20 questions taken in the set of an evaluation campaign and which were well answered by SQuaLIA. Each of the 20 questions have been typed by 17 participants natives, non natives and dyslexics. They were vocally instructed the target of each question. Each of the 4 analysis step of the system involves a loss of accuracy, until an average of 60% of right answers at the end of the process. The main cause of this loss seem to be the orthographic mistakes users make on nouns. The most natural way for users to query search engines for specific information is to type a question. From this natural interaction the interest for efficient question raises Many evaluation campaigns on question answerin...
Laurianne Sitbon, Patrice Bellot, Philippe Blache
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2008
Where LREC
Authors Laurianne Sitbon, Patrice Bellot, Philippe Blache
Comments (0)