Sciweavers

COLING
1996

Towards a More Careful Evaluation of Broad Coverage Parsing Systems

13 years 7 months ago
Towards a More Careful Evaluation of Broad Coverage Parsing Systems
Since treebanks have become available to researchers a wide variety of techniques has been used to make broad coverage parsing systems. This makes quantitative evaluation very important, but the current evaluation methods have a number of drawbacks such as arbitrary choices in the treebank and the difficulty in measuring statistical significance. We suggest a more detailed method for testing a parsing system using constituent boundaries, with a number of measures that give more information than current measures, and evaluate the quality of the test. We also show that statistical significance cannot be calculated in a straightforward way, and suggest a calculation method for the case of Bracket Recall.
Wide R. Hogenhout, Yuji Matsumoto
Added 02 Nov 2010
Updated 02 Nov 2010
Type Conference
Year 1996
Where COLING
Authors Wide R. Hogenhout, Yuji Matsumoto
Comments (0)