Sciweavers

Share
ACL
2008

Assessing the Costs of Sampling Methods in Active Learning for Annotation

12 years 1 months ago
Assessing the Costs of Sampling Methods in Active Learning for Annotation
Traditional Active Learning (AL) techniques assume that the annotation of each datum costs the same. This is not the case when annotating sequences; some sequences will take longer than others. We show that the AL technique which performs best depends on how cost is measured. Applying an hourly cost model based on the results of an annotation user study, we approximate the amount of time necessary to annotate a given sentence. This model allows us to evaluate the effectiveness of AL sampling methods in terms of time spent in annotation. We acheive a 77% reduction in hours from a random baseline to achieve 96.5% tag accuracy on the Penn Treebank. More significantly, we make the case for measuring cost in assessing AL methods.
Robbie Haertel, Eric K. Ringger, Kevin D. Seppi, J
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2008
Where ACL
Authors Robbie Haertel, Eric K. Ringger, Kevin D. Seppi, James L. Carroll, Peter McClanahan
Comments (0)
books