Approximating Learning Curves for Active-Learning-Driven Annotation

Active learning (AL) is getting more and more popular as a methodology to considerably reduce the annotation effort when building training material for statistical learning methods for various NLP tasks. A crucial issue rarely addressed, however, is when to actually stop the annotation process to profit from the savings in efforts. This question is tightly related to estimating the classifier performance after a certain amount of data has already been annotated. While learning curves are the default means to monitor the progress of the annotation process in terms of classifier performance, this requires a labeled gold standard which

• Type: Conference
• Year: 2008
• Where: LREC
• Authors: Katrin Tomanek, Udo Hahn
• Keywords: Annotation, Annotation Process, Education, Labeled Gold Standard, LREC 2008