Sciweavers

LREC
2010

Towards Semi-Automated Annotation for Prepositional Phrase Attachment

13 years 5 months ago
Towards Semi-Automated Annotation for Prepositional Phrase Attachment
This paper investigates whether high-quality annotations for tasks involving semantic disambiguation can be obtained without a major investment in time or expense. We examine the use of untrained human volunteers from Amazons Mechanical Turk in disambiguating prepositional phrase (PP) attachment over sentences drawn from the Wall Street Journal corpus. Our goal is to compare the performance of these crowdsourced judgments to the annotations supplied by trained linguists for the Penn Treebank project in order to indicate the viability of this approach for annotation projects that involve contextual disambiguation. The results of our experiments on a sample of the Wall Street Journal corpus show that invoking majority agreement between multiple human workers can yield PP attachments with fairly high precision. This confirms that a crowdsourcing approach to syntactic annotation holds promise for the generation of training corpora in new domains and genres where high-quality annotations a...
Sara Rosenthal, William Lipovsky, Kathleen McKeown
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2010
Where LREC
Authors Sara Rosenthal, William Lipovsky, Kathleen McKeown, Kapil Thadani, Jacob Andreas
Comments (0)