Cheap, Fast and Good Enough: Automatic Speech Recognition with Non-Expert Transcription

Deploying an automatic speech recognition system with reasonable performance requires expensive and time-consuming in-domain transcription. Previous work demonstrated that non-professional annotation through Amazon's Mechanical Turk can match professional quality. We use Mechanical Turk to transcribe conversational speech for as little as one thirtieth the cost of professional transcription. The higher disagreement of non-professional transcribers does not have a significant effect on system performance. While previous work demonstrated that redundant transcription can improve data quality, we found that resources are better spent collecting more data. Finally, we describe a quality control method without needing professional transcription.

• Type: Journal
• Year: 2010
• Where: NAACL
• Authors: Scott Novotney, Chris Callison-Burch
• Keywords: Automatic Speech Recognition, Computational Linguistics, Mechanical Turk, NAACL 2010, Professional Transcription