Sciweavers

ACL
2011

Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability

12 years 7 months ago
Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability
In statistical machine translation, a researcher seeks to determine whether some innovation (e.g., a new feature, model, or inference algorithm) improves translation quality in comparison to a baseline system. To answer this question, he runs an experiment to evaluate the behavior of the two systems on held-out data. In this paper, we consider how to make such experiments more statistically reliable. We provide a systematic analysis of the effects of optimizer instability—an extraneous variable that is seldom controlled for—on experimental outcomes, and make recommendations for reporting results more accurately.
Jonathan H. Clark, Chris Dyer, Alon Lavie, Noah A.
Added 24 Aug 2011
Updated 24 Aug 2011
Type Journal
Year 2011
Where ACL
Authors Jonathan H. Clark, Chris Dyer, Alon Lavie, Noah A. Smith
Comments (0)