Variational Decoding for Statistical Machine Translation

Statistical models in machine translation exhibit spurious ambiguity. That is, the probability of an output string is split among many distinct derivations (e.g., trees or segmentations). In principle, the goodness of a string is measured by the total probability of its many derivations. However, finding the best string (e.g., during decoding) is then computationally intractable. Therefore, most systems use a simple Viterbi approximation that measures the goodness of a string using only its most probable derivation. Instead, we develop a variational approximation, which considers all the derivations but still allows tractable decoding. Our particular variational distributions are parameterized as n-gram models. We also analytically show that interpolating these n-gram models for different n is similar to minimumrisk decoding for BLEU (Tromble et al., 2008). Experiments show that our approach improves the state of the art.

• Type: Journal
• Year: 2009
• Where: ACL
• Authors: Zhifei Li, Jason Eisner, Sanjeev Khudanpur
• Keywords: ACL 2009, Computational Linguistics, Exhibit Spurious Ambiguity, N-gram Models, String