Sciweavers

Share
IALP
2010

Hierarchical Pitman-Yor Language Model for Machine Translation

10 years 6 months ago
Hierarchical Pitman-Yor Language Model for Machine Translation
The hierarchical Pitman-Yor process-based smoothing method applied to language model was proposed by Goldwater and by Teh; the performance of this smoothing method is shown comparable with the modified Kneser-Ney method in terms of perplexity. Although this method was presented four years ago, there has been no paper which reports that this language model indeed improves translation quality in the context of Machine Translation (MT). This is important for the MT community since an improvement in perplexity does not always lead to an improvement in BLEU score; for example, the success of word alignment measured by Alignment Error Rate (AER) does not often lead to an improvement in BLEU. This paper reports in the context of MT that an improvement in perplexity really leads to an improvement in BLEU score. It turned out that an application of the Hierarchical Pitman-Yor Language Model (HPYLM) requires a minor change in the conventional decoding process. Additionally to this, we propose a ...
Tsuyoshi Okita, Andy Way
Added 17 May 2011
Updated 17 May 2011
Type Journal
Year 2010
Where IALP
Authors Tsuyoshi Okita, Andy Way
Comments (0)
books