Refining Generative Language Models using Discriminative Learning

We propose a new approach to language modeling which utilizes discriminative learning methods. Our approach is an iterative one: starting with an initial language model, in each iteration we generate 'false' sentences from the current model, and then train a classifier to discriminate between them and sentences from the training corpus. To the extent that this succeeds, the classifier is incorporated into the model by lowering the probability of sentences classified as false, and the process is repeated. We demonstrate the effectiveness of this approach on a natural lan

• Type: Conference
• Year: 2008
• Where: EMNLP
• Authors: Ben Sandbank
• Keywords: Current Model, Discriminative Learning Methods, EMNLP 2008, Initial Language Model, Natural Language Processing