Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm

This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the perceptron algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in

• Type: Conference
• Year: 2004
• Where: ACL
• Authors: Brian Roark, Murat Saraclar, Michael Collins, Mark Johnson
• Keywords: ACL 2004, ACL 2007, Conditional Random Fields, Discriminative Language Modeling, Perceptron Algorithm