Sciweavers

ICASSP
2011
IEEE

Using multiple visual tandem streams in audio-visual speech recognition

12 years 7 months ago
Using multiple visual tandem streams in audio-visual speech recognition
The method which is called the “tandem approach” in speech recognition has been shown to increase performance by using classifier posterior probabilities as observations in a hidden Markov model. We study the effect of using visual tandem features in audio-visual speech recognition using a novel setup which uses multiple classifiers to obtain multiple visual tandem features. We adopt the approach of multi-stream hidden Markov models where visual tandem features from two different classifiers are considered as additional streams in the model. It is shown in our experiments that using multiple visual tandem features improve the recognition accuracy in various noise conditions. In addition, in order to handle asynchrony between audio and visual observations, we employ coupled hidden Markov models and obtain improved performance as compared to the synchronous model.
Ibrahim Saygin Topkaya, Hakan Erdogan
Added 21 Aug 2011
Updated 21 Aug 2011
Type Journal
Year 2011
Where ICASSP
Authors Ibrahim Saygin Topkaya, Hakan Erdogan
Comments (0)