Two-Level Bimodal Association for Audio-Visual Speech Recognition

14 years 1 months ago
Two-Level Bimodal Association for Audio-Visual Speech Recognition
This paper proposes a new method for bimodal information fusion in audio-visual speech recognition, where cross-modal association is considered in two levels. First, the acoustic and the visual data streams are combined at the feature level by using the canonical correlation analysis, which deals with the problems of audio-visual synchronization and utilizing the cross-modal correlation. Second, information streams are integrated at the decision level for adaptive fusion of the streams according to the noise condition of the given speech datum. Experimental results demonstrate that the proposed method is effective for producing noise-robust recognition performance without a priori knowledge about the noise conditions of the speech data.
Jong-Seok Lee, Touradj Ebrahimi
Added 25 May 2010
Updated 25 May 2010
Type Conference
Year 2009
Authors Jong-Seok Lee, Touradj Ebrahimi
Comments (0)