Video coding based on audio-visual attention

This paper proposes an efficient video coding method based on audio-visual attention, which is motivated by the fact that cross-modal interaction significantly affects humans' perception of multimedia content. First, we propose an audio-visual source localization method to locate the sound source in a video sequence. Then, its result is used for applying spatial blurring to video frames in order to reduce redundant high-frequency information and achieve coding efficiency. We demonstrate the effectiveness of the proposed method for H.264/AVC coding along with the results of a subjective evaluation.

• Type: Journal
• Year: 2009
• Where: ICMCS
• Authors: Jong-Seok Lee, Francesca De Simone, Touradj Ebrahimi
• Keywords: Audio-visual Attention, Audio-visual Source Localization, ICMCS 2009, Multimedia, Video