Soundtrack classification by transient events

We present a method for video classification based on information in the soundtrack. Unlike previous approaches which describe the audio via statistics of mel-frequency cepstral coefficient (MFCC) features calculated on uniformlyspaced frames, we investigate an approach to focusing our representation on audio transients corresponding to soundtrack events. These event-related features can reflect the “foreground” of the soundtrack and capture its short-term temporal structure better than conventional frame-based statistics. We evaluate our method on a test set of 1873 YouTube videos labeled with 25 semantic concepts. Retrieval results based on transient features alone are comparable to an MFCC-based system, and fusing the two representations achieves a relative improvement of 7.5% in mean average precision (MAP).

• Type: Journal
• Year: 2011
• Where: ICASSP
• Authors: Courtenay V. Cotton, Daniel P. W. Ellis, Alexander C. Loui
• Keywords: ICASSP 2011, Mean Average Precision, Short-term Temporal Structure, Signal Processing, Soundtrack