PLSA-based image auto-annotation: constraining the latent space

We address the problem of unsupervised image auto-annotation with probabilistic latent space models. Unlike most previous works, which build latent space representations assuming equal relevance for the text and visual modalities, we propose a new way of modeling multi-modal co-occurrences, constraining the definition of the latent space to ensure its consistency in semantic terms (words), while retaining the ability to jointly model visual information. The concept is implemented by a linked pair of Probabilistic Latent Semantic Analysis (PLSA) models. On a 16000-image collection, we show with extensive experiments that our approach significantly outperforms previous joint models. Categories and Subject Descriptors H.3.1 [Information Storage and Retrieval]: Content Analysis and Indexing—Indexing methods General Terms Algorithms, Theory, Languages Keywords Automatic Annotation of Images, Semantic Indexing, PLSA

• Type: Conference
• Year: 2004
• Where: MM
• Authors: Florent Monay, Daniel Gatica-Perez
• Keywords: Latent Space, MM 2004, Probabilistic Latent Semantic Analysis, Probabilistic Latent Space