Sciweavers

Share
MM
2010
ACM

A new approach to cross-modal multimedia retrieval

10 years 3 months ago
A new approach to cross-modal multimedia retrieval
The problem of joint modeling the text and image components of multimedia documents is studied. The text component is represented as a sample from a hidden topic model, learned with latent Dirichlet allocation, and images are represented as bags of visual (SIFT) features. Two hypotheses are investigated: that 1) there is a benefit to explicitly modeling correlations between the two components, and 2) this modeling is more effective in feature spaces with higher levbstraction. Correlations between the two components are learned with canonical correlation analysis. Abstraction is achieved by representing text and images at a more general, semantic level. The two hypotheses are studied in the context of the task of cross-modal document retrieval. This includes retrieving the text that most closely matches a query image, or retrieving the images that most closely match a query text. It is shown that accounting for crossrrelations and semantic abstraction both improve retrieval accuracy. T...
Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Cov
Added 06 Dec 2010
Updated 06 Dec 2010
Type Conference
Year 2010
Where MM
Authors Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert R. G. Lanckriet, Roger Levy, Nuno Vasconcelos
Comments (0)
books