Cross-Media Alignment of Names and Faces

In this paper we report on our experiments on aligning names and faces as found in images and captions of online news websites. Developing accurate technologies for linking names and faces is valuable when retrieving or mining information from multimedia collections. We perform exhaustive and systematic experiments exploiting the (a)symmetry between the visual and textual modalities. This leads to different schemes for assigning names to the faces, assigning faces to the names, and establishing name-face link pairs. On top of that, we investigate generic approaches to the use of textual and visual structural information to predict the presence of the corresponding entity in the other modality. The proposed methods are completely unsupervised and are inspired by methods for aligning phrases and words in texts of different languages developed for constructing dictionaries for machine translation. The results are competitive with state of the art performance on the "Labeled Faces in ...

• Type: Journal
• Year: 2010
• Where: TMM
• Authors: Phi The Pham, Marie-Francine Moens, Tinne Tuytelaars
• Keywords: Labeled Faces, Management, Textual Modalities, TMM 2010, Visual Structural Information