Towards Semantic Embedding in Visual Vocabulary

9 years 4 months ago
Towards Semantic Embedding in Visual Vocabulary
Visual vocabulary serves as a fundamental component in many computer vision tasks, such as object recognition, visual search, and scene modeling. While state-of-the-art approaches build visual vocabulary based solely on visual statistics of local image patches, the correlative image labels are left unexploited in generating visual words. In this work, we present a semantic embedding framework to integrate semantic information from Flickr labels for supervised vocabulary construction. Our main contribution is a Hidden Markov Random Field modeling to supervise feature space quantization, with specialized considerations to label correlations: Local visual features are modeled as an Observed Field, which follows visual metrics to partition feature space. Semantic labels are modeled as a Hidden Field, which imposes generative supervision to the Observed Field with WordNet-based correlation constraints as Gibbs distribution. By simplifying the Markov property in the Hidden Field, both unsup...
R.-R. Ji, Hongxun Yao, Xiaoshuai Sun
Added 23 Jun 2010
Updated 23 Jun 2010
Type Conference
Year 2010
Where CVPR
Authors R.-R. Ji, Hongxun Yao, Xiaoshuai Sun
Comments (0)