Sciweavers

Share
JAIR
2008

Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction

9 years 3 months ago
Gesture Salience as a Hidden Variable for Coreference Resolution and Keyframe Extraction
Gesture is a non-verbal modality that can contribute crucial information to the understanding of natural language. But not all gestures are informative, and non-communicative hand motions may confuse natural language processing (NLP) and impede learning. People have little difficulty ignoring irrelevant hand movements and focusing on meaningful gestures, suggesting that an automatic system could also be trained to perform this task. However, the informativeness of a gesture is context-dependent and labeling enough data to cover all cases would be expensive. We present conditional modality fusion, a conditional hidden-variable model that learns to predict which gestures are salient for coreference resolution, the task of determining whether two noun phrases refer to the same semantic entity. Moreover, our approach uses only coreference annotations, and not annotations of gesture salience itself. We show that gesture features improve performance on coreference resolution, and that by at...
Jacob Eisenstein, Regina Barzilay, Randall Davis
Added 12 Dec 2010
Updated 12 Dec 2010
Type Journal
Year 2008
Where JAIR
Authors Jacob Eisenstein, Regina Barzilay, Randall Davis
Comments (0)
books