Discourse Topic and Gestural Form

10 years 1 months ago
Discourse Topic and Gestural Form
Coverbal gesture provides a channel for the visual expression of ideas. While some gestural emblems have culturally predefined forms (e.g., "thumbs up"), the relationship between gesture and meaning is, in general, not conventionalized. It is natural to ask whether such gestures can be interpreted in a speaker-independent way, or whether gestural form is determined by the speaker's idiosyncratic view of the discourse topic. We address this question using an audiovisual dataset across multiple speakers and topics. Our analysis employs a hierarchical Bayesian author-topic model, in which gestural patterns are stochastically generated by a mixture of speaker-specific and topic-specific priors. These gestural patterns are characterized using automaticallyextracted visual features, based on spatio-temporal interest points. This framework detects significant crossspeaker patterns in gesture that are governed by the discourse topic, suggesting that even unstructured gesticulat...
Jacob Eisenstein, Regina Barzilay, Randall Davis
Added 02 Oct 2010
Updated 02 Oct 2010
Type Conference
Year 2008
Where AAAI
Authors Jacob Eisenstein, Regina Barzilay, Randall Davis
Comments (0)