Interpreting Pointing Gestures and Spoken Requests - A Probabilistic, Salience-based Approach

We present a probabilistic, salience-based approach to the interpretation of pointing gestures together with spoken utterances. Our mechanism models dependencies between spatial and temporal aspects of gestures and features of utterances. For our evaluation, we collected a corpus of requests which optionally included pointing. Our results show that pointing information improves interpretation accuracy.

• Type: Journal
• Year: 2010
• Where: COLING
• Authors: Ingrid Zukerman, Gideon Kowadlo, Patrick Ye
• Keywords: COLING 2010, Computational Linguistics, Gestures, Mechanism Models, Temporal Aspects