Sciweavers

JAIR
2006

Cognitive Principles in Robust Multimodal Interpretation

13 years 5 months ago
Cognitive Principles in Robust Multimodal Interpretation
Multimodal conversational interfaces provide a natural means for users to communicate with computer systems through multiple modalities such as speech and gesture. To build effective multimodal interfaces, automated interpretation of user multimodal inputs is important. Inspired by the previous investigation on cognitive status in multimodal human machine interaction, we have developed a greedy algorithm for interpreting user referring expressions (i.e., multimodal reference resolution). This algorithm incorporates the cognitive principles of Conversational Implicature and Givenness Hierarchy and applies constraints from various sources (e.g., temporal, semantic, and contextual) to resolve references. Our empirical results have shown the advantage of this algorithm in efficiently resolving a variety of user references. Because of its simplicity and generality, this approach has the potential to improve the robustness of multimodal input interpretation.
Joyce Yue Chai, Zahar Prasov, Shaolin Qu
Added 13 Dec 2010
Updated 13 Dec 2010
Type Journal
Year 2006
Where JAIR
Authors Joyce Yue Chai, Zahar Prasov, Shaolin Qu
Comments (0)