Sciweavers

Share
IUI
2005
ACM

Multimodal new vocabulary recognition through speech and handwriting in a whiteboard scheduling application

10 years 3 months ago
Multimodal new vocabulary recognition through speech and handwriting in a whiteboard scheduling application
Our goal is to automatically recognize and enroll new vocabulary in a multimodal interface. To accomplish this our technique aims to leverage the mutually disambiguating aspects of co-referenced, co-temporal handwriting and speech. The co-referenced semantics are spatially and temporally determined by our multimodal interface for schedule chart creation. This paper motivates and describes our technique for recognizing out-of-vocabulary (OOV) terms and enrolling them dynamically in the system. We report results for the detection and segmentation of OOV words within a small multimodal test set. On the same test set we also report utterance, word and pronunciation level error rates both over individual input modes and multimodally. We show that combining information from handwriting and speech yields significantly better results than achievable by either mode alone. Categories and Subject Descriptors I.2.6 [Artificial Intelligence]: Learning – language acquisition, knowledge acquisitio...
Edward C. Kaiser
Added 26 Jun 2010
Updated 26 Jun 2010
Type Conference
Year 2005
Where IUI
Authors Edward C. Kaiser
Comments (0)
books