A user interface framework for multimodal VR interactions

13 years 10 months ago
A user interface framework for multimodal VR interactions
This article presents a User Interface (UI) framework for multimodal interactions targeted at immersive virtual environments. Its configurable input and gesture processing components provide an advanced behavior graph capable of routing continuous data streams asynchronously. The framework introduces a Knowledge Representation Layer which augments objects of the simulated environment with Semantic Entities as a central object model that bridges and interfaces Virtual Reality (VR) and Artificial Intelligence (AI) representations. Specialized node types use these facilities to implement required processing tasks like gesture detection, preprocessing of the visual scene for multimodal integration, or translation of movements into multimodally initialized gestural interactions. A modified Augmented Transition Nettwork (ATN) approach accesses the knowledge layer as well as the preprocessing components to integrate linguistic, gestural, and context information in parallel. The overall fr...
Marc Erich Latoschik
Added 27 Jun 2010
Updated 27 Jun 2010
Type Conference
Year 2005
Where ICMI
Authors Marc Erich Latoschik
Comments (0)