We present in this paper a formal approach for the representation of multimodal information. This approach, thanks to the to use of typed feature structures and hypergraphs, gener...
Multimodal grammars provide an expressive formalism for multimodal integration and understanding. However, handcrafted multimodal grammars can be brittle with respect to unexpecte...
Combinatory Categorial Grammar (CCG) is generally construed as a fully lexicalized formalism, where all grammars use one and the same universal set of rules, and crosslinguistic v...
In orderto realize their full potential, multimodalsystems need to support not just input from multiple modes, but also synchronized integration of modes. Johnston et al (1997) mo...
It has always been difficult for language understanding systems to handle spontaneous speech with satisfactory robustness, primarily due to such problems as the fragments, disflue...
Bor-shen Lin, Berlin Chen, Hsin-Min Wang, Lin-Shan...