Sciweavers

Share
ICMI
2010
Springer

Modelling and analyzing multimodal dyadic interactions using social networks

8 years 11 months ago
Modelling and analyzing multimodal dyadic interactions using social networks
Social network analysis became a common technique used to model and quantify the properties of social interactions. In this paper, we propose an integrated framework to explore the characteristics of a social network extracted from multimodal dyadic interactions. First, speech detection is performed through an audio/visual fusion scheme based on stacked sequential learning. In the audio domain, speech is detected through clusterization of audio features. Clusters are modelled by means of an One-state Hidden Markov Model containing a diagonal covariance Gaussian Mixture Model. In the visual domain, speech detection is performed through differential-based feature extraction from the segmented mouth region, and a dynamic programming matching procedure. Second, in order to model the dyadic interactions, we employed the Influence Model whose states encode the previous integrated audio/visual data. Third, the social network is extracted based on the estimated influences. For our study, we u...
Sergio Escalera, Petia Radeva, Jordi Vitrià
Added 12 Feb 2011
Updated 12 Feb 2011
Type Journal
Year 2010
Where ICMI
Authors Sergio Escalera, Petia Radeva, Jordi Vitrià, Xavier Baró, Bogdan Raducanu
Comments (0)
books