Sciweavers

MM
2010
ACM

Sonify your face: facial expressions for sound generation

14 years 2 days ago
Sonify your face: facial expressions for sound generation
We present a novel visual creativity tool that automatically recognizes facial expressions and tracks facial muscle movements in real time to produce sounds. The facial expression recognition module detects and tracks a face and outputs a feature vector of motions of specific locations in the face. The feature vector is used as input to a Bayesian network which classifies facial expressions into several categories (e.g., angry, disgusted, happy, etc.). The classification results are used along with the feature vector to generate a combination of sounds that change in real time depending on the person`s facial expressions. We explain the artistic motivation behind the work, the basic components of our tool, and possible applications in the arts (performance, installation) and in the medical domain. Finally, we report on the experience of approximately 25 users of our system at a conference demonstration session, of 9 participants in a pilot study to assess the system`s usability, and d...
Roberto Valenti, Alejandro Jaimes, Nicu Sebe
Added 06 Dec 2010
Updated 06 Dec 2010
Type Conference
Year 2010
Where MM
Authors Roberto Valenti, Alejandro Jaimes, Nicu Sebe
Comments (0)