Avatar-mediated face tracking and lip reading for human computer interaction

Advanced human computer interaction requires automatic reading of human face in order to make the computer interact with human in the same way as human-to-human communication. We developed an automatic face tracking and lip reading system through a 3D face avatar to facilitate HCI applications in speech learning, emotional state monitoring, and non-verbal human computer interface design. The system implements a novel active face feature tracking algorithm with an uncalibrated camera. The 3D face pose is estimated and tracked by a Kalman filter-based matching process with a dynamic face model updating and constraint. The obtained facial motion parameters are transferred to an individualized 3D face avatar. As a result, a person’s lip shape or expressions can be cloned to the animated 3D face avatar, by which all lip shapes from the same speech of different subjects can be easily compared and measured. This real time system targets the automatic facial expression analysis and synthe...

• Type: Conference
• Year: 2004
• Where: MM
• Authors: Xiaozhou Wei, Lijun Yin, Zhiwei Zhu, Qiang Ji
• Keywords: Automatic Face Tracking, Face Avatar, Human Face, MM 2004