Multimodal Tracking for Smart Videoconferencing and Video Surveillance

Many applications require the ability to track the 3-D motion of the subjects. We build a particle filter based framework for multimodal tracking using multiple cameras and multiple microphone arrays. In order to calibrate the resulting system, we propose a method to determine the locations of all microphones using at least five loudspeakers and under assumption that for each loudspeaker there exists a microphone very close to it. We derive the maximum likelihood (ML) estimator, which reduces to the solution of the non-linear least squares problem. We verify the correctness and robustness of the multimodal tracker and of the self-calibration algorithm both with Monte-Carlo simulations and on real data from three experimental setups.

• Type: Conference
• Year: 2007
• Where: CVPR
• Authors: Dmitry N. Zotkin, Vikas C. Raykar, Ramani Duraiswami, Larry S. Davis
• Keywords: Computer Vision, CVPR 2007, Filter Based Framework, Multimodal Tracker, Multimodal Tracking, Multiple Microphone Arrays, Self-calibration Algorithm