Sciweavers

Share
SI3D
2016
ACM

Barehanded music: real-time hand interaction for virtual piano

3 years 6 months ago
Barehanded music: real-time hand interaction for virtual piano
This paper presents an efficient data-driven approach to track fingertip and detect finger tapping for virtual piano using an RGB-D camera. We collect 7200 depth images covering the most common finger articulation for playing piano, and train a random regression forest using depth context features of randomly sampled pixels in training images. In the online tracking stage, we firstly segment the hand from the plane in contact by fusing the information from both color and depth images. Then we use the trained random forest to estimate the 3D position of fingertips and wrist in each frame, and predict finger tapping based on the estimated fingertip motion. Finally, we build a kinematic chain and recover the articulation parameters for each finger. In contrast to the existing hand tracking algorithms that often require hands are in the air and cannot interact with physical objects, our method is designed for hand interaction with planar objects, which is desired for the virtual ...
Hui Liang, Jin Wang, Qian Sun, Yong-Jin Liu, Junso
Added 09 Apr 2016
Updated 09 Apr 2016
Type Journal
Year 2016
Where SI3D
Authors Hui Liang, Jin Wang, Qian Sun, Yong-Jin Liu, Junsong Yuan, Jun Luo, Ying He
Comments (0)
books