Simultaneous Segmentation and 3D Reconstruction of Monocular Image Sequences

When trying to extract 3D scene information and camera motion from an image sequence alone, it is often necessary to cope with independently moving objects. Recent research has unveiled some of the mathematical foundations of the problem, but a general and practical algorithm, which can handle long, realistic sequences, is still missing. In this paper, we identify the necessary parts of such an algorithm, highlight both unexplored theoretical issues and practical challenges, and propose solutions. Theoretical issues include proper handling of different situations, in which the number of independent motions changes: objects can enter the scene, objects previously moving together can split and follow independent trajectories, or independently moving objects can merge into one common motion. We derive model scoring criteria to handle these changes in the number of segments. A further theoretical issue is the resolution of the relative scale ambiguity between such changes. Practical issue...

• Type: Conference
• Year: 2007
• Where: ICCV
• Authors: Kemal Egemen Ozden, Konrad Schindler, Luc J. Van Gool
• Keywords: Computer Vision, Foreground Objects, ICCV 2007, Theoretical Issue, Unexplored Theoretical Issues