In this paper we propose a voting-based object boundary reconstruction approach. Tensor voting has been studied by many people recently, and it can be used for boundary estimation on curves or irregular trajectories. However, the complexity of saliency tensor creation limits its applications in real-time systems. In order to have an efficient solution, we introduce an alternative voting approach. Rather than creating saliency tensors, we use a “2-pass” method for orientation estimation. For the first pass, Sobel detector is applied on a coarse boundary image to get the gradient map, then the orientation information is updated by accumulating votes on the corresponding direction. In the second pass, edge linking is performed based on the pixels orientation map, and extra lines are eliminated by detecting intersections. The approach has been applied to various video clips under different conditions, and the experimental results demonstrate significant improvement on the final extrac...