Labeling objects in images is an essential prerequisite for many visual learning and recognition applications that depend on training data, such as image retrieval, object detection and recognition. Manually creating labels in images is not only time-consuming but also subject to human labeling errors, and eventually, becomes impossible for a large scale image database. Semi-supervised learning (SSL) algorithms such as Gaussian random field (GRF) can be applied to labeling objects in images since they have the ability to include a large amount of unlabeled data while requiring only a small amount of labeled data. However, the one-shot property of GRF prevents it from achieving good labeling performance. In this paper, we presents a novel object labeling tool, SmartLabel, to semi-automatically label objects in images. The algorithm of SmartLabel has four innovations over GRF: 1) soft labeling, 2) graph construction with spatial constraints, 3) iterated harmonic energy minimization, an...