Data collection for both training and testing a classifier is a tedious but essential step towards face detection and recognition. It is a piece of cake to collect more than hundreds of thousands of examples from web and digital camera nowadays. How to train a face detector based on the collected immense face database? This paper presents a manifold-based method to select a training set. That is to say we learn the manifold from the collected enormous face database and then subsample and interweave the training set by the estimated geodesic distance in the low-dimensional manifold embedding. By the resulting training set, we train an AdaBoost-based face detector. The trained detector is tested on the MIT+CMU frontal face test set. The experimental results show that the proposed method based on the manifold is efficient to train a classifier confronted with the huge database.