This paper presents a novel face recognition method based on cascade Linear Discriminant Analysis (LDA) of the component-based face representation. In the proposed method, a face image is represented as four components with overlap at the neighboring area rather than a whole face patch. Firstly, LDA is conducted on the principal components of each component individually to extract component discriminant features. Then, these features are further concatenated to undergo another LDA to extract the final face descriptor, which actually have assigned different weights to different component features. Our experiments on the FERET face database have illustrated the effectiveness of the proposed method compared with the traditional Fisherface method both for face recognition and verification.