Sufficient Dimension Reduction via Squared-loss Mutual Information Estimation

The goal of sufficient dimension reduction in supervised learning is to find the lowdimensional subspace of input features that is `sufficient' for predicting output values. In this paper, we propose a novel sufficient dimension reduction method using a squaredloss variant of mutual information as a dependency measure. We utilize an analytic approximator of squared-loss mutual information based on density ratio estimation, which is shown to possess suitable convergence properties. We then develop a natural gradient algorithm for sufficient subspace search. Numerical experiments show that the proposed method compares favorably with existing dimension reduction approaches.

• Type: Journal
• Year: 2010
• Where: JMLR
• Authors: Taiji Suzuki, Masashi Sugiyama
• Keywords: JMLR 2010, Mutual Information, Sufficient Dimension Reduction, Sufficient Subspace Search