Perspectives on Sparse Bayesian Learning

Recently, relevance vector machines (RVM) have been fashioned from a sparse Bayesian learning (SBL) framework to perform supervised learning using a weight prior that encourages sparsity of representation. The methodology incorporates an additional set of hyperparameters governing the prior, one for each weight, and then adopts a specific approximation to the full marginalization over all weights and hyperparameters. Despite its empirical success however, no rigorous motivation for this particular approximation is currently available. To address this issue, we demonstrate that SBL can be recast as the application of a rigorous variational approximation to the full model by expressing the prior in a dual form. This formulation obviates the necessity of assuming any hyperpriors and leads to natural, intuitive explanations of why sparsity is achieved in practice.

• Type: Conference
• Year: 2003
• Where: NIPS
• Authors: David P. Wipf, Jason A. Palmer, Bhaskar D. Rao
• Keywords: Approximation, NIPS 2003, NIPS 2007, Relevance Vector Machines, Sparse Bayesian Learning