Sciweavers

Share
ICONIP
2007

Natural Conjugate Gradient in Variational Inference

8 years 8 months ago
Natural Conjugate Gradient in Variational Inference
Variational methods for approximate inference in machine learning often adapt a parametric probability distribution to optimize a given objective function. This view is especially useful when applying variational Bayes (VB) to models outside the conjugate-exponential family. For them, variational Bayesian expectation maximization (VB EM) algorithms are not easily available, and gradient-based methods are often used as alternatives. Traditional natural gradient methods use the Riemannian structure (or geometry) of the predictive distribution to speed up maximum likelihood estimation. We propose using the geometry of the variational approximating distribution instead to speed up a conjugate gradient method for variational learning and inference. The computational overhead is small due to the simplicity of the approximating distribution. Experiments with real-world speech data show significant speedups over alternative learning algorithms.
Antti Honkela, Matti Tornio, Tapani Raiko, Juha Ka
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2007
Where ICONIP
Authors Antti Honkela, Matti Tornio, Tapani Raiko, Juha Karhunen
Comments (0)
books