Sciweavers

Share
JMLR
2010

Tempered Markov Chain Monte Carlo for training of Restricted Boltzmann Machines

8 years 9 months ago
Tempered Markov Chain Monte Carlo for training of Restricted Boltzmann Machines
Alternating Gibbs sampling is the most common scheme used for sampling from Restricted Boltzmann Machines (RBM), a crucial component in deep architectures such as Deep Belief Networks. However, we find that it often does a very poor job of rendering the diversity of modes captured by the trained model. We suspect that this hinders the advantage that could in principle be brought by training algorithms relying on Gibbs sampling for uncovering spurious modes, such as the Persistent Contrastive Divergence algorithm. To alleviate this problem, we explore the use of tempered Markov Chain Monte-Carlo for sampling in RBMs. We find both through visualization of samples and measures of likelihood that it helps both sampling and learning.
Guillaume Desjardins, Aaron C. Courville, Yoshua B
Added 19 May 2011
Updated 19 May 2011
Type Journal
Year 2010
Where JMLR
Authors Guillaume Desjardins, Aaron C. Courville, Yoshua Bengio, Pascal Vincent, Olivier Delalleau
Comments (0)
books