Sciweavers

Share
NN
2015
Springer

Measuring the usefulness of hidden units in Boltzmann machines with mutual information

5 years 7 months ago
Measuring the usefulness of hidden units in Boltzmann machines with mutual information
Abstract. Restricted Boltzmann machines (RBMs) and deep Boltzmann machines (DBMs) are important models in deep learning, but it is often difficult to measure their performance in general, or measure the importance of individual hidden units in specific. We propose to use mutual information to measure the usefulness of individual hidden units in Boltzmann machines. The measure serves as an upper bound for the information the neuron can pass on, enabling detection of a particular kind of poor training results. We confirm experimentally, that the proposed measure is telling how much the performance of the model drops when some of the units of an RBM are pruned away. Our experiments on DBMs highlight differences among different pretraining options.
Mathias Berglund, Tapani Raiko, KyungHyun Cho
Added 15 Apr 2016
Updated 15 Apr 2016
Type Journal
Year 2015
Where NN
Authors Mathias Berglund, Tapani Raiko, KyungHyun Cho
Comments (0)
books