Sciweavers

NECO
2007

Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance

13 years 2 months ago
Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance
It is well-known that, in unidentifiable models, the Bayes estimation provides much better generalization performance than the maximum likelihood (ML) estimation. However, its accurate approximation by Markov chain Monte Carlo methods requires huge computational costs. As an alternative, a tractable approximation method, called the variational Bayes (VB) approach, has recently been proposed and been attracting people’s attention. Its advantage over the expectation maximization (EM) algorithm, often used for realizing the ML estimation, has been experimentally shown in many applications, nevertheless, has not been theoretically shown yet. In this paper, through the analysis of the simplest unidentifiable models, we theoretically show some properties of the VB approach. We first prove that, in three-layer linear neural networks, the VB approach is asymptotically equivalent to a positive-part James-Stein type shrinkage estimation. Then, we theoretically clarify its free energy, gene...
Shinichi Nakajima, Sumio Watanabe
Added 27 Dec 2010
Updated 27 Dec 2010
Type Journal
Year 2007
Where NECO
Authors Shinichi Nakajima, Sumio Watanabe
Comments (0)