Learning Generative Models with the Up-Propagation Algorithm

10 years 5 months ago
Learning Generative Models with the Up-Propagation Algorithm
Up-propagation is an algorithm for inverting and learning neural network generative models. Sensory input is processed by inverting a model that generates patterns from hidden variables using top-down connections. The inversion process is iterative, utilizing a negative feedback loop that depends on an error signal propagated by bottom-up connections. The error signal is also used to learn the generative model from examples. The algorithm is benchmarked against principal component analysis in experiments on images of handwritten digits. In his doctrine of unconscious inference, Helmholtz argued that perceptions are formed by the interaction of bottom-up sensory data with top-down expectations. According to one interpretation of this doctrine, perception is a procedure of sequential hypothesis testing. We propose a new algorithm, called up-propagation, that realizes this interpretation in layered neural networks. It uses top-down connections to generate hypotheses, and bottom-up connec...
Jong-Hoon Oh, H. Sebastian Seung
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 1997
Where NIPS
Authors Jong-Hoon Oh, H. Sebastian Seung
Comments (0)