A multiplicative up-propagation algorithm

We present a generalization of the nonnegative matrix factorization (NMF), where a multilayer generative network with nonnegative weights is used to approximate the observed nonnegative data. The multilayer generative network with nonnegativity constraints, is learned by a multiplicative uppropagation algorithm, where the weights in each layer are updated in a multiplicative fashion while the mismatch ratio is propagated from the bottom to the top layer. The monotonic convergence of the multiplicative up-propagation algorithm is shown. In contrast to NMF, the multiplicative uppropagation is an algorithm that can learn hierarchical representations, where complex higher-level representations are defined in terms of less complex lower-level representations. The interesting behavior of our algorithm is demonstrated with face image data.

• Type: Conference
• Year: 2004
• Where: ICML
• Authors: Jong-Hoon Ahn, Seungjin Choi, Jong-Hoon Oh
• Keywords: ICML 2004, Machine Learning, Multilayer Generative Network, Multiplicative Up-propagation Algorithm, Multiplicative Uppropagation Algorithm