Sciweavers

Share
ICANN
2009
Springer

An EM Based Training Algorithm for Recurrent Neural Networks

10 years 4 months ago
An EM Based Training Algorithm for Recurrent Neural Networks
Recurrent neural networks serve as black-box models for nonlinear dynamical systems identification and time series prediction. Training of recurrent networks typically minimizes the quadratic difference of the network output and an observed time series. This implicitely assumes that the dynamics of the underlying system is deterministic, which is not a realistic assumption in many cases. In contrast, statespace models allow for noise in both the internal state transitions and the mapping from internal states to observations. Here, we consider recurrent networks as nonlinear state space models and suggest a training algorithm based on Expectation-Maximization. A nonlinear transfer function for the hidden neurons leads to an intractable inference problem. We investigate the use of a Particle Smoother to approximate the E-step and simultaneously estimate the expectations required in the Mstep. The method is demonstrated for a sythetic data set and a time series prediction task arising i...
Jan Unkelbach, Yi Sun, Jürgen Schmidhuber
Added 25 Jul 2010
Updated 25 Jul 2010
Type Conference
Year 2009
Where ICANN
Authors Jan Unkelbach, Yi Sun, Jürgen Schmidhuber
Comments (0)
books