Sciweavers

JMLR
2002

Learning Precise Timing with LSTM Recurrent Networks

13 years 3 months ago
Learning Precise Timing with LSTM Recurrent Networks
The temporal distance between events conveys information essential for numerous sequential tasks such as motor control and rhythm detection. While Hidden Markov Models tend to ignore this information, recurrent neural networks (RNNs) can in principle learn to make use of it. We focus on Long Short-Term Memory (LSTM) because it has been shown to outperform other RNNs on tasks involving long time lags. We find that LSTM augmented by "peephole connections" from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars. Without external resets or teacher forcing, our LSTM variant also learns to generate stable streams of precisely timed spikes and other highly nonlinear periodic patterns. This makes LSTM a promising approach for tasks that require the accurate measurement or generation of time intervals.
Felix A. Gers, Nicol N. Schraudolph, Jürgen S
Added 22 Dec 2010
Updated 22 Dec 2010
Type Journal
Year 2002
Where JMLR
Authors Felix A. Gers, Nicol N. Schraudolph, Jürgen Schmidhuber
Comments (0)