Connectivity, Dynamics, and Memory in Reservoir Computing with Binary and Analog Neurons

Abstract: Reservoir Computing (RC) systems are powerful models for online computations on input sequences. They consist of a memoryless readout neuron which is trained on top of a randomly connected recurrent neural network. RC systems are commonly used in two flavors: with analog or binary (spiking) neurons in the recurrent circuits. Previous work indicated a fundamental difference in the behavior of these two implementations of the RC idea. The performance of a RC system built from binary neurons seems to depend strongly on the network connectivity structure. In networks of analog neurons such clear dependency has not been observed. In this article we address this apparent dichotomy by investigating the influence of the network connectivity (parametrized by the neuron in-degree) on a family of network models that interpolates between analog and binary networks. Our analyses are based on a novel estimation of the Lyapunov exponent of the network dynamics with the help of branching p...

• Type: Journal
• Year: 2010
• Where: NECO
• Authors: Lars Büsing, Benjamin Schrauwen, Robert A. Legenstein
• Keywords: NECO 2010, Network Connectivity, Neuron, Recurrent Neural Network