Reinforcement learning of a continuous motor sequence with hidden states

—Reinforcement learning is the scheme for unsupervised learning in which robots are expected to acquire behavior skills through self-explorations based on reward signals. There are some difficulties, however, in applying conventional reinforcement learning algorithms to motion control tasks of a robot because most algorithms are concerned with discrete state space and based on the assumption of complete observability of the state. Real-world environments often have partial observablility; therefore, robots have to estimate the unobservable hidden states. This paper proposes a method to solve these two problems by combining the reinforcement learning algorithm and a learning algorithm for a continuous time recurrent neural network (CTRNN). The CTRNN can learn spatiotemporal structures in a continuous time and space domain, and can preserve the contextual flow by a self-organizing appropriate internal memory structure. This enables the robot to deal with the hidden state problem. We ...

• Type: Journal
• Year: 2007
• Where: AR
• Authors: Hiroaki Arie, Tetsuya Ogata, Jun Tani, Shigeki Sugano
• Keywords: Algorithm, AR 2007, Hidden State, Robot