Value Function Approximation on Non-Linear Manifolds for Robot Motor Control

— The least squares approach works efficiently in value function approximation, given appropriate basis functions. Because of its smoothness, the Gaussian kernel is a popular and useful choice as a basis function. However, it does not allow for discontinuity which typically arises in realworld reinforcement learning tasks. In this paper, we propose a new basis function based on geodesic Gaussian kernels, which exploits the non-linear manifold structure induced by the Markov decision processes. The usefulness of the proposed method is successfully demonstrated in a simulated robot arm control and Khepera robot navigation.

• Type: Conference
• Year: 2007
• Where: ICRA
• Authors: Masashi Sugiyama, Hirotaka Hachiya, Christopher Towell, Sethu Vijayakumar
• Keywords: Basis Function, Gaussian Kernels, ICRA 2007, Robotics, Value Function Approximation