Multidimensional Triangulation and Interpolation for Reinforcement Learning

9 years 8 months ago
Multidimensional Triangulation and Interpolation for Reinforcement Learning
Dynamic Programming, Q-learning and other discrete Markov Decision Process solvers can be applied to continuous d-dimensional state-spaces by quantizing the state space into an array of boxes. This is often problematic above two dimensions: a coarse quantization can lead to poor policies, and ne quantization is too expensive. Possible solutions are variable-resolution discretization, or function approximation by neural nets. A third option, which has been little studied in the reinforcement learning literature, is interpolation on a coarse grid. In this paper we study interpolation techniques that can result in vast improvements in the online behavior of the resulting control systems: multilinear interpolation, and an interpolation algorithm based on an interesting regular triangulation of d-dimensional space. We adapt these interpolators under three reinforcement learning paradigms: (i) o ine value iteration with a known model, (ii) Q-learning, and (iii) online value iteration with a ...
Scott Davies
Added 02 Nov 2010
Updated 02 Nov 2010
Type Conference
Year 1996
Where NIPS
Authors Scott Davies
Comments (0)