Sciweavers

Share
JMLR
2010

A Generalized Path Integral Control Approach to Reinforcement Learning

7 years 10 months ago
A Generalized Path Integral Control Approach to Reinforcement Learning
With the goal to generate more scalable algorithms with higher efficiency and fewer open parameters, reinforcement learning (RL) has recently moved towards combining classical techniques from optimal control and dynamic programming with modern learning techniques from statistical estimation theory. In this vein, this paper suggests to use the framework of stochastic optimal control with path integrals to derive a novel approach to RL with parameterized policies. While solidly grounded in value function estimation and optimal control based on the stochastic Hamilton-JacobiBellman (HJB) equations, policy improvements can be transformed into an approximation problem of a path integral which has no open algorithmic parameters other than the exploration noise. The resulting algorithm can be conceived of as model-based, semi-model-based, or even model free, depending on how the learning problem is structured. The update equations have no danger of numerical instabilities as neither matrix i...
Evangelos Theodorou, Jonas Buchli, Stefan Schaal
Added 19 May 2011
Updated 19 May 2011
Type Journal
Year 2010
Where JMLR
Authors Evangelos Theodorou, Jonas Buchli, Stefan Schaal
Comments (0)
books