Sciweavers

Share
ICML
2006
IEEE

Learning the structure of Factored Markov Decision Processes in reinforcement learning problems

10 years 3 months ago
Learning the structure of Factored Markov Decision Processes in reinforcement learning problems
Recent decision-theoric planning algorithms are able to find optimal solutions in large problems, using Factored Markov Decision Processes (fmdps). However, these algorithms need a perfect knowledge of the structure of the problem. In this paper, we propose sdyna, a general framework for addressing large reinforcement learning problems by trial-and-error and with no initial knowledge of their structure. sdyna integrates incremental planning algorithms based on fmdps with supervised learning techniques building structured representations of the problem. We describe spiti, an instantiation of sdyna, that uses incremental decision tree induction to learn the structure of a problem combined with an incremental version of the Structured Value Iteration algorithm. We show that spiti can build a factored representation of a reinforcement learning problem and may improve the policy faster than tabular reinforcement learning algorithms by exploiting the generalization property of decision tree...
Thomas Degris, Olivier Sigaud, Pierre-Henri Wuille
Added 17 Nov 2009
Updated 17 Nov 2009
Type Conference
Year 2006
Where ICML
Authors Thomas Degris, Olivier Sigaud, Pierre-Henri Wuillemin
Comments (0)
books