Sciweavers

Share
PRICAI
2000
Springer

Generating Hierarchical Structure in Reinforcement Learning from State Variables

9 years 10 months ago
Generating Hierarchical Structure in Reinforcement Learning from State Variables
This paper presents the CQ algorithm which decomposes and solves a Markov Decision Process (MDP) by automatically generating a hierarchy of smaller MDPs using state variables. The CQ algorithm uses a heuristic which is applicable for problems that can be modelled by a set of state variables that conform to a special ordering, defined in this paper as a "nested Markov ordering". The benefits of this approach are: (1) the automatic generation of actions and termination conditions at all levels in the hierarchy, and (2) linear scaling with the number of variables under certain conditions. This approach draws heavily on Dietterich's MAXQ value function decomposition and Hauskrecht, Meuleau, Kaelbling, Dean, Boutilier's and others region based decomposition of MDPs. The CQ algorithm is described and its functionality illustrated using a four room example. Different solutions are generated with different numbers of hierarchical levels to solve Dietterich's taxi tasks...
Bernhard Hengst
Added 25 Aug 2010
Updated 25 Aug 2010
Type Conference
Year 2000
Where PRICAI
Authors Bernhard Hengst
Comments (0)
books