Sciweavers

Free Online Productivity Tools i2Speak i2Symbol i2OCR iTex2Img iWeb2Print iWeb2Shot i2Type iPdf2Split iPdf2Merge i2Bopomofo i2Arabic i2Style i2Image i2PDF iLatex2Rtf Sci2ools

15

PRICAI
2000
Springer

favoriteEmaildiscussreport

193views Artificial Intelligence» more PRICAI 2000»

Generating Hierarchical Structure in Reinforcement Learning from State Variables

13 years 7 months ago

Generating Hierarchical Structure in Reinforcement Learning from State Variables

Download www.csee.umbc.edu

This paper presents the CQ algorithm which decomposes and solves a Markov Decision Process (MDP) by automatically generating a hierarchy of smaller MDPs using state variables. The CQ algorithm uses a heuristic which is applicable for problems that can be modelled by a set of state variables that conform to a special ordering, defined in this paper as a "nested Markov ordering". The benefits of this approach are: (1) the automatic generation of actions and termination conditions at all levels in the hierarchy, and (2) linear scaling with the number of variables under certain conditions. This approach draws heavily on Dietterich's MAXQ value function decomposition and Hauskrecht, Meuleau, Kaelbling, Dean, Boutilier's and others region based decomposition of MDPs. The CQ algorithm is described and its functionality illustrated using a four room example. Different solutions are generated with different numbers of hierarchical levels to solve Dietterich's taxi tasks...

Bernhard Hengst

Real-time Traffic

Artificial Intelligence | CQ Algorithm | Markov Decision Process | PRICAI 2000 | State Variables |

claim paper

Related Content

» Discovering Hierarchy in Reinforcement Learning with HEXQ

» Efficient Behavior Learning Based on State Value Estimation of Self and Others

» Scaling ant colony optimization with hierarchical reinforcement learning partitioning

» A causal approach to hierarchical decomposition of factored MDPs

» Anticipatory Learning Classifier Systems and Factored Reinforcement Learning

» Using background knowledge to speed reinforcement learning in physical agents

» Bayesian MultiTask Reinforcement Learning

» Protovalue functions developmental reinforcement learning

» Genetic Network Programming with Reinforcement Learning and Its Performance Evaluation

Post Info
More Details (n/a)

Added	25 Aug 2010
Updated	25 Aug 2010
Type	Conference
Year	2000
Where	PRICAI
Authors	Bernhard Hengst

Comments (0)