Adaptive Aggregation for Reinforcement Learning with Efficient Exploration: Deterministic Domains

We propose a model-based learning algorithm, the Adaptive Aggregation Algorithm (AAA), that aims to solve the online, continuous state space reinforcement learning problem in a deterministic domain. The proposed algorithm uses an adaptive state aggregation approach, going from coarse to fine grids over the state space, which enables to use finer resolution in the "important" areas of the state space, and coarser resolution elsewhere. We consider an on-line learning approach, in which we discover these important areas on-line, using an uncertainty intervals exploration technique. Polynomial learning rates in terms of mistake bound (in a PAC framework) are established for this algorithm, under appropriate continuity assumptions.

• Type: Conference
• Year: 2008
• Where: COLT
• Authors: Andrey Bernstein, Nahum Shimkin
• Keywords: Adaptive Aggregation Algorithm, Adaptive State Aggregation, COLT 2008, Machine Learning, State Space