Sciweavers

AIIA
2007
Springer

Reinforcement Learning in Complex Environments Through Multiple Adaptive Partitions

13 years 9 months ago
Reinforcement Learning in Complex Environments Through Multiple Adaptive Partitions
The application of Reinforcement Learning (RL) algorithms to learn tasks for robots is often limited by the large dimension of the state space, which may make prohibitive its application on a tabular model. In this paper, we introduce LEAP (Learning Entities Adaptive Partitioning), a model-free learning algorithm that uses overlapping partitions which are dynamically modified to learn near-optimal policies with a small number of parameters. Starting from a coarse aggregation of the state space, LEAP generates refined partitions whenever it detects an incoherence between the current action values and the actual rewards from the environment. Since in highly stochastic problems the adaptive process can lead to over-refinement, we introduce a mechanism that prunes the macrostates without affecting the learned policy. Through refinement and pruning, LEAP builds a multi-resolution state representation specialized only where it is actually needed. In the last section, we present some exp...
Andrea Bonarini, Alessandro Lazaric, Marcello Rest
Added 07 Jun 2010
Updated 07 Jun 2010
Type Conference
Year 2007
Where AIIA
Authors Andrea Bonarini, Alessandro Lazaric, Marcello Restelli
Comments (0)