Feature-Discovering Approximate Value Iteration Methods

10 years 12 months ago
Feature-Discovering Approximate Value Iteration Methods
Sets of features in Markov decision processes can play a critical role ximately representing value and in abstracting the state space. Selection of features is crucial to the success of a system and is most often conducted by a human. We study the problem of automatically selecting problem features, and propose and evaluate a simple approach reducing the problem of selecting a new feature to standard classification learning. We learn a classifier that predicts the sign of the Bellman error over a training set of states. By iteratively adding new classifiers as features with this method, training between iterations with approximate value iteration, we find a Tetris feature set that outperforms randomly constructed features significantly, and obtains a score of about three-tenths of the highest score obtained by using a carefully hand-constructed feature set. We also show that features learned with this method outperform those learned with the previous method of Patrascu et al. [4] ...
Jia-Hong Wu, Robert Givan
Added 28 Jun 2010
Updated 28 Jun 2010
Type Conference
Year 2005
Where SARA
Authors Jia-Hong Wu, Robert Givan
Comments (0)