Decision Tree Methods for Finding Reusable MDP Homomorphisms

straction is a useful tool for agents interacting with environments. Good state abstractions are compact, reuseable, and easy to learn from sample data. This paper and extends two existing classes of state abstraction methods to achieve these criteria. The first class of methods search for MDP homomorphisms (Ravindran 2004), which produce models of reward and transition probabilities in an state space. The second class of methods, like the UTree algorithm (McCallum 1995), learn compact models of the value function quickly from sample data. Models based on MDP homomorphisms can easily be extended such that they are usable across tasks with similar reward functions. However, value based methods like UTree cannot be extended in this fashion. We present results showing a new, combined algorithm that fulfills all three criteria: the resulting models are compact, can be learned quickly from sample data, and can be used across a class of reward functions.

• Type: Conference
• Year: 2006
• Where: AAAI
• Authors: Alicia P. Wolfe, Andrew G. Barto
• Keywords: AAAI 2006, Intelligent Agents, MDP Homomorphisms, Sample Data, State Abstraction