Model Minimization in Markov Decision Processes

10 years 7 months ago
Model Minimization in Markov Decision Processes
Many stochastic planning problems can be represented using Markov Decision Processes (MDPs). A difficulty with using these MDP representations is that the common algorithms for solving them run in time polynomial in the size of the state space, where this size is extremely large for most real-world planning problems of interest. Recent AI research has addressed this problem by representing the MDP in a factored form. Factored MDPs, however, are not amenable to traditional solution methods that call for an explicit enumeration of the state space. One familiar way to solve MDP problems with very large state spaces is to form a reduced (or aggregated) MDP with the same properties as the original MDP by combining “equivalent” states. In this paper, we discuss applying this approach to solving factored MDP problems— we avoid enumerating the state space by describing large blocks of “equivalent” states in factored form, with the block descriptions being inferred directly from the ...
Thomas Dean, Robert Givan
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 1997
Where AAAI
Authors Thomas Dean, Robert Givan
Comments (0)