Solving Factored MDPs with Exponential-Family Transition Models

10 years 3 months ago
Solving Factored MDPs with Exponential-Family Transition Models
Markov decision processes (MDPs) with discrete and continuous state and action components can be solved efficiently by hybrid approximate linear programming (HALP). The main idea of the approach is to approximate the optimal value function by a linear combination of basis functions and optimize it by linear programming. In this paper, we extend the existing HALP paradigm beyond the mixture of beta transition model. As a result, we permit modeling of other transition functions, such as normal and gamma densities, without approximating them. To allow for efficient solutions to the expectation terms in HALP, we identify a rich class of conjugate basis functions. Finally, we demonstrate the generalized HALP framework on a rover planning problem, which exhibits continuous time and resource uncertainty.
Branislav Kveton, Milos Hauskrecht
Added 30 Oct 2010
Updated 30 Oct 2010
Type Conference
Year 2006
Where AIPS
Authors Branislav Kveton, Milos Hauskrecht
Comments (0)