Sciweavers

Share
AMAI
2006
Springer

Symmetric approximate linear programming for factored MDPs with application to constrained problems

8 years 8 months ago
Symmetric approximate linear programming for factored MDPs with application to constrained problems
A weakness of classical Markov decision processes (MDPs) is that they scale very poorly due to the flat state-space representation. Factored MDPs address this representational problem by exploiting problem structure to specify the transition and reward functions of an MDP in a compact manner. However, in general, solutions to factored MDPs do not retain the structure and compactness of the problem representation, forcing approximate solutions, with approximate linear programming (ALP) emerging as a promising MDPapproximation technique. To date, most ALP work has focused on the primal-LP formulation, while the dual LP, which forms the basis for solving constrained Markov problems, has received much less attention. We show that a straightforward linear approximation of the dual optimization variables is problematic, because some of the required computations cannot be carried out efficiently. Nonetheless, we develop a composite approach that symmetrically approximates the primal and dual...
Dmitri A. Dolgov, Edmund H. Durfee
Added 10 Dec 2010
Updated 10 Dec 2010
Type Journal
Year 2006
Where AMAI
Authors Dmitri A. Dolgov, Edmund H. Durfee
Comments (0)
books