Sciweavers

Share
AAAI
1997

Incremental Methods for Computing Bounds in Partially Observable Markov Decision Processes

10 years 5 months ago
Incremental Methods for Computing Bounds in Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) allow one to model complex dynamic decision or control problems that include both action outcome uncertainty and imperfect observability. The control problem is formulated as a dynamic optimization problem with a value function combining costs or rewards from multiple steps. In this paper we propose, analyse and test various incremental methods for computing bounds on the value function for control problems with infinite discounted horizon criteria. The methods described and tested include novel incremental versions of grid-based linear interpolation method and simple lower bound method with Sondik’s updates. Both of these can work with arbitrary points of the belief space and can be enhanced by various heuristic point selection strategies. Also introduced is a new method for computing an initial upper bound – the fast informed bound method. This method is able to improve significantly on the standard and commonly usedupper...
Milos Hauskrecht
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 1997
Where AAAI
Authors Milos Hauskrecht
Comments (0)
books