A density projection approach to dimension reduction for continuous-state POMDPs

Abstract— Research on numerical solution methods for partially observable Markov decision processes (POMDPs) has primarily focused on discrete-state models, and these algorithms do not generally extend to continuous-state POMDPs, due to the infinite dimensionality of the belief space. In this paper, we develop a computationally viable and theoretically sound method for solving continuous-state POMDPs by effectively reducing the dimensionality of the belief space via density projection. The density projection technique is also incorporated into particle filtering to provide a filtering scheme for online decision making. We provide an error bound between the value function induced by the policy obtained by our method and the true value function of the POMDP. Finally, we illustrate the effectiveness of our method through an inventory control problem.

• Type: Conference
• Year: 2008
• Where: CDC
• Authors: Enlu Zhou, Michael C. Fu, Steven I. Marcus
• Keywords: Belief Space, CDC 2008, Continuous-state Pomdps, Control Systems, Density Projection