Solving Concurrent Markov Decision Processes

Typically, Markov decision problems (MDPs) assume a single action is executed per decision epoch, but in the real world one may frequently execute certain actions in parallel. This paper explores concurrent MDPs, MDPs which allow multiple non-conflicting actions to be executed simultaneously, and presents two new algorithms. Our first approach exploits two provably sound pruning rules, and thus guarantees solution optimality. Our second technique is a fast, samplingbased algorithm, which produces close-to-optimal solutions extremely quickly. Experiments show that our approaches outperform the existing algorithms producing up to two orders of magnitude speedup.

• Type: Conference
• Year: 2004
• Where: AAAI
• Authors: Mausam, Daniel S. Weld
• Keywords: AAAI 2004, Guarantees Solution Optimality, Intelligent Agents, Markov Decision Problems, Multiple Non-conflicting Actions