Finding Best k Policies

Abstract. An optimal probabilistic-planning algorithm solves a problem, usually modeled by a Markov decision process, by finding its optimal policy. In this paper, we study the k best policies problem. The problem is to find the k best policies. The k best policies, k > 1, cannot be found directly using dynamic programming. Na¨ıvely, finding the k-th best policy can be Turing reduced to the optimal planning problem, but the number of problems queried in the na¨ıve algorithm is exponential in k. We show empirically that solving k best policy problem by using this reduction requires unreasonable amounts of time even when k = 3. We then provide a new algorithm, based on our theoretical contribution to prove that the k-th best policy differs from the i-th policy, for some i < k, on exactly one state. We show that the time complexity of the algorithm is quadratic in k, but the number of optimal planning problems it solves is linear in k. We demonstrate empirically that the new...

• Type: Conference
• Year: 2009
• Where: ALDT
• Authors: Peng Dai, Judy Goldsmith
• Keywords: ALDT 2009, Algorithmic Decision Theory, Optimal Planning Problems, Optimal Policy, Optimal Probabilistic-planning Algorithm