NP-Hardness of checking the unichain condition in average cost MDPs

The unichain condition requires that every policy in an MDP result in a single ergodic class, and guarantees that the optimal average cost is independent of the initial state. We show that checking whether the unichain condition fails to hold is an NP-complete problem. We conclude with a brief discussion of the merits of the more general weak accessibility condition. © 2006 Elsevier B.V. All rights reserved.

• Type: Journal
• Year: 2007
• Where: ORL
• Authors: John N. Tsitsiklis
• Keywords: Optimal Average Cost, ORL 2007, Single Ergodic Class, Unichain Condition