Efficient Approximation of Optimal Control for Markov Games

The success of probabilistic model checking for discrete-time Markov decision processes and continuous-time Markov chains has led to rich academic and industrial applications. The analysis of their combination in continuous-time Markov decisions processes, however, is currently restricted to toy examples. This is due to the fact that current analysis techniques for time-bounded reachability require a running time linear in the reciprocal -1 of the required precision . For the high precision usually sought (for example, six to ten digits), this simply renders these techniques infeasible. We discuss a surprising combination of discretisation and partial unravelling, which leads to memoryful near optimal schedulers that can be computed in time linear only in the square or cube root of -1 . The proposed techniques also reduce the dependency on the expected number of discrete transitions within the given time bound significantly. Our techniques naturally extend to the analysis of continuous...

• Type: Journal
• Year: 2010
• Where: CORR
• Authors: Markus Rabe, Sven Schewe, Lijun Zhang
• Keywords: Continuous-time Markov, Continuous-time Markov Chains, CORR 2010, Education, Markov Decisions