Sciweavers

Share
CDC
2009
IEEE

A simulation-based method for aggregating Markov chains

11 years 6 months ago
A simulation-based method for aggregating Markov chains
— This paper addresses model reduction for a Markov chain on a large state space. A simulation-based framework is introduced to perform state aggregation of the Markov chain based on observations of a single sample path. The Kullback-Leibler (K-L) divergence rate is employed as a metric to measure the distance between two stationary Markov chains. Model reduction with respect to this metric is cast as an infinite-horizon average cost optimal control problem. In this way an optimal policy corresponds to an optimal partition of the state space with respect to the K-L divergence rate. The optimal control problem is simplified in an approximate dynamic programming (ADP) framework: First, a relaxation of the policy space is performed, and based on this a parameterization of the set of optimal policies is introduced. This makes possible a stochastic approximation approach to compute the best policy within a given parameterized class. The algorithm can be implemented using a single sample...
Kun Deng, Prashant G. Mehta, Sean P. Meyn
Added 21 Jul 2010
Updated 21 Jul 2010
Type Conference
Year 2009
Where CDC
Authors Kun Deng, Prashant G. Mehta, Sean P. Meyn
Comments (0)
books