Limit Theorems for the Sample Entropy of Hidden Markov Chains

The Shannon-McMillan-Breiman theorem asserts that the sample entropy of a stationary and ergodic stochastic process converges to the entropy rate of the same process almost surely. In this paper, we focus our attention on the convergence behavior of the sample entropy of a hidden Markov chain. Under certain positivity assumption, we prove that a central limit theorem (CLT) with some Berry-Esseen bound (such bound characterizes rate of convergence of CLT) for the sample entropy of a hidden Markov chain, and we use this CLT to establish a law of iterated logarithm (LIL) for the sample entropy.

• Type: Journal
• Year: 2011
• Where: CORR
• Authors: Guangyue Han
• Keywords: CORR 2011, Education, Ergodic Stochastic Process, Hidden Markov Chain, Sample Entropy