Shannon Information and Kolmogorov Complexity

9 years 10 months ago
Shannon Information and Kolmogorov Complexity
The elementary theories of Shannon information and Kolmogorov complexity are cmpared, the extent to which they have a common purpose, and where they are fundamentally different. The focus is on: Shannon entropy versus Kolmogorov complexity, the relation of both to universal coding, Shannon mutual information versus Kolmogorov (`algorithmic') mutual information, probabilistic sufficient statistic versus algorithmic sufficient statistic (related to lossy compression in the Shannon theory versus meaningful information in the Kolmogorov theory), and rate distortion theory versus Kolmogorov's structure function. Part of the material has appeared in print before, scattered through various publications, but this is the first comprehensive systematic comparison. The last mentioned relations are new. Contents
Peter Grünwald, Paul M. B. Vitányi
Added 17 Dec 2010
Updated 17 Dec 2010
Type Journal
Year 2004
Where CORR
Authors Peter Grünwald, Paul M. B. Vitányi
Comments (0)