On Unique Decodability

11 years 1 months ago
On Unique Decodability
In this paper, we propose a revisitation of the topic of unique decodability and of some fundamental theorems of lossless coding. It is widely believed that, for any discrete source X, every "uniquely decodable" block code satisfies E[l(X1;X2;. . . ;Xn)] H(X1;X2;. . . ;Xn) where X1;X2;. . . ;Xn are the first n symbols of the source, E[l(X1;X2;. . . ;Xn)] is the expected length of the code for those symbols, and H(X1;X2;. . . ;Xn) is their joint entropy. We show that, for certain sources with memory, the above inequality only holds when a limiting definition of "uniquely decodable code" is considered. In particular, the above inequality is usually assumed to hold for any "practical code" due to a debatable application of McMillan's theorem to sources with memory. We thus propose a clarification of the topic, also providing an extended version of McMillan's theorem to be used for Markovian sources.
Marco Dalai, Riccardo Leonardi
Added 09 Dec 2010
Updated 09 Dec 2010
Type Journal
Year 2008
Where CORR
Authors Marco Dalai, Riccardo Leonardi
Comments (0)