Following the Flattened Leader

10 years 9 months ago
Following the Flattened Leader
We analyze the regret, measured in terms of log loss, of the maximum likelihood (ML) sequential prediction strategy. This "follow the leader" strategy also defines one of the main versions of Minimum Description Length model selection. We proved in prior work for single parameter exponential family models that (a) in the misspecified case, the redundancy of follow-the-leader is not 1 2 log n+O(1), as it is for other universal prediction strategies; as such, the strategy also yields suboptimal individual sequence regret and inferior model selection performance; and (b) that in general it is not possible to achieve the optimal redundancy when predictions are constrained to the distributions in the considered model. Here we describe a simple "flattening" of the sequential ML and related predictors, that does achieve the optimal worst case individual sequence regret of (k/2) log n + O(1) for k parameter exponential family models for bounded outcome spaces; for unbounde...
Wojciech Kotlowski, Peter Grünwald, Steven de
Added 10 Feb 2011
Updated 10 Feb 2011
Type Journal
Year 2010
Where COLT
Authors Wojciech Kotlowski, Peter Grünwald, Steven de Rooij
Comments (0)