On Finding Predictors for Arbitrary Families of Processes

The problem is sequence prediction in the following setting. A sequence x1, . . . , xn, . . . of discrete-valued observations is generated according to some unknown probabilistic law (measure) µ. After observing each outcome, it is required to give the conditional probabilities of the next observation. The measure µ belongs to an arbitrary but known class C of stochastic process measures. We are interested in predictors ρ whose conditional probabilities converge (in some sense) to the “true” µ-conditional probabilities, if any µ ∈ C is chosen to generate the sequence. The contribution of this work is in characterizing the families C for which such predictors exist, and in providing a specific and simple form in which to look for a solution. We show that if any predictor works, then there exists a Bayesian predictor, whose prior is discrete, and which works too. We also find several sufficient and necessary conditions for the existence of a predictor, in terms of topologic...

• Type: Journal
• Year: 2010
• Where: JMLR
• Authors: Daniil Ryabko
• Keywords: Conditional Probabilities, JMLR 2010, Sequence Prediction, Unknown Probabilistic Law