Using additive expert ensembles to cope with concept drift

We consider online learning where the target concept can change over time. Previous work on expert prediction algorithms has bounded the worst-case performance on any subsequence of the training data relative to the performance of the best expert. However, because these "experts" may be difficult to implement, we take a more general approach and bound performance relative to the actual performance of any online learner on this single subsequence. We present the additive expert ensemble algorithm AddExp, a new, general method for using any online learner for drifting concepts. We adapt techniques for analyzing expert prediction algorithms to prove mistake and loss bounds for a discrete and a continuous version of AddExp. Finally, we present pruning methods and empirical results for data sets with concept drift.

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Jeremy Z. Kolter, Marcus A. Maloof
• Keywords: Additive Expert, Expert Prediction Algorithms, ICML 2005, Machine Learning, Online Learner