Sciweavers

Share
KDD
2009
ACM

Issues in evaluation of stream learning algorithms

12 years 2 months ago
Issues in evaluation of stream learning algorithms
Learning from data streams is a research area of increasing importance. Nowadays, several stream learning algorithms have been developed. Most of them learn decision models that continuously evolve over time, run in resource-aware environments, detect and react to changes in the environment generating data. One important issue, not yet conveniently addressed, is the design of experimental work to evaluate and compare decision models that evolve over time. There are no golden standards for assessing performance in non-stationary environments. This paper proposes a general framework for assessing predictive stream learning algorithms. We defend the use of Predictive Sequential methods for error estimate – the prequential error. The prequential error allows us to monitor the evolution of the performance of models that evolve over time. Nevertheless, it is known to be a pessimistic estimator in comparison to holdout estimates. To obtain more reliable estimators we need some forgetting m...
João Gama, Raquel Sebastião, Pedro P
Added 26 Jul 2010
Updated 26 Jul 2010
Type Conference
Year 2009
Where KDD
Authors João Gama, Raquel Sebastião, Pedro Pereira Rodrigues
Comments (0)
books