A Randomized ANOVA Procedure for Comparing Performance Curves

Three factors are related in analyses of performance curves such as learning curves: the amount of training, the learning algorithm, and performance. Often we want to know whether the algorithm affects performance and whether the effect of training on performance depends on the algorithm. Analysis of variance would be an ideal technique but for carryover effects, which violate the assumptions of parametric analysis of variance and can produce dramatic increases in Type I errors. We propose a novel, randomized version of the two-way analysis of variance which avoids this problem. In experiments we analyze Type I errors and the power of our technique, using common machine learning datasets.

• Type: Conference
• Year: 1998
• Where: ICML
• Authors: Justus H. Piater, Paul R. Cohen, Xiaoqin Zhang, Michael Atighetchi
• Keywords: Algorithm Affects Performance, ICML 1998, Machine Learning, Machine Learning Datasets, Performance Curves