Sciweavers

Share
JMLR
2012

Bayesian Comparison of Machine Learning Algorithms on Single and Multiple Datasets

8 years 6 months ago
Bayesian Comparison of Machine Learning Algorithms on Single and Multiple Datasets
We propose a new method for comparing learning algorithms on multiple tasks which is based on a novel non-parametric test that we call the Poisson binomial test. The key aspect of this work is that we provide a formal definition for what is meant to have an algorithm that is better than another. Also, we are able to take into account the dependencies induced when evaluating classifiers on the same test set. Finally we make optimal use (in the Bayesian sense) of all the testing data we have. We demonstrate empirically that our approach is more reliable than the sign test and the Wilcoxon signed rank test, the current state of the art for algorithm comparisons.
Alexandre Lacoste, François Laviolette, Mar
Added 27 Sep 2012
Updated 27 Sep 2012
Type Journal
Year 2012
Where JMLR
Authors Alexandre Lacoste, François Laviolette, Mario Marchand
Comments (0)
books