A Bayesian Metric for Evaluating Machine Learning Algorithms

10 years 7 months ago
A Bayesian Metric for Evaluating Machine Learning Algorithms
How to assess the performance of machine learning algorithms is a problem of increasing interest and urgency as the data mining application of myriad algorithms grows. The standard approach of employing predictive accuracy has, we argue rightly, been losing favor in the AI community. The alternative of cost-sensitive metrics provides a far better approach, given the availability of useful cost functions. For situations where no useful cost function can be found we need other alternatives to predictive accuracy. We propose that information-theoretic reward functions be applied. The first such proposal for assessing specifically machine learning algorithms was made by Kononenko and Bratko [1]. Here we improve upon our alternative Bayesian metric [2], which provides a fair betting assessment of any machine learner. We include an empirical analysis of various Bayesian classification learners, ranging from Naive Bayes learners to causal discovery algorithms.
Lucas R. Hope, Kevin B. Korb
Added 01 Jul 2010
Updated 01 Jul 2010
Type Conference
Year 2004
Authors Lucas R. Hope, Kevin B. Korb
Comments (0)