Occam's Razor Just Got Sharper

8 years 11 months ago
Occam's Razor Just Got Sharper
Occam’s razor is the principle that, given two hypotheses consistent with the observed data, the simpler one should be preferred. Many machine learning algorithms follow this principle and search for a small hypothesis within the version space. The principle has been the subject of a heated debate with theoretical and empirical arguments both for and against it. Earlier empirical studies lacked sufficient coverage to resolve the debate. In this work we provide convincing empirical evidence for Occam’s razor in the context of decision tree induction. By applying a variety of sophisticated sampling techniques, our methodology samples the version space for many real-world domains and tests the correlation between the size of a tree and its accuracy. We show that indeed a smaller tree is likely to be more accurate, and that this correlation is statistically significant across most domains.
Saher Esmeir, Shaul Markovitch
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2007
Authors Saher Esmeir, Shaul Markovitch
Comments (0)