The Role of Occam's Razor in Knowledge Discovery

13 years 5 months ago
The Role of Occam's Razor in Knowledge Discovery
Many KDD systems incorporate an implicit or explicit preference for simpler models, but this use of "Occam's razor" has been strongly criticized by several authors (e.g., Schaffer, 1993; Webb, 1996). This controversy arises partly because Occam's razor has been interpreted in two quite different ways. The first interpretation (simplicity is a goal in itself) is essentially correct, but is at heart a preference for more comprehensible models. The second interpretation (simplicity leads to greater accuracy) is much more problematic. A critical review of the theoretical arguments for and against it shows that it is unfounded as a universal principle, and demonstrably false. A review of empirical evidence shows that it also fails as a practical heuristic. This article argues that its continued use in KDD risks causing significant opportunities to be missed, and should therefore be restricted to the comparatively few applications where it is appropriate. The article prop...
Pedro Domingos
Added 22 Dec 2010
Updated 22 Dec 2010
Type Journal
Year 1999
Authors Pedro Domingos
Comments (0)