Improving Stability of Decision Trees

13 years 6 months ago
Improving Stability of Decision Trees
Decision-tree algorithms are known to be unstable: small variations in the training set can result in different trees and different predictions for the same validation examples. Both accuracy and stability can be improved by learning multiple models from bootstrap samples of training data, but the "meta-learner" approach makes the extracted knowledge hardly interpretable. In the following paper, we present the Info-Fuzzy Network (IFN), a novel information-theoretic method for building stable and comprehensible decision-tree models. The stability of the IFN algorithm is ensured by restricting the tree structure to using the same feature for all nodes of the same tree level and by the built-in statistical significance tests. The IFN method is shown empirically to produce more compact and stable models than the "meta-learner" techniques, while preserving a reasonable level of predictive accuracy. Keywords. Decision trees, info-fuzzy network, stability, similarity, cla...
Mark Last, Oded Maimon, Einat Minkov
Added 22 Dec 2010
Updated 22 Dec 2010
Type Journal
Year 2002
Authors Mark Last, Oded Maimon, Einat Minkov
Comments (0)