Closed-form dual perturb and combine for tree-based models

This paper studies the aggregation of predictions made by tree-based models for several perturbed versions of the attribute vector of a test case. A closed-form approximation of this scheme combined with cross-validation to tune the level of perturbation is proposed. This yields soft-tree models in a parameter free way, and preserves their interpretability. Empirical evaluations, on classification and regression problems, show that accuracy and bias/variance tradeoff are improved significantly at the price of an acceptable computational overhead. The method is further compared and combined with tree bagging.

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Pierre Geurts, Louis Wehenkel
• Keywords: Acceptable Computational Overhead, ICML 2005, Machine Learning, Parameter Free Way, Perturbed Versions