Sciweavers

Share
PROMISE
2010

Replication of defect prediction studies: problems, pitfalls and recommendations

8 years 5 months ago
Replication of defect prediction studies: problems, pitfalls and recommendations
Background: The main goal of the PROMISE repository is to enable reproducible, and thus verifiable or refutable research. Over time, plenty of data sets became available, especially for defect prediction problems. Aims: In this study, we investigate possible problems and pitfalls that occur during replication. This information can be used for future replication studies, and serve as a guideline for researchers reporting novel results. Method: We replicate two recent defect prediction studies comparing different data sets and learning algorithms, and report missing information and problems. Results: Even with access to the original data sets, replicating previous studies may not lead to the exact same results. The choice of evaluation procedures, performance measures and presentation has a large influence on the reproducibility. Additionally, we show that trivial and random models can be used to identify overly optimistic evaluation measures. Conclusions: The best way to conduct easily...
Thilo Mende
Added 20 May 2011
Updated 20 May 2011
Type Journal
Year 2010
Where PROMISE
Authors Thilo Mende
Comments (0)
books