Efficient Discovery of Confounders in Large Data Sets

10 years 7 months ago
Efficient Discovery of Confounders in Large Data Sets
Given a large transaction database, association analysis is concerned with efficiently finding strongly related objects. Unlike traditional associate analysis, where relationships among variables are searched at a global level, we examine confounding factors at a local level. Indeed, many real-world phenomena are localized to specific regions and times. These relationships may not be visible when the entire data set is analyzed. Specially, confounding effects that change the direction of correlation is the most significant. Along this line, we propose to efficiently find confounding effects attributable to local associations. Specifically, we derive an upper bound by a necessary condition of confounders, which can help us prune the search space and efficiently identify confounders. Experimental results show that the proposed CONFOUND algorithm can effectively identify confounders and the computational performance is an order of magnitude faster than benchmark methods. Keywords- Correla...
Wenjun Zhou, Hui Xiong
Added 18 Feb 2011
Updated 18 Feb 2011
Type Journal
Year 2009
Where ICDM
Authors Wenjun Zhou, Hui Xiong
Comments (0)