Sciweavers

SIGMOD
2001
ACM

Independence is Good: Dependency-Based Histogram Synopses for High-Dimensional Data

14 years 3 months ago
Independence is Good: Dependency-Based Histogram Synopses for High-Dimensional Data
Approximating the joint data distribution of a multi-dimensional data set through a compact and accurate histogram synopsis is a fundamental problem arising in numerous practical scenarios, including query optimization and approximate query answering. Existing solutions either rely on simplistic independence assumptions or try to directly approximate the full joint data distribution over the complete set of attributes. Unfortunately, both approaches are doomed to fail for high-dimensional data sets with complex correlation patterns between attributes. In this paper, we propose a novel approach to histogram-based synopses that employs the solid foundation of statistical interaction models to explicitly identify and exploit the stacharacteristics of the data. Abstractly, our key idea is to break the synopsis into (1) a statistical interaction model that accurately captures significant correlation and independence patterns in data, and (2) a collection of histograms on low-dimensional ma...
Amol Deshpande, Minos N. Garofalakis, Rajeev Rasto
Added 08 Dec 2009
Updated 08 Dec 2009
Type Conference
Year 2001
Where SIGMOD
Authors Amol Deshpande, Minos N. Garofalakis, Rajeev Rastogi
Comments (0)