A Pareto model for OLAP view size estimation

13 years 7 months ago
A Pareto model for OLAP view size estimation
On Line Analytical Processing (OLAP) aims at gaining useful information quickly from large amounts of data residing in a data warehouse. To improve the quickness of response to queries, preaggregation is a useful strategy. However, it is usually impossible to pre-aggregate along all combinations of the dimensions. The multidimensional aspects of the data lead to combinatorial explosion in the number and potential storage size of the aggregates. We must selectively pre-aggregate. Cost/benefit analysis involves estimating the storage requirements of the aggregates in question. We present an original algorithm for estimating the number of rows in an aggregate based on the Pareto distribution model. We test the Pareto Model Algorithm empirically against three published algorithms, and conclude the Pareto Model Algorithm is consistently the best of these algorithms for estimating view size. 1 Motivation Accumulation of data in industry and organizations has led to large archives of data in...
Thomas P. Nadeau, Toby J. Teorey
Added 31 Oct 2010
Updated 31 Oct 2010
Type Conference
Year 2001
Authors Thomas P. Nadeau, Toby J. Teorey
Comments (0)