Estimating arbitrary subset sums with few probes

Suppose we have a large table T of items i, each with a weight wi, e.g., people and their salary. In a general preprocessing step for estimating arbitrary subset sums, we assign each item a random priority depending on its weight. Suppose we want to estimate the sum of an arbitrary subset I T. For any q > 2, considering only the q highest priority items from I, we obtain an unbiased estimator of the sum whose relative standard deviation is O(1/ q). Thus to get an expected approximation factor of 1 ? , it suffices to consider O(1/2 ) items from I. Our estimator needs no knowledge of the number of items in the subset I, but we can also estimate that number if we want to estimate averages. The above scheme performs the same role as the on-line aggregation of Hellerstein et al. (SIGMOD'97) but it has the advantage of having expected good performance for any possible sequence of weights. In particular, the performance does not deteriorate in the common case of heavy-tailed weight...

• Type: Conference
• Year: 2005
• Where: PODS
• Authors: Noga Alon, Nick G. Duffield, Carsten Lund, Mikkel Thorup
• Keywords: Arbitrary Subset Sums, Database, General Preprocessing Step, PODS 2005, Specific Permission And/