Processing Data-Stream Join Aggregates Using Skimmed Sketches

There is a growing interest in on-line algorithms for analyzing and querying data streams, that examine each stream element only once and have at their disposal, only a limited amount of memory. Providing (perhaps approximate) answers to aggregate queries over such streams is a crucial requirement for many application environments; examples include large IP network installations where performance data from different parts of the network needs to be continuously collected and analyzed. In this paper, we present the skimmed-sketch algorithm for estimating the join size of two streams. (Our techniques also readily extend to other join-aggregate queries.) To the best of our knowledge, our skimmed-sketch technique is the first comprehensive join-size estimation algorithm to provide tight error guarantees while: (1) achieving the lower bound on the space required by any join-size estimation method in a streaming environment, (2) handling streams containing general update operations (inserts ...

• Type: Conference
• Year: 2004
• Where: EDBT
• Authors: Sumit Ganguly, Minos N. Garofalakis, Rajeev Rastogi
• Keywords: Database, Dense Frequencies, EDBT 2004, Join-size Estimation Algorithm, Synthetic Data Streams