Join-distinct aggregate estimation over update streams

There is growing interest in algorithms for processing and querying continuous data streams (i.e., data that is seen only once in a fixed order) with limited memory resources. Providing (perhaps approximate) answers to queries over such streams is a crucial requirement for many application environments; examples include large IP network installations where performance data from different parts of the network needs to be continuously collected and analyzed. The ability to estimate the number of distinct (sub)tuples in the result of a join operation correlating two data streams (i.e., the cardinality of a projection with duplicate elimination over a join) is an important requirement for several data-analysis scenarios. For instance, to enable real-time traffic analysis and load balancing, a network-monitoring application may need to estimate the number of distinct (source, destination) IP-address pairs occurring in the stream of IP packets observed by router R1, where the source address...

• Type: Conference
• Year: 2005
• Where: PODS
• Authors: Sumit Ganguly, Minos N. Garofalakis, Amit Kumar, Rajeev Rastogi
• Keywords: Continuous Data Streams, Database, General Update Streams, Individual Update Streams, PODS 2005