Adaptive Ordering of Pipelined Stream Filters

We consider the problem of pipelined filters, where a continuous stream of tuples is processed by a set of commutative filters. Pipelined filters are common in stream applications and capture a large class of multiway stream joins. We focus on the problem of ordering the filters adaptively to minimize processing cost in an environment where stream and filter characteristics vary unpredictably over time. Our core algorithm, A-Greedy (for Adaptive Greedy), has strong theoretical guarantees: If stream and filter characteristics were to stabilize, A-Greedy would converge to an ordering within a small constant factor of optimal. (In experiments A-Greedy usually converges to the optimal ordering.) One very important feature of A-Greedy is that it monitors and responds to selectivities that are correlated across filters (i.e., that are nonindependent), which provides the strong quality guarantee but incurs run-time overhead. We identify a three-way tradeoff among provable convergence to good...

• Type: Conference
• Year: 2004
• Where: SIGMOD
• Authors: Shivnath Babu, Rajeev Motwani, Kamesh Munagala, Itaru Nishizawa, Jennifer Widom
• Keywords: Continuous Stream, Database, Multiway Stream Joins, SIGMOD 2004, Stream Prototype Data