Inter-Operator Feedback in Data Stream Management Systems via Punctuation

10 years 11 months ago
Inter-Operator Feedback in Data Stream Management Systems via Punctuation
High-volume, high-speed data streams may overwhelm the capabilities of stream processing systems; techniques such as data prioritization, avoidance of unnecessary processing and ondemand result production may be necessary to reduce processing requirements. However, the dynamic nature of data streams, in terms of both rate and content, makes the application of such techniques challenging. Such techniques have been addressed in the context of static and centralized query optimization; however, they have not been fully addressed for data-stream management systems. In this work, we present a comprehensive framework designed to support prioritization, avoidance of unnecessary work, and on-demand result production over distributed, unreliable, bursty, disordered data sources, typical of many streams. We propose a form of inter-operator feedback, which flows against the stream direction, to communicate the information needed to enable execution of these techniques. This feedback leverages pu...
Rafael Fernández-Moctezuma, Kristin Tufte,
Added 08 Nov 2010
Updated 08 Nov 2010
Type Conference
Year 2009
Where CIDR
Authors Rafael Fernández-Moctezuma, Kristin Tufte, Jin Li
Comments (0)