Sciweavers

PPOPP
2003
ACM

Optimizing data aggregation for cluster-based internet services

13 years 9 months ago
Optimizing data aggregation for cluster-based internet services
Large-scale cluster-based Internet services often host partitioned datasets to provide incremental scalability. The aggregation of results produced from multiple partitions is a fundamental building block for the delivery of these services. This paper presents the design and implementation of a programming primitive – Data Aggregation Call (DAC) – to exploit partition parallelism for clusterbased Internet services. A DAC request specifies a local processing operator and a global reduction operator, and it aggregates the local processing results from participating nodes through the global reduction operator. Applications may allow a DAC request to return partial aggregation results as a tradeoff between quality and availability. Our architecture design aims at improving interactive responses with sustained throughput for typical cluster environments where platform heterogeneity and software/hardware failures are common. At the cluster level, our load-adaptive reduction tree constr...
Lingkun Chu, Hong Tang, Tao Yang, Kai Shen
Added 05 Jul 2010
Updated 05 Jul 2010
Type Conference
Year 2003
Where PPOPP
Authors Lingkun Chu, Hong Tang, Tao Yang, Kai Shen
Comments (0)