Sciweavers

Share
CLOUD
2010
ACM

Nephele/PACTs: a programming model and execution framework for web-scale analytical processing

10 years 4 months ago
Nephele/PACTs: a programming model and execution framework for web-scale analytical processing
We present a parallel data processor centered around a programming model of so called Parallelization Contracts (PACTs) and the scalable parallel execution engine Nephele [18]. The PACT programming model is a generalization of the well-known map/reduce programming model, extending it with further second-order functions, as well as with Output Contracts that give guarantees about the behavior of a function. We describe methods to transform a PACT program into a data flow for Nephele, which executes its sequential building blocks in parallel and deals with communication, synchronization and fault tolerance. Our definition of PACTs allows to apply several types of optimizations on the data flow during the transformation. The system as a whole is designed to be as generic as (and compatible to) map/reduce systems, while overcoming several of their major weaknesses: 1) The functions map and reduce alone are not sufficient to express many data processing tasks both naturally and efficien...
Dominic Battré, Stephan Ewen, Fabian Hueske
Added 10 Jul 2010
Updated 10 Jul 2010
Type Conference
Year 2010
Where CLOUD
Authors Dominic Battré, Stephan Ewen, Fabian Hueske, Odej Kao, Volker Markl, Daniel Warneke
Comments (0)
books