Middleware support for many-task computing

11 years 7 months ago
Middleware support for many-task computing
Many-task computing aims to bridge the gap between two computing paradigms, high throughput computing and high performance computing. Many-task computing denotes highperformance computations comprising multiple distinct activities, coupled via file system operations. The aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. Traditional techniques found in production systems in the scientific community to support many-task computing do not scale to today’s largest systems, due to issues in local resource manager scalability and granularity, efficient utilization of the raw hardware, long wait queue times, and shared/parallel file system contention and scalability. To address these limitations, we adopted a “top-down” approach to building a middleware called Falkon, to support the most demanding manytask computing applications at the largest scales. Falkon (Fast and Light-weight tasK executiON framework) integrates (1) multi-level scheduling ...
Ioan Raicu, Ian T. Foster, Mike Wilde, Zhao Zhang,
Added 23 Aug 2011
Updated 23 Aug 2011
Type Journal
Year 2010
Authors Ioan Raicu, Ian T. Foster, Mike Wilde, Zhao Zhang, Kamil Iskra, Peter H. Beckman, Yong Zhao, Alexander S. Szalay, Alok N. Choudhary, Philip Little, et al.
Comments (0)