On the Harmfulness of Redundant Batch Requests

11 years 5 months ago
On the Harmfulness of Redundant Batch Requests
Most parallel computing resources are controlled by batch schedulers that place requests for computation in a queue until access to compute nodes is granted. Queue waiting times are notoriously hard to predict, making it difficult for users not only to estimate when their applications may start, but also to pick among multiple batch-scheduled resources the one that will produce the shortest turnaround time. As a result, an increasing number of users resort to “redundant requests”: several requests are simultaneously submitted to multiple batch schedulers on behalf of a single job; once one of these requests is granted access to compute nodes, the others are canceled. Using simulation as well as experiments with a production batch scheduler we investigate whether redundant requests are harmful in terms of (i) schedule performance and fairness, (ii) system load, and (iii) system predictability. We find that two main issues with redundant requests are load on the middleware and unf...
Henri Casanova
Added 11 Jun 2010
Updated 11 Jun 2010
Type Conference
Year 2006
Where HPDC
Authors Henri Casanova
Comments (0)