On the Harmfulness of Redundant Batch Requests

Most parallel computing resources are controlled by batch schedulers that place requests for computation in a queue until access to compute nodes is granted. Queue waiting times are notoriously hard to predict, making it difficult for users not only to estimate when their applications may start, but also to pick among multiple batch-scheduled resources the one that will produce the shortest turnaround time. As a result, an increasing number of users resort to “redundant requests”: several requests are simultaneously submitted to multiple batch schedulers on behalf of a single job; once one of these requests is granted access to compute nodes, the others are canceled. Using simulation as well as experiments with a production batch scheduler we investigate whether redundant requests are harmful in terms of (i) schedule performance and fairness, (ii) system load, and (iii) system predictability. We find that two main issues with redundant requests are load on the middleware and unf...

• Type: Conference
• Year: 2006
• Where: HPDC
• Authors: Henri Casanova
• Keywords: Batch Scheduler, Distributed And Parallel Computing, HPDC 2006, Most Parallel Computing, Redundant Requests