Fault Tolerance and Recovery of Scientific Workflows on Computational Grids

In this paper, we describe the design and implementation of two mechanisms for fault-tolerance and recovery for complex scientific workflows on computational grids. We present our algorithms for over-provisioning and migration, which are our primary strategies for fault-tolerance. We consider application performance models, resource reliability models, network latency and bandwidth and queue wait times for batch-queues on compute resources for determining the correct fault-tolerance strategy. Our goal is to balance reliability and performance in the presence of soft real-time constraints like deadlines and expected success probabilities, and to do it in a way that is transparent to scientists. We have evaluated our strategies by developing a Fault-Tolerance and Recovery (FTR) service and deploying it as a part of the Linked Environments for Atmospheric Discovery (LEAD) production infrastructure. Results from real usage scenarios in LEAD show that the failure rate of individual steps in...

• Type: Conference
• Year: 2008
• Where: CCGRID
• Authors: Gopi Kandaswamy, Anirban Mandal, Daniel A. Reed
• Keywords: CCGRID 2008, Complex Scientific Workflows, Correct Fault-tolerance Strategy, Distributed And Parallel Computing, Fault-tolerance