Sciweavers

IPPS
2008
IEEE

A dynamic scheduling approach for coordinated wide-area data transfers using GridFTP

13 years 9 months ago
A dynamic scheduling approach for coordinated wide-area data transfers using GridFTP
Many scientific applications need to stage large volumes of files from one set of machines to another set of machines in a wide-area network. Efficient execution of such data transfers needs to take into account the heterogeneous nature of the environment and dynamic availability of shared resources. This paper proposes an algorithm that dynamically schedules a batch of data transfer requests with the goal of minimizing the overall transfer time. The proposed algorithm performs simultaneous transfer of chunks of files from multiple file replicas, if the replicas exist. Adaptive replica selection is employed to transfer different chunks of the same file by taking dynamically changing network bandwidths into account. We utilize GridFTP as the underlying mechanism for data transfers. The algorithm makes use of information from past GridFTP transfers to estimate network bandwidths and resource availability. The efficiency of the algorithm is evaluated on a wide-area testbed.
Gaurav Khanna 0002, Ümit V. Çataly&uum
Added 31 May 2010
Updated 31 May 2010
Type Conference
Year 2008
Where IPPS
Authors Gaurav Khanna 0002, Ümit V. Çatalyürek, Tahsin M. Kurç, Rajkumar Kettimuthu, P. Sadayappan, Joel H. Saltz
Comments (0)