Sciweavers

IPPS
2007
IEEE

Improving MPI Independent Write Performance Using A Two-Stage Write-Behind Buffering Method

13 years 10 months ago
Improving MPI Independent Write Performance Using A Two-Stage Write-Behind Buffering Method
Many large-scale production applications often have very long executions times and require periodic data checkpoints in order to save the state of the computation for program restart and/or tracing application progress. These write-only operations often dominate the overall application runtime, which makes them a good optimization target. Existing approaches for write-behind data buffering at the MPI I/O level have been proposed, but challenges still exist for addressing system-level I/O issues. We propose a twostage write-behind buffering scheme for handing checkpoint operations. The first-stage of buffering accumulates write data for better network utilization and the second-stage of buffering enables the alignment for the write requests to the file stripe boundaries. Aligned I/O requests avoid file lock contention that can seriously degrade I/O performance. We present our performance evaluation using BTIO benchmarks on both GPFS and Lustre file systems. With the twostage buffer...
Wei-keng Liao, Avery Ching, Kenin Coloma, Alok N.
Added 03 Jun 2010
Updated 03 Jun 2010
Type Conference
Year 2007
Where IPPS
Authors Wei-keng Liao, Avery Ching, Kenin Coloma, Alok N. Choudhary, Mahmut T. Kandemir
Comments (0)