Fast Parallel Non-Contiguous File Access

9 years 10 months ago
Fast Parallel Non-Contiguous File Access
Many applications of parallel I/O perform non-contiguous file accesses: instead of accessing a single (large) block of data in a file, a number of (smaller) blocks of data scattered throughout the file needs to be accessed in each logical I/O operation. However, only few file system interfaces directly support this kind of non-contiguous file access. In contrast, the most commonly used parallel programming interface, MPI, incorporates a flexible model of parallel I/O through its MPI-IO interface. With MPI-IO, arbitrary non-contiguous file accesses are supported in a uniform fashion by the use of derived MPI datatypes set up by the user to reflect the desired I/O pattern. Despite a considerable amount of recent work in this area, current MPI-IO implementations suffer from low performance of such non-contiguous accesses when compared to the performance of the storage system for contiguous accesses. In this paper we analyze an important bottleneck in the efficient handling of no...
Joachim Worringen, Jesper Larsson Träff, Hube
Added 05 Jul 2010
Updated 05 Jul 2010
Type Conference
Year 2003
Where SC
Authors Joachim Worringen, Jesper Larsson Träff, Hubert Ritzdorf
Comments (0)