Sciweavers

CCGRID
2011
IEEE

A Segment-Level Adaptive Data Layout Scheme for Improved Load Balance in Parallel File Systems

12 years 7 months ago
A Segment-Level Adaptive Data Layout Scheme for Improved Load Balance in Parallel File Systems
Abstract—Parallel file systems are designed to mask the everincreasing gap between CPU and disk speeds via parallel I/O processing. While they have become an indispensable component of modern high-end computing systems, their inadequate performance is a critical issue facing the HPC community today. Conventionally, a parallel file system stripes a file across multiple file servers with a fixed stripe size. The stripe size is a vital performance parameter, but the optimal value for it is often application dependent. How to determine the optimal stripe size is a difficult research problem. Based on the observation that many applications have different data-access clusters in one file, with each cluster having a distinguished data access pattern, we propose in this paper a segmented data layout scheme for parallel file systems. The basic idea behind the segmented approach is to divide a file logically into segments such that an optimal stripe size can be identified for each se...
Huaiming Song, Yanlong Yin, Xian-He Sun, Rajeev Th
Added 18 Aug 2011
Updated 18 Aug 2011
Type Journal
Year 2011
Where CCGRID
Authors Huaiming Song, Yanlong Yin, Xian-He Sun, Rajeev Thakur, Samuel Lang
Comments (0)