Sciweavers

Share
IPPS
2010
IEEE

Reconciling scratch space consumption, exposure, and volatility to achieve timely staging of job input data

10 years 2 months ago
Reconciling scratch space consumption, exposure, and volatility to achieve timely staging of job input data
Innovative scientific applications and emerging dense data sources are creating a data deluge for highend computing systems. Processing such large input data typically involves copying (or staging) onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. The current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail re-staging and consequently reduced job throughput. To address this, we present a timely staging framework that uses a combination of job startup time predictions, user-specified intermediate nodes, and decentralized data delivery to coincide input data staging with job start-up. By delaying staging to when it is necessary, the exposure to failures and its effects can be reduced. Evaluation using both PlanetLab and simulations based on three years of Jaguar (No. 1 in Top500) job logs show as much as 85.9% reduction in staging times compared to dir...
Henry M. Monti, Ali Raza Butt, Sudharshan S. Vazhk
Added 13 Feb 2011
Updated 13 Feb 2011
Type Journal
Year 2010
Where IPPS
Authors Henry M. Monti, Ali Raza Butt, Sudharshan S. Vazhkudai
Comments (0)
books