Sciweavers

Share
GRID
2006
Springer

Multi-Replication with Intelligent Staging in Data-Intensive Grid Applications

10 years 4 months ago
Multi-Replication with Intelligent Staging in Data-Intensive Grid Applications
Existing data grid scheduling systems handle huge data I/O via replica location services coupled with simple staging, decoupled from scheduling of computing tasks. However, when the application/workflow scales, we observe considerable degradations in performance, compared to processing within a tightly-coupled cluster. For example, when numerous nodes access the same set of files simultaneously, major performance degradation occurs even if replicas are used, due to bottlenecks that manifest in the infrastructure. Instead of resorting to expensive solutions such as parallel file systems, we propose alleviating the situation by tightly coupling replica and data transfer management with computation scheduling. In particular we propose three techniques: (1) dynamic aggregation and O(1) replication of data-staging requests across multiple nodes using a multi-replication framework, (2) replica-centric scheduling -data re-use and time-to-replication as compute scheduling metrics on the grid a...
Yuya Machida, Shin'ichiro Takizawa, Hidemoto Nakad
Added 12 Dec 2010
Updated 12 Dec 2010
Type Journal
Year 2006
Where GRID
Authors Yuya Machida, Shin'ichiro Takizawa, Hidemoto Nakada, Satoshi Matsuoka
Comments (0)
books