Speeding up NGB with distributed file streaming framework

Grid computing provides a very rich environment for scientific calculations. In addition to the challenges it provides, it also offers new opportunities for optimization. In this paper we have utilized DFS (Distributed File Streaming) framework to speed up NAS Grid Benchmark workflows. By studying I/O patterns of NGB codes we have identified program locations where it is possible to overlap computation and data workflow phases. By integrating DFS into NGB, we demonstrate a useful method of improving overall workflow efficiency by streaming the output of the current process to make an input of the following stage, reducing a workflow to a series of distributed producer consumer stages. DFS framework eliminates file transfers and in the process makes process scheduling more efficient, leading to overall performance improvements in the turnaround time for HC (Helical Chain) data flow graph under Globus grid environment with the embedded DFS over the original version of the benchmark.

• Type: Conference
• Year: 2006
• Where: IPPS
• Authors: Bingchen Li, Kang Chen, Zhiteng Huang, H. L. Rajic, R. H. Kuhn
• Keywords: DFS Framework Eliminates, Distributed And Parallel Computing, Grid, IPPS 2006, NAS Grid Benchmark