Scalable parallel I/O alternatives for massively parallel partitioned solver systems

Abstract--With the development of high-performance computing, I/O issues have become the bottleneck for many massively parallel applications. This paper investigates scalable parallel I/O alternatives for massively parallel partitioned solver systems. Typically such systems have synchronized "loops" and will write data in a well defined block I/O format consisting of a header and data portion. Our target use for such an parallel I/O subsystem is checkpoint-restart where writing is by far the most common operation and reading typically only happens during either initialization or during a restart operation because of a system failure. We compare four parallel I/O strategies: 1 POSIX File Per Processor (1PFPP), a synchronized parallel IO library (syncIO), "Poor-Man's" Parallel I/O (PMPIO) and a new "reduced blocking" strategy (rbIO). Performance tests using real CFD solver data from PHASTA (an unstructured grid finite element Navier-Stokes solver [1]) s...

• Type: Journal
• Year: 2010
• Where: IPPS
• Authors: Jing Fu, Ning Liu, Onkar Sahni, Kenneth E. Jansen, Mark S. Shephard, Christopher D. Carothers
• Keywords: Distributed And Parallel Computing, File Per Processor, IPPS 2010, Massively Parallel, Parallel Io Library