Phaser accumulators: A new reduction construct for dynamic parallelism

A reduction is a computation in which a common operation, such as a sum, is to be performed across multiple pieces of data, each supplied by a separate task. We introduce phaser accumulators, a new reduction construct that meshes seamlessly with phasers to support dynamic parallelism in a phased (iterative) setting. By separating reduction computations into the parts of sending data, performing the computation itself, and retrieving the result, we enable overlap of communication and computation in a manner analogous to that of split-phase barriers. Additionally, this separation enables exploration of implementation strategies that differ as to when the reduction itself is performed: eagerly when the data is supplied, or lazily when a synchronization point is reached. We implement accumulators as extensions to phasers in the Habanero dialect of the X10 programming language. Performance evaluations of the EPCC Syncbench, Spectralnorm, and CG benchmarks on AMD Opteron, Intel Xeon, and Su...

• Type: Conference
• Year: 2009
• Where: IPPS
• Authors: Jun Shirako, David M. Peixotto, Vivek Sarkar, William N. Scherer III
• Keywords: Distributed And Parallel Computing, IPPS 2009, Phaser Accumulators, Reduction Construct, UltraSPARC T2