Designing multi-leader-based Allgather algorithms for multi-core clusters

The increasing demand for computational cycles is being met by the use of multi-core processors. Having large number of cores per node necessitates multi-core aware designs to extract the best performance. The Message Passing Interface (MPI) is the dominant parallel programming model on modern high performance computing clusters. The MPI collective operations take a significant portion of the communication time for an application. The existing optimizations for collectives exploit shared memory for intranode communication to improve performance. However, it still would not scale well as the number of cores per node increase. In this work, we propose a novel and scalable multileader-based hierarchical Allgather design. This design allows better cache sharing for Non-Uniform Memory Access (NUMA) machines and makes better use of the network speed available with high performance interconnects such as InfiniBand. The new multi-leader-based scheme achieves a performance improvement of up ...

• Type: Conference
• Year: 2009
• Where: IPPS
• Authors: Krishna Chaitanya Kandalla, Hari Subramoni, Gopalakrishnan Santhanaraman, Matthew J. Koop, Dhabaleswar K. Panda
• Keywords: Distributed And Parallel Computing, IPPS 2009, MPI Collective Operations, Multi-core Aware Designs, Performance Computing Clusters