High Throughput Intra-Node MPI Communication with Open-MX

11 years 27 days ago
High Throughput Intra-Node MPI Communication with Open-MX
Abstract—The increasing number of cores per node in highperformance computing requires an efficient intra-node MPI communication subsystem. Most existing MPI implementations rely on two copies across a shared memory-mapped file. Open-MX offers a single-copy mechanism that is tightly integrated in its regular communication stack, making it transparently available to the MX backend of many MPI layers. We describe this implementation and its offloaded copy backend using I/OAT hardware. Memory pinning requirements are then discussed, and overlapped pinning is introduced to enable the start of Open-MX intra-node data transfer earlier. Performance evaluation shows that this local communication stack performs better than MPICH2 and Open MPI for large messages, reaching up to 70 % better throughput in microbenchmarks when using I/OAT copy offload. Thanks to a singlecopy being involved, the Open-MX intra-node communication throughput also does not heavily depend on cache sharing between p...
Brice Goglin
Added 19 May 2010
Updated 19 May 2010
Type Conference
Year 2009
Where PDP
Authors Brice Goglin
Comments (0)