Load Balancing Hybrid Programming Models for SMP Clusters and Fully Permutable Loops

This paper emphasizes on load balancing issues associated with hybrid programming models for the parallelization of fully permutable nested loops onto SMP clusters. Hybrid parallel programming models usually suffer from intrinsic load imbalance between threads, mainly because most existing message passing libraries generally provide limited multi-threading support, allowing only the master thread to perform inter-node message passing communication. In order to mitigate this effect, we propose a generic method for the application of static load balancing on the coarse-grain hybrid model for the appropriate distribution of the computational load to the working threads. We experimentally evaluate the efficiency of the proposed scheme against a micro-kernel benchmark, and demonstrate the potential of such load balancing schemes for the extraction of maximum performance out of hybrid parallel programs.

• Type: Conference
• Year: 2005
• Where: ICPPW
• Authors: Nikolaos Drosinos, Nectarios Koziris
• Keywords: Distributed And Parallel Computing, Hybrid Parallel, ICPPW 2005, Intrinsic Load Imbalance, Load Balancing Issues