A scalable framework for heterogeneous GPU-based clusters

11 years 9 months ago
A scalable framework for heterogeneous GPU-based clusters
GPU-based heterogeneous clusters continue to draw attention from vendors and HPC users due to their high energy efficiency and much improved single-node computational performance, however, there is little parallel software available that can utilize all CPU cores and all GPUs on the heterogeneous system efficiently. On a heterogeneous cluster, the performance of a GPU (or a compute node) increases in a much faster rate than the performance of the PCI-Express connection (or the interconnection network) such that communication eventually becomes the bottleneck of the entire system. To overcome the bottleneck, we developed a multi-level partitioning and distribution method that guarantees a near-optimal communication volume. We have also extended heterogeneous tile algorithms to work on distributed-memory GPU clusters. Our main idea is to execute a serial program and generate hybrid-size tasks, and follow a dataflow programming model to fire the tasks on different compute nodes. We t...
Fengguang Song, Jack Dongarra
Added 29 Sep 2012
Updated 29 Sep 2012
Type Journal
Year 2012
Where SPAA
Authors Fengguang Song, Jack Dongarra
Comments (0)