PARRAY: a unifying array representation for heterogeneous parallelism

This paper introduces a programming interface called PARRAY (or Parallelizing ARRAYs) that supports system-level succinct programming for heterogeneous parallel systems like GPU clusters. The current practice of software development requires combining several low-level libraries like Pthread, OpenMP, CUDA and MPI. Achieving productivity and portability is hard with different numbers and models of GPUs. PARRAY extends mainstream C programming with novel array types of the following features: 1) the dimensions of an array type are nested in a tree structure, conceptually reflecting the memory hierarchy; 2) the definition of an array type may contain references to other array types, allowing sophisticated array types to be created for parallelization; 3) threads also form arrays that allow programming in a Single-ProgramMultiple-Codeblock (SPMC) style to unify various sophisticated communication patterns. This leads to shorter, more portable and maintainable parallel codes, while the p...

• Type: Journal
• Year: 2012
• Where: PPOPP
• Authors: Yifeng Chen, Xiang Cui, Hong Mei
• Keywords: Array Types, Distributed And Parallel Computing, Memory Hierarchy, PPOPP 2012, Source Code Generator