Nonuniformly Communicating Noncontiguous Data: A Case Study with PETSc and MPI

Due to the complexity associated with developing parallel applications, scientists and engineers rely on highlevel software libraries such as PETSc, ScaLAPACK and PESSL to ease this task. Such libraries assist application developers by providing abstractions for mathematical operations, data representation and management of parallel layouts of the data, while internally using communication libraries such as MPI and PVM. With high-level libraries managing data layout and communication internally, it can be expected that they organize application data suitably for performing the library operations optimally. However, this places additional overhead on the underlying communication library by making the data layout noncontiguous in memory and communication volumes (data transferred to each process) nonuniform. In this paper, we analyze the overheads associated with these two aspects (noncontiguous data layouts and nonuniform communication volumes) in the context of the PETSc software tool...

• Type: Conference
• Year: 2007
• Where: IPPS
• Authors: Pavan Balaji, Darius Buntinas, Satish Balay, Barry F. Smith, Rajeev Thakur, William Gropp
• Keywords: Communication, Communication Volumes, Data Layouts, Distributed And Parallel Computing, IPPS 2007