Many-Thread Aware Prefetching Mechanisms for GPGPU Applications

Abstract-- We consider the problem of how to improve memory latency tolerance in massively multithreaded GPGPUs when the thread-level parallelism of an application is not sufficient to hide memory latency. One solution used in conventional CPU systems is prefetching, both in hardware and software. However, we show that straightforwardly applying such mechanisms to GPGPU systems does not deliver the expected performance benefits and can in fact hurt performance when not used judiciously. This paper proposes new hardware and software prefetching mechanisms tailored to GPGPU systems, which we refer to as many-thread aware prefetching (MT-prefetching) mechanisms. Our software MT-prefetching mechanism, called interthread prefetching, exploits the existence of common memory access behavior among fine-grained threads. For hardware MTprefetching, we describe a scalable prefetcher training algorithm along with a hardware-based inter-thread prefetching mechanism. In some cases, blindly applying ...

• Type: Journal
• Year: 2010
• Where: MICRO
• Authors: Jaekyu Lee, Nagesh B. Lakshminarayana, Hyesoon Kim, Richard W. Vuduc
• Keywords: Hardware, Memory Latency, MICRO 2010, Performance