A Programming Model for Massive Data Parallelism with Data Dependencies

Conference · Thu Jan 01 04:00:00 EST 2009

OSTI ID:964332

Cui, Xiaohui ^[1]; Mueller, Frank ^[2]; Potok, Thomas E ^[1]; Zhang, Yongpeng ^[1]

---

1. ORNL
2. North Carolina State University

Accelerating processors can often be more cost and energy effective for a wide range of data-parallel computing problems than general-purpose processors. For graphics processor units (GPUs), this is particularly the case when program development is aided by environments such as NVIDIA s Compute Unified Device Architecture (CUDA), which dramatically reduces the gap between domain-specific architectures and general purpose programming. Nonetheless, general-purpose GPU (GPGPU) programming remains subject to several restrictions. Most significantly, the separation of host (CPU) and accelerator (GPU) address spaces requires explicit management of GPU memory resources, especially for massive data parallelism that well exceeds the memory capacity of GPUs. One solution to this problem is to transfer data between the GPU and host memories frequently. In this work, we investigate another approach. We run massively data-parallel applications on GPU clusters. We further propose a programming model for massive data parallelism with data dependencies for this scenario. Experience from micro benchmarks and real-world applications shows that our model provides not only ease of programming but also significant performance gains.

• Research Organization: Oak Ridge National Laboratory (ORNL)
• Sponsoring Organization: ORNL LDRD Seed-Money; ORNL work for others
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 964332
• Country of Publication: United States
• Language: English

Related Subjects

43 PARTICLE ACCELERATORS
ACCELERATORS
ARCHITECTURE
BENCHMARKS
CAPACITY
MANAGEMENT
PERFORMANCE
PROGRAMMING