Optimizing Bandwidth Limited Problems Using One-SidedCommunication and Overlap
Partitioned Global Address Space languages like Unified Parallel C (UPC) are typically valued for their expressiveness, especially for computations with fine-grained random accesses. In this paper we show that the one-sided communication model used in these languages also has a significant performance advantage for bandwidth-limited applications. We demonstrate this benefit through communication microbenchmarks and a case-study that compares UPC and MPI implementations of the NAS Fourier Transform (FT) benchmark. Our optimizations rely on aggressively overlapping communication with computation but spreading communication events throughout the course of the local computation. This alleviates the potential communication bottleneck that occurs when the communication is packed into a single phase (e.g., the large all-to-all in a multidimensional FFT). Even though the new algorithms require more messages for the same total volume of data, the resulting overlap leads to speedups of over 1.75x and 1.9x for the two-sided and one-sided implementations, respectively, when compared to the default NAS Fortran/MPI release. Our best one-sided implementations show an average improvement of 15 percent over our best two-sided implementations. We attribute this difference to the lower software overhead of one-sided communication, which is partly fundamental to the semantic difference between one-sided and two-sided communication. Our UPC results use the Berkeley UPC compiler with the GASNet communication system, and demonstrate the portability and scalability of that language and implementation, with performance approaching 0.5TFlop/s on the FT benchmark running on 512 processors.
- Research Organization:
- Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
- Sponsoring Organization:
- USDOE Director. Office of Science. Office of AdvancedScientific Computing Research
- DOE Contract Number:
- DE-AC02-05CH11231
- OSTI ID:
- 891353
- Report Number(s):
- LBNL-59207; R&D Project: K52018; BnR: KJ0102000; TRN: US200621%%848
- Resource Relation:
- Conference: IPDPS 2006, Rhodes Island, Greece, Tuesday, 25April - Saturday, 29 April 2006
- Country of Publication:
- United States
- Language:
- English
Similar Records
Efficient On-demand Connection Management Mechanisms with PGAS Models on InfiniBand
GASNet-EX: A High-Performance, Portable Communication Library for Exascale