Evaluating the Potential of Cray Gemini Interconnect for PGAS Communication Runtime Systems

Conference · Wed Aug 24 00:00:00 EDT 2011

DOI:https://doi.org/10.1109/HOTI.2011.19· OSTI ID:1239518

Vishnu, Abhinav; ten Bruggencate, Monika; Olson, Ryan M.

The Cray Gemini Interconnect has been recently introduced as the next generation network for building scalable multi-petascale supercomputers. The Cray XE6 systems, which use the Gemini Interconnect are becoming available with Message Passing Interface (MPI) and Partitioned Global Address Space (PGAS) Models such as as Global Arrays, Unified Parallel C, Co-Array Fortran and Cascade High Performance Language. These PGAS models use one-sided communication runtime systems such as MPI-Remote Memory Access, Aggregate Remote Memory Copy Interface and proprietary communication runtime systems. The primary objective of our work is to study the potential of Cray Gemini Interconnect by designing application specific micro-benchmarks using the DMAPP userspace library provided by the Gemini Interconnect. To meet this objective, we design micro-benchmarks to study the performance of simple communication primitives and application specific micro-benchmarks to understand the behavior of Gemini Interconnect at scale. In our experiments, the Gemini Interconnect can achieve a peak bandwidth of 6911 MB/s and a latency of 1us for get communication primitive. Scalability tests for atomic memory operations and shift communication operation up to 65536 processes shows the efficacy of the Gemini Interconnect.

• Research Organization: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1239518
• Report Number(s): PNNL-SA-80414
• Resource Relation: Conference: IEEE 19th Annual Symposium on High Performance Interconnects (HOT 2011), August 24-26, 2011, Santa Clara, California, 70-77
• Country of Publication: United States
• Language: English