Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Exploiting Atomic Operations for Barrier on Cray XE/XK Systems

Conference · · EuroMPI 2012: Recent Advances in the Message Passing Interface
 [1];  [1];  [1];  [1];  [2];  [2]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  2. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
+ Show Author Affiliations
Barrier is a collective operation used by many scientific applications and parallel libraries for synchronization. Typically, a Barrier operation is implemented by exchanging a short data message that requires demultiplexing, thereby adding undesired latency to the operation. In this work, we reduce the latency of Barrier operations for Cray XE/XK systems by leveraging the atomic operations provided by the Gemini interconnect, tailoring algorithms to utilize these capabilities, and utilizing a hierarchical design to arrive at an efficient implementation. Our micro-benchmark evaluation shows that for a 4,096 process Barrier operation, the atomic-operations-based Barrier outperforms the data exchange Barrier by 52% and the native Barrier by 111%.
Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE Office of Science; USDOE
OSTI ID:
1567525
Conference Information:
Journal Name: EuroMPI 2012: Recent Advances in the Message Passing Interface
Country of Publication:
United States
Language:
English

References (4)

Two algorithms for barrier synchronization journal February 1988
Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation book January 2004
The Gemini System Interconnect
  • Alverson, Robert; Roweth, Duncan; Kaplan, Larry
  • 2010 IEEE 18th Annual Symposium on High-Performance Interconnects (HOTI), 2010 18th IEEE Symposium on High Performance Interconnects https://doi.org/10.1109/HOTI.2010.23
conference August 2010
Cheetah: A Framework for Scalable Hierarchical Collective Operations conference May 2011

Similar Records

Open MPI for Cray XE/XK Systems
Conference · Sat Dec 31 23:00:00 EST 2011 · OSTI ID:1049087
A uGNI-based Asynchronous Message-driven Runtime System for Cray Supercomputers with Gemini Interconnect
Conference · Thu Aug 16 00:00:00 EDT 2012 · 2012 IEEE 26TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS) · OSTI ID:1567313
A uGNI-Based Asynchronous Message-driven Runtime System for Cray Supercomputers with Gemini Interconnect
Conference · Sat Dec 31 23:00:00 EST 2011 · OSTI ID:1056922

Related Subjects

Atomic Operations
Barrier
Collectives
Cray
Gemini
MPI