Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Measuring Thread Timing to Assess the Feasibility of Early-Bird Message Delivery Across Systems and Scales

Journal Article · · Concurrency and Computation. Practice and Experience
DOI:https://doi.org/10.1002/cpe.8342· OSTI ID:2516801
 [1];  [2];  [2];  [2];  [3]
  1. Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States); Univ. of New Mexico, Albuquerque, NM (United States)
  2. Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
  3. Univ. of New Mexico, Albuquerque, NM (United States)
+ Show Author Affiliations

Early-bird communication is a communication/computation overlap technique that leverages fine-grained communication to improve application run-time. Communication is divided such that each individual thread can initiate transmission of its portion of the data upon completion rather than waiting for a dedicated communication phase. The benefit of early-bird communication depends on the completion timing of the individual threads: On the one hand, if all threads are complete at nearly the same time, the overheads of sending multiple messages will accumulate, leading to performance that is worse than if a single message had been sent. On the other hand, if thread completions are spread out in time, those that complete earlier can send data while others continue working, leading to performance that is better than if a single message had been sent. The challenge is that the completion times are currently unknown and can vary based on application, problem size, system software, and underlying hardware. In this paper, we address this lacuna by measuring and evaluating the potential overlap afforded by early-bird communication for a selection of proxy applications. These measurements help us understand whether a given application could benefit from early-bird communication. Here, we present our technique for gathering this data and evaluate data collected from three proxy applications: MiniFE, MiniMD, and MiniQMC. Each application is run on three systems with distinct CPU architectures and strong scales across three run sizes. To characterize the behavior of these workloads, we study the trends of thread timings at both a macro level, across all threads across all runs of an application, and a micro level, that is, within a single process of a single run. We observe that our tested applications exhibit significantly different thread arrival distributions. The machine used had a significant impact, with the window of potential overlap varying by as much as an order of magnitude.

Research Organization:
Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
NA0003525; NA0003966
OSTI ID:
2516801
Alternate ID(s):
OSTI ID: 2483373
Report Number(s):
SAND--2025-01664J
Journal Information:
Concurrency and Computation. Practice and Experience, Journal Name: Concurrency and Computation. Practice and Experience Journal Issue: 1 Vol. 37; ISSN 1532-0626
Publisher:
WileyCopyright Statement
Country of Publication:
United States
Language:
English

References (17)

A survey of MPI usage in the US exascale computing project: A survey of MPI usage in the U. S. exascale computing project journal September 2018
LAMMPS - a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales journal February 2022
Test suite for evaluating performance of multithreaded MPI communication journal December 2009
EDF Statistics for Goodness of Fit and Some Comparisons journal September 1974
An omnibus test of normality for moderate and large size samples journal January 1971
CMB: A Configurable Messaging Benchmark to Explore Fine-Grained Communication conference May 2024
A Quantitative Analysis of OS Noise conference May 2011
Scalable load-balance measurement for SPMD codes
  • Gamblin, Todd; de Supinski, Bronis R.; Schulz, Martin
  • 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2008.5222553
conference November 2008
Enabling Efficient Multithreaded MPI Communication through a Library-Based Implementation of MPI Endpoints
  • Sridharan, Srinivas; Dinan, James; Kalamkar, Dhiraj D.
  • SC14: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2014.45
conference November 2014
A new approach for performance analysis of openMP programs conference June 2013
Enabling MPI interoperability through flexible communication endpoints conference January 2013
Improving concurrency and asynchrony in multithreaded MPI applications using software offloading
  • Vaidyanathan, Karthikeyan; Kalamkar, Dhiraj D.; Pamnany, Kiran
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15 https://doi.org/10.1145/2807591.2807602
conference January 2015
Grain graphs journal February 2016
Modeling and Benchmarking the Potential Benefit of Early-Bird Transmission in Fine-Grained Communication conference August 2023
Measuring Thread Timing to Assess the Feasibility of Early-bird Message Delivery conference August 2023
Characterizing Task-Based OpenMP Programs journal April 2015
WOMBAT: A Scalable and High-performance Astrophysical Magnetohydrodynamics Code journal February 2017

Similar Records

Message passing with queues and channels
Patent · Tue Sep 24 00:00:00 EDT 2013 · OSTI ID:1096003
Send-side matching of data communications messages
Patent · Tue Jun 17 00:00:00 EDT 2014 · OSTI ID:1134199
Send-side matching of data communications messages
Patent · Tue Jul 01 00:00:00 EDT 2014 · OSTI ID:1136754

Related Subjects

97 MATHEMATICS AND COMPUTING