A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications

Ghazimirsaeed, S. Mahdieh; Grant, Ryan E.; Afsahi, Ahmad

doi:10.1016/j.parco.2019.102547

Title: A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications

Full Record
Other Related Research

Abstract

The Message Passing Interface (MPI) libraries use message queues to guarantee correct message ordering between communicating processes. Message queues are in the critical path of MPI communications and thus, the performance of message queue operations can have significant impact on the performance of applications. Collective communications are widely used in MPI applications and they can have considerable impact on generating long message queues. In this paper, we propose a unified message matching mechanism that improves the message queue search time by distinguishing messages coming from point-to-point and collective communications and using a distinct message queue data structure for them. For collective operations, it dynamically profiles the impact of each collective call on message queues during the application runtime and uses this information to adapt the message queue data structure for each collective dynamically. Moreover, we use a partner/non-partner message queue data structure for the messages coming from point-to-point communications. The proposed approach can successfully reduce the queue search time while maintaining scalable memory consumption. The evaluation results show that we can obtain up to 5.5x runtime speedup for applications with long list traversals. Moreover, we can gain up to 15% and 94% queue search time improvement for all elements inmore »« less

Authors:

Ghazimirsaeed, S. Mahdieh ^[1]; Grant, Ryan E. ^[2]; Afsahi, Ahmad ^[1]

Queen's Univ., Kingston, ON (Canada)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Publication Date:: Thu Sep 05 00:00:00 EDT 2019

Research Org.:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1570290

Alternate Identifier(s):: OSTI ID: 1702563

Report Number(s):: SAND-2019-10114J
Journal ID: ISSN 0167-8191; 678886

Grant/Contract Number:: AC04-94AL85000; NA0003525

Resource Type:: Accepted Manuscript

Journal Name:: Parallel Computing

Additional Journal Information:: Journal Volume: 89; Journal Issue: C; Journal ID: ISSN 0167-8191

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; MPI; Message matching; Message queue; Collective communications; Point-to-point communications

Citation Formats


                    Ghazimirsaeed, S. Mahdieh, Grant, Ryan E., and Afsahi, Ahmad. A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications.  United States: N. p., 2019. 
Web.  doi:10.1016/j.parco.2019.102547.

Copy to clipboard


                    Ghazimirsaeed, S. Mahdieh, Grant, Ryan E., & Afsahi, Ahmad. A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications.  United States.  https://doi.org/10.1016/j.parco.2019.102547

Copy to clipboard


                    Ghazimirsaeed, S. Mahdieh, Grant, Ryan E., and Afsahi, Ahmad. Thu .  
"A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications".  United States.  https://doi.org/10.1016/j.parco.2019.102547.  https://www.osti.gov/servlets/purl/1570290.

Copy to clipboard


                    
@article{osti_1570290,

  title        = {A dynamic, unified design for dedicated message matching engines for collective and point-to-point communications},

  author       = {Ghazimirsaeed, S. Mahdieh and Grant, Ryan E. and Afsahi, Ahmad},

  abstractNote = {The Message Passing Interface (MPI) libraries use message queues to guarantee correct message ordering between communicating processes. Message queues are in the critical path of MPI communications and thus, the performance of message queue operations can have significant impact on the performance of applications. Collective communications are widely used in MPI applications and they can have considerable impact on generating long message queues. In this paper, we propose a unified message matching mechanism that improves the message queue search time by distinguishing messages coming from point-to-point and collective communications and using a distinct message queue data structure for them. For collective operations, it dynamically profiles the impact of each collective call on message queues during the application runtime and uses this information to adapt the message queue data structure for each collective dynamically. Moreover, we use a partner/non-partner message queue data structure for the messages coming from point-to-point communications. The proposed approach can successfully reduce the queue search time while maintaining scalable memory consumption. The evaluation results show that we can obtain up to 5.5x runtime speedup for applications with long list traversals. Moreover, we can gain up to 15% and 94% queue search time improvement for all elements in applications with short and medium list traversals, respectively.},

  doi          = {10.1016/j.parco.2019.102547},

  journal      = {Parallel Computing},

  number       = C,

  volume       = 89,

  place        = {United States},

  year         = {Thu Sep 05 00:00:00 EDT 2019},

  month        = {Thu Sep 05 00:00:00 EDT 2019}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (Publisher)

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1016/j.parco.2019.102547

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 2 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Characterizing MPI matching via trace-based simulation

Journal Article Ferreira, Kurt Brian ; Levy, Scott Larson Nicoll ; Pedretti, Kevin ; ... - Parallel Computing

With the increased scale expected on future leadership-class systems, detailed information about the resource usage and performance of MPI message matching provides important insights into how to maintain application performance on next-generation systems. However, obtaining MPI message matching performance data is often not possible without significant effort. A common approach is to instrument an MPI implementation to collect relevant statistics. While this approach can provide important data, collecting matching data at runtime perturbs the application's execution, including its matching performance, and is highly dependent on the MPI library's matchlist implementation. In this paper, we introduce a trace-based simulation approach tomore »« less
https://doi.org/10.1145/3127024.3127040

Full Text Available
Using Simulation to Examine the Effect of MPI Message Matching Costs on Application Performance

Journal Article Levy, Scott ; Ferreira, Kurt B. ; Schonbein, Whit ; ... - Parallel Computing

Attaining high performance with MPI applications requires efficient message matching to minimize message processing overheads and the latency these overheads introduce into application communication. We use a validated simulation-based approach to examine the relationship between MPI message matching performance and application time-to-solution. Specifically, we examine how the performance of several important HPC workloads is affected by the time required for matching. Our analysis yields several important contributions: (i) the performance of current workloads is unlikely to be significantly affected by MPI matching unless match queue operations get much slower or match queues get much longer; (ii) match queue designs thatmore »« less
Cited by 1
https://doi.org/10.1016/j.parco.2019.02.008

Full Text Available
Characteristics of the Unexpected Message Queue of MPI applications

Conference Keller, Rainer ; Graham, Richard L

High Performance Computing systems are used on a regular basis to run a myriad of application codes, yet a surprising dearth of information exists with respect to communications characteristics. Even less information is available on the low-level communication libraries, such as the length of MPI Unexpected Message Queues (UMQs) and the length of time such messages spend in these queues. Such information is vital to developing appropriate strategies for handling such data at the library and system level. In this paper we present data on the communication characteristics of three applications GTC, LSMS, and S3D. We present data on themore »« less
Evaluating MPI resource usage summary statistics

Journal Article Ferreira, Kurt B. ; Levy, Scott - Parallel Computing

The Message Passing Interface (MPI) remains the dominant programming model for scientific applications running on today’s high-performance computing (HPC) systems. This dominance stems from MPI’s powerful semantics for inter-process communication that has enabled scientists to write applications for simulating important physical phenomena. MPI does not, however, specify how messages and synchronization should be carried out. Those details are typically dependent on low-level architecture details and the message characteristics of the application. Therefore, analyzing an application’s MPI resource usage is critical to tuning MPI’s performance on a particular platform. The result of this analysis is typically a discussion of the meanmore »« less
https://doi.org/10.1016/j.parco.2021.102825

Full Text Available
GMH: A Message Passing Toolkit for GPU Clusters

Conference Jie Chen, W. Watson, Weizhen Mao

Driven by the market demand for high-definition 3D graphics, commodity graphics processing units (GPUs) have evolved into highly parallel, multi-threaded, many-core processors, which are ideal for data parallel computing. Many applications have been ported to run on a single GPU with tremendous speedups using general C-style programming languages such as CUDA. However, large applications require multiple GPUs and demand explicit message passing. This paper presents a message passing toolkit, called GMH (GPU Message Handler), on NVIDIA GPUs. This toolkit utilizes a data-parallel thread group as a way to map multiple GPUs on a single host to an MPI rank, andmore »« less
https://doi.org/10.1109/ICPADS.2010.35

Similar Records