An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation

Lopez, Matthew Graham; Shamis, Pavel; Gorentla Venkata, Manjunath

Title: An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation

Full Record
Other Related Research

Abstract

Alltoallv() is a collective operation which allows all pro- cesses to exchange variable amounts of data with all other processes in the communication group. This means that Alltoallv() requires not only O(N2) communications, but typically also additional exchanges of the data lengths that will be transmitted in the eventual Alltoallv() call. This pre-exchange is used to calculate the proper offsets for the re- ceiving buffers on the target processes. However, we propose two new can- didate interfaces for Alltoallv() that would mitigate the need for the user to set up this extra exchange of information at the possible cost of memory efficiency. We explain the new interface variants and show how a single call can be used in place of the traditional Alltoall()/Alltoallv() pair. We then discuss the performance tradeoffs for overall communica- tion and memory costs, as well as both software and hardware-based optimizations and their applicability to the various proposed interfaces.

Authors:

Lopez, Matthew Graham ^[1]; Shamis, Pavel ^[1]; Gorentla Venkata, Manjunath ^[1]

ORNL

Publication Date:: 2015-01-01

Research Org.:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)

Sponsoring Org.:: Work for Others (WFO)

OSTI Identifier:: 1265652

DOE Contract Number:: AC05-00OR22725

Resource Type:: Conference

Resource Relation:: Conference: OpenSHMEM and Related Technologies, Annapolis, MD, USA, 20150804, 20150806

Country of Publication:: United States

Language:: English

Citation Formats


                    Lopez, Matthew Graham, Shamis, Pavel, and Gorentla Venkata, Manjunath. An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation.  United States: N. p., 2015. 
        Web.

Copy to clipboard


                    Lopez, Matthew Graham, Shamis, Pavel, & Gorentla Venkata, Manjunath. An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation.  United States.

Copy to clipboard


                    Lopez, Matthew Graham, Shamis, Pavel, and Gorentla Venkata, Manjunath. 2015.  
        "An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation".  United States.  https://www.osti.gov/servlets/purl/1265652.

Copy to clipboard


                    
@article{osti_1265652,

  title        = {An Evaluation of OpenSHMEM Interfaces for the Variable-length Alltoallv() Collective Operation},

  author       = {Lopez, Matthew Graham and Shamis, Pavel and Gorentla Venkata, Manjunath},

  abstractNote = {Alltoallv() is a collective operation which allows all pro- cesses to exchange variable amounts of data with all other processes in the communication group. This means that Alltoallv() requires not only O(N2) communications, but typically also additional exchanges of the data lengths that will be transmitted in the eventual Alltoallv() call. This pre-exchange is used to calculate the proper offsets for the re- ceiving buffers on the target processes. However, we propose two new can- didate interfaces for Alltoallv() that would mitigate the need for the user to set up this extra exchange of information at the possible cost of memory efficiency. We explain the new interface variants and show how a single call can be used in place of the traditional Alltoall()/Alltoallv() pair. We then discuss the performance tradeoffs for overall communica- tion and memory costs, as well as both software and hardware-based optimizations and their applicability to the various proposed interfaces.},

  doi          = {},

  url          = {https://www.osti.gov/biblio/1265652},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2015},

  month        = {1}

}

Copy to clipboard

Conference:

View Conference (0.41 MB)

Other availability

Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

An Evaluation of OpenSHMEM Interfaces for the Variable-Length Alltoallv() Collective Operation, In: OpenSHMEM and Related Technologies. Experiences, Implementations, and Technologies

Conference Lopez, M.; Shamis, Pavel; Venkata, Manjunath

Alltoallv() is a collective operation which allows all processes to exchange variable amounts of data with all other processes in the communication group. This means that Alltoallv() requires not only O(N2) communications, but typically also additional exchanges of the data lengths that will be transmitted in the eventual Alltoallv() call. This pre-exchange is used to calculate the proper offsets for the receiving buffers on the target processes. However, we propose two new candidate interfaces for Alltoallv() that would mitigate the need for the user to set up this extra exchange of information at the possible cost of memory efficiency. Wemore »« less
https://doi.org/10.1007/978-3-319-26428-8_6
Evaluating OpenSHMEM Explicit Remote Memory Access Operations and Merged Requests

Conference Boehm, Swen; Pophale, Swaroop; Gorentla Venkata, Manjunath

The OpenSHMEM Library Specification has evolved consid- erably since version 1.0. Recently, non-blocking implicit Remote Memory Access (RMA) operations were introduced in OpenSHMEM 1.3. These provide a way to achieve better overlap between communication and computation. However, the implicit non-blocking operations do not pro- vide a separate handle to track and complete the individual RMA opera- tions. They are guaranteed to be completed after either a shmem quiet(), shmem barrier() or a shmem barrier all() is called. These are global com- pletion and synchronization operations. Though this semantic is expected to achieve a higher message rate for the applications, themore »« less
Designing a High Performance OpenSHMEM Implementation using Universal Common Communication Substrate as a Communication Middleware

Conference Welch, Donald; Shamis, Pavel; Gorentla Venkata, Manjunath; ...

OpenSHMEM is an effort to standardize the well-known SHMEM parallel programming library. The project aims to produce an open-source and portable SHMEM API and is led by ORNL and UH. In this paper, we optimize the current OpenSHMEM reference implementa- tion, based on GASNet, to achieve higher performance characteristics. To achieve these desired performance characteristics, we have redesigned an important component of the OpenSHMEM implementation, the network layer, to leverage a low-level communication library designed for imple- menting parallel programming models called UCCS. In particular, UCCS provides an interface and semantics such as native atomic operations and remote memory operationsmore »« less
SLOAVx: Scalable LOgarithmic AlltoallV Algorithm for Hierarchical Multicore Systems

Conference Xu, Cong; Venkata, M.; Graham, R.; ... - PROCEEDINGS OF THE 2013 13TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID 2013)

Scientific applications use collective communication operations in Message Passing Interface (MPI) for global synchronization and data exchanges. Alltoall and AlltoallV are two important collective operations. They are used by MPI jobs to exchange messages among all of MPI processes. AlltoallV is a generalization of Alltoall, supporting messages of varying sizes. However, the existing MPI AlltoallV implementation has linear complexity, i.e., each process has to send messages to all other processes in the job. Such linear complexity can result in sub optimal scalability of MPI applications when they are deployed on millions of cores. To address above challenge, in this paper,more »« less
https://doi.org/10.1109/CCGrid.2013.22
Evaluating System Parameters on a Dragonfly using Simulation and Visualization

Technical Report Bhatele, Abhinav; Jain, Nikhil; Livnat, Yarden; ...

The dragon y topology is becoming a popular choice for build- ing high-radix, low-diameter networks with high-bandwidth links. Even with a powerful network, preliminary experi- ments on Edison at NERSC have shown that for communica- tion heavy applications, job interference and thus presumably job placement remains an important factor. In this paper, we explore the e ects of job placement, job sizes, parallel workloads and network con gurations on network through- put to better understand inter-job interference. We use a simulation tool called Damsel y to model the network be- havior of Edison and study the impact of various systemmore »« less
https://doi.org/10.2172/1241972

Full Text Available

Similar Records