skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Scalable High Performance Message Passing over InfiniBand for Open MPI

Abstract

InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.

Authors:
; ; ;
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
923113
Report Number(s):
UCRL-CONF-235949
TRN: US200804%%944
DOE Contract Number:  
W-7405-ENG-48
Resource Type:
Conference
Resource Relation:
Conference: Presented at: Communication in Clusters and Cluster Computer Interconnected Systems, Aachen, Germany, Dec 12 - Dec 12, 2007
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; COMMUNICATIONS; COMPUTERS; IMPLEMENTATION; PERFORMANCE; RELIABILITY; TRANSPORT

Citation Formats

Friedley, A, Hoefler, T, Leininger, M L, and Lumsdaine, A. Scalable High Performance Message Passing over InfiniBand for Open MPI. United States: N. p., 2007. Web.
Friedley, A, Hoefler, T, Leininger, M L, & Lumsdaine, A. Scalable High Performance Message Passing over InfiniBand for Open MPI. United States.
Friedley, A, Hoefler, T, Leininger, M L, and Lumsdaine, A. 2007. "Scalable High Performance Message Passing over InfiniBand for Open MPI". United States. https://www.osti.gov/servlets/purl/923113.
@article{osti_923113,
title = {Scalable High Performance Message Passing over InfiniBand for Open MPI},
author = {Friedley, A and Hoefler, T and Leininger, M L and Lumsdaine, A},
abstractNote = {InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.},
doi = {},
url = {https://www.osti.gov/biblio/923113}, journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Oct 24 00:00:00 EDT 2007},
month = {Wed Oct 24 00:00:00 EDT 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: