Scalable High Performance Message Passing over InfiniBand for Open MPI
Abstract
InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.
- Authors:
- Publication Date:
- Research Org.:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 923113
- Report Number(s):
- UCRL-CONF-235949
TRN: US200804%%944
- DOE Contract Number:
- W-7405-ENG-48
- Resource Type:
- Conference
- Resource Relation:
- Conference: Presented at: Communication in Clusters and Cluster Computer Interconnected Systems, Aachen, Germany, Dec 12 - Dec 12, 2007
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; COMMUNICATIONS; COMPUTERS; IMPLEMENTATION; PERFORMANCE; RELIABILITY; TRANSPORT
Citation Formats
Friedley, A, Hoefler, T, Leininger, M L, and Lumsdaine, A. Scalable High Performance Message Passing over InfiniBand for Open MPI. United States: N. p., 2007.
Web.
Friedley, A, Hoefler, T, Leininger, M L, & Lumsdaine, A. Scalable High Performance Message Passing over InfiniBand for Open MPI. United States.
Friedley, A, Hoefler, T, Leininger, M L, and Lumsdaine, A. 2007.
"Scalable High Performance Message Passing over InfiniBand for Open MPI". United States. https://www.osti.gov/servlets/purl/923113.
@article{osti_923113,
title = {Scalable High Performance Message Passing over InfiniBand for Open MPI},
author = {Friedley, A and Hoefler, T and Leininger, M L and Lumsdaine, A},
abstractNote = {InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.},
doi = {},
url = {https://www.osti.gov/biblio/923113},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Oct 24 00:00:00 EDT 2007},
month = {Wed Oct 24 00:00:00 EDT 2007}
}