skip to main content

Title: Scalable High Performance Message Passing over InfiniBand for Open MPI

InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.
Authors:
; ; ;
Publication Date:
OSTI Identifier:
923113
Report Number(s):
UCRL-CONF-235949
TRN: US200804%%944
DOE Contract Number:
W-7405-ENG-48
Resource Type:
Conference
Resource Relation:
Conference: Presented at: Communication in Clusters and Cluster Computer Interconnected Systems, Aachen, Germany, Dec 12 - Dec 12, 2007
Research Org:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; COMMUNICATIONS; COMPUTERS; IMPLEMENTATION; PERFORMANCE; RELIABILITY; TRANSPORT