Scalable High Performance Message Passing over InfiniBand for Open MPI
InfiniBand (IB) is a popular network technology for modern high-performance computing systems. MPI implementations traditionally support IB using a reliable, connection-oriented (RC) transport. However, per-process resource usage that grows linearly with the number of processes, makes this approach prohibitive for large-scale systems. IB provides an alternative in the form of a connectionless unreliable datagram transport (UD), which allows for near-constant resource usage and initialization overhead as the process count increases. This paper describes a UD-based implementation for IB in Open MPI as a scalable alternative to existing RC-based schemes. We use the software reliability capabilities of Open MPI to provide the guaranteed delivery semantics required by MPI. Results show that UD not only requires fewer resources at scale, but also allows for shorter MPI startup times. A connectionless model also improves performance for applications that tend to send small messages to many different processes.
- Publication Date:
- OSTI Identifier:
- Report Number(s):
- DOE Contract Number:
- Resource Type:
- Resource Relation:
- Conference: Presented at: Communication in Clusters and Cluster Computer Interconnected Systems, Aachen, Germany, Dec 12 - Dec 12, 2007
- Research Org:
- Lawrence Livermore National Laboratory (LLNL), Livermore, CA
- Sponsoring Org:
- Country of Publication:
- United States
- 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; COMMUNICATIONS; COMPUTERS; IMPLEMENTATION; PERFORMANCE; RELIABILITY; TRANSPORT
Enter terms in the toolbar above to search the full text of this document for pages containing specific keywords.