RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric
- Sandia National Laboratories (SNL-CA), Livermore, CA (United States)
Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) has the potential to provide performance that rivals traditional high performance fabrics. If this potential proves out, significant impacts on system procurement decisions could follow. This work provides a series of small scale performance results which are used to compare and contrast the performance of RoCE-enabled Ethernet with TCP-based Ethernet and an HPC network. Additionally, a discussion of the maturity of RoCE firmware/software stacks and documentation is provided along with useful approaches for probing performance. A detailed description of two experimental setups known to have good RoCE performance is given, including step-by-step configuration and the exact hardware and software revisions employed. At small scales, RoCE is found to have significant performance advantages over "out-of-the-box" TCP protocols and is competitive with state-of-the-art high performance networks. Further examination of RoCE using a wider array of benchmarks and at greater scale is warranted.
- Research Organization:
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA)
- DOE Contract Number:
- AC04-94AL85000
- OSTI ID:
- 1573446
- Report Number(s):
- SAND-2019-13444; 681101
- Country of Publication:
- United States
- Language:
- English
Similar Records
Final Report for Project DE-FC02-06ER25755 [Pmodels2]
Comparison of High Performance Network Options: EDR InfiniBand vs.100Gb RDMA Capable Ethernet