skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric.

Abstract

Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) has the potential to provide performance that rivals traditional high performance fabrics. If this potential proves out, significant impacts on system procurement decisions could follow. This work provides a series of small scale performance results which are used to compare and contrast the performance of RoCE-enabled Ethernet with TCP-based Ethernet and an HPC network. Additionally, a discussion of the maturity of RoCE firmware/software stacks and documentation is provided along with useful approaches for probing performance. A detailed description of two experimental setups known to have good RoCE performance is given, including step-by-step configuration and the exact hardware and software revisions employed. At small scales, RoCE is found to have significant performance advantages over "out-of-the-box" TCP protocols and is competitive with state-of-the-art high performance networks. Further examination of RoCE using a wider array of benchmarks and at greater scale is warranted.

Authors:
;
Publication Date:
Research Org.:
Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1573446
Report Number(s):
SAND2019-13444
681101
DOE Contract Number:  
AC04-94AL85000
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English

Citation Formats

Kenny, Joseph P., and Ulmer, Craig D. RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric.. United States: N. p., 2019. Web. doi:10.2172/1573446.
Kenny, Joseph P., & Ulmer, Craig D. RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric.. United States. doi:10.2172/1573446.
Kenny, Joseph P., and Ulmer, Craig D. Tue . "RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric.". United States. doi:10.2172/1573446. https://www.osti.gov/servlets/purl/1573446.
@article{osti_1573446,
title = {RoCE: Promising Technology for Ethernet as a High Performance Networking Fabric.},
author = {Kenny, Joseph P. and Ulmer, Craig D.},
abstractNote = {Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) has the potential to provide performance that rivals traditional high performance fabrics. If this potential proves out, significant impacts on system procurement decisions could follow. This work provides a series of small scale performance results which are used to compare and contrast the performance of RoCE-enabled Ethernet with TCP-based Ethernet and an HPC network. Additionally, a discussion of the maturity of RoCE firmware/software stacks and documentation is provided along with useful approaches for probing performance. A detailed description of two experimental setups known to have good RoCE performance is given, including step-by-step configuration and the exact hardware and software revisions employed. At small scales, RoCE is found to have significant performance advantages over "out-of-the-box" TCP protocols and is competitive with state-of-the-art high performance networks. Further examination of RoCE using a wider array of benchmarks and at greater scale is warranted.},
doi = {10.2172/1573446},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {10}
}