skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Advanced flow-control mechanisms for the sockets direct protocol over infiniband.

Abstract

The Sockets Direct Protocol (SDP) is an industry standard to allow existing TCP/IP applications to be executed on high-speed networks such as InfiniBand (IB). Like many other high-speed networks, IB requires the receiver process to inform the network interface card (NIC), before the data arrives, about buffers in which incoming data has to be placed. To ensure that the receiver process is ready to receive data, the sender process typically performs flow-control on the data transmission. Existing designs of SDP flow-control are naive and do not take advantage of several interesting features provided by IB. Specifically, features such as RDMA are only used for performing zero-copy communication, although RDMA has more capabilities such as sender-side buffer management (where a sender process can manage SDP resources for the sender as well as the receiver). Similarly, IB also provides hardware flow-control capabilities that have not been studied in previous literature. In this paper, we utilize these capabilities to improve the SDP flow-control over IB using two designs: RDMA-based flow-control and NIC-assisted RDMA-based flow-control. We evaluate the designs using micro-benchmarks and real applications. Our evaluations reveal that these designs can improve the resource usage of SDP and consequently its performance by an order-of-magnitudemore » in some cases. Moreover we can achieve 10-20% improvement in performance for various applications.« less

Authors:
; ; ; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC); National Science Foundation (NSF); RNet Technologies
OSTI Identifier:
971468
Report Number(s):
ANL/MCS/CP-59460
TRN: US201004%%164
DOE Contract Number:  
DE-AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: 2007 International Conference on Parallel Processing ICPP 2007; Sep. 10, 2007 - Sep. 14, 2007; XiAn, China
Country of Publication:
United States
Language:
ENGLISH
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; BUFFERS; DATA TRANSMISSION; MANAGEMENT; PARALLEL PROCESSING; PERFORMANCE

Citation Formats

Balaji, P., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Dell Inc., and Ohio State Univ.. Advanced flow-control mechanisms for the sockets direct protocol over infiniband.. United States: N. p., 2007. Web. doi:10.1109/ICPP.2007.14.
Balaji, P., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Dell Inc., & Ohio State Univ.. Advanced flow-control mechanisms for the sockets direct protocol over infiniband.. United States. doi:10.1109/ICPP.2007.14.
Balaji, P., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Dell Inc., and Ohio State Univ.. Mon . "Advanced flow-control mechanisms for the sockets direct protocol over infiniband.". United States. doi:10.1109/ICPP.2007.14.
@article{osti_971468,
title = {Advanced flow-control mechanisms for the sockets direct protocol over infiniband.},
author = {Balaji, P. and Bhagvat, S. and Panda, D. K. and Thakur, R. and Gropp, W. and Mathematics and Computer Science and Dell Inc. and Ohio State Univ.},
abstractNote = {The Sockets Direct Protocol (SDP) is an industry standard to allow existing TCP/IP applications to be executed on high-speed networks such as InfiniBand (IB). Like many other high-speed networks, IB requires the receiver process to inform the network interface card (NIC), before the data arrives, about buffers in which incoming data has to be placed. To ensure that the receiver process is ready to receive data, the sender process typically performs flow-control on the data transmission. Existing designs of SDP flow-control are naive and do not take advantage of several interesting features provided by IB. Specifically, features such as RDMA are only used for performing zero-copy communication, although RDMA has more capabilities such as sender-side buffer management (where a sender process can manage SDP resources for the sender as well as the receiver). Similarly, IB also provides hardware flow-control capabilities that have not been studied in previous literature. In this paper, we utilize these capabilities to improve the SDP flow-control over IB using two designs: RDMA-based flow-control and NIC-assisted RDMA-based flow-control. We evaluate the designs using micro-benchmarks and real applications. Our evaluations reveal that these designs can improve the resource usage of SDP and consequently its performance by an order-of-magnitude in some cases. Moreover we can achieve 10-20% improvement in performance for various applications.},
doi = {10.1109/ICPP.2007.14},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: