skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Analyzing the impact of supportinjg out-of-order communication on in-order performance with iWARP.

Abstract

Due to the growing need to tolerate network faults and congestion in high-end computing systems, supporting multiple network communication paths is becoming increasingly important. However, multi-path communication comes with the disadvantage of out-of-order arrival of packets (because packets may traverse different paths). While modern networking stacks such as the Internet Wide-Area RDMA Protocol (iWARP) over 10-Gigabit Ethernet (10GE) support multi-path communication, their current implementations do not handle out-of-order packets primarily owing to the overhead on in-order communication that it adds. Specifically, in iWARP, supporting out-of-order packets requires every packet to carry additional information causing significant overhead on packets that arrive in-order. Thus, in this paper, we analyze the trade-offs in designing a feature-complete iWARP stack, i.e., one that provides support for out-of-order arriving packets, and thus, multi-path systems, while focusing on the performance of in-order communication. We propose three feature-complete designs of iWARP and analyze the pros and cons of each of these designs using performance experiments based on several micro-benchmarks as well as an iso-surface visual rendering application. Our analysis reveals that the iWARP design providing the best overall performance depends on the particular characteristics of the upper layers and that different designs are optimal based on the metricmore » of interest.« less

Authors:
; ; ; ; ; ; ; ; ;
Publication Date:
Research Org.:
Argonne National Lab. (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE Office of Science (SC); National Science Foundation (NSF)
OSTI Identifier:
971474
Report Number(s):
ANL/MCS/CP-59876
TRN: US201004%%29
DOE Contract Number:
DE-AC02-06CH11357
Resource Type:
Conference
Resource Relation:
Conference: Supercomputing 2007; Nov. 10, 2007 - Nov. 16, 2007; Reno, NV
Country of Publication:
United States
Language:
ENGLISH
Subject:
97 MATHEMATICAL METHODS AND COMPUTING; 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; DATA TRANSMISSION; DESIGN; INTERNET; METRICS; PERFORMANCE; COMPUTER NETWORKS; I CODES

Citation Formats

Balaji, P., Feng, W., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Virginia Tech, Dell Inc., and Ohio State Univ. Analyzing the impact of supportinjg out-of-order communication on in-order performance with iWARP.. United States: N. p., 2007. Web. doi:10.1145/1362622.1362670.
Balaji, P., Feng, W., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Virginia Tech, Dell Inc., & Ohio State Univ. Analyzing the impact of supportinjg out-of-order communication on in-order performance with iWARP.. United States. doi:10.1145/1362622.1362670.
Balaji, P., Feng, W., Bhagvat, S., Panda, D. K., Thakur, R., Gropp, W., Mathematics and Computer Science, Virginia Tech, Dell Inc., and Ohio State Univ. Mon . "Analyzing the impact of supportinjg out-of-order communication on in-order performance with iWARP.". United States. doi:10.1145/1362622.1362670.
@article{osti_971474,
title = {Analyzing the impact of supportinjg out-of-order communication on in-order performance with iWARP.},
author = {Balaji, P. and Feng, W. and Bhagvat, S. and Panda, D. K. and Thakur, R. and Gropp, W. and Mathematics and Computer Science and Virginia Tech and Dell Inc. and Ohio State Univ.},
abstractNote = {Due to the growing need to tolerate network faults and congestion in high-end computing systems, supporting multiple network communication paths is becoming increasingly important. However, multi-path communication comes with the disadvantage of out-of-order arrival of packets (because packets may traverse different paths). While modern networking stacks such as the Internet Wide-Area RDMA Protocol (iWARP) over 10-Gigabit Ethernet (10GE) support multi-path communication, their current implementations do not handle out-of-order packets primarily owing to the overhead on in-order communication that it adds. Specifically, in iWARP, supporting out-of-order packets requires every packet to carry additional information causing significant overhead on packets that arrive in-order. Thus, in this paper, we analyze the trade-offs in designing a feature-complete iWARP stack, i.e., one that provides support for out-of-order arriving packets, and thus, multi-path systems, while focusing on the performance of in-order communication. We propose three feature-complete designs of iWARP and analyze the pros and cons of each of these designs using performance experiments based on several micro-benchmarks as well as an iso-surface visual rendering application. Our analysis reveals that the iWARP design providing the best overall performance depends on the particular characteristics of the upper layers and that different designs are optimal based on the metric of interest.},
doi = {10.1145/1362622.1362670},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Mon Jan 01 00:00:00 EST 2007},
month = {Mon Jan 01 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • Abstract not provided.
  • Smart grids are susceptible to cyber-attack as a result of new communication, control and computation techniques employed in the grid. In this paper, we characterize and analyze the resiliency of smart grid communication architecture, specifically an RF mesh based architecture, under cyber attacks. We analyze the resiliency of the communication architecture by studying the performance of high-level smart grid functions such as metering, and demand response which depend on communication. Disrupting the operation of these functions impacts the operational resiliency of the smart grid. Our analysis shows that it takes an attacker only a small fraction of meters to compromisemore » the communication resiliency of the smart grid. We discuss the implications of our result to critical smart grid functions and to the overall security of the smart grid.« less
  • Smart grids are susceptible to cyber-attack as a result of new communication, control and computation techniques employed in the grid. In this paper, we characterize and analyze the resiliency of smart grid communication architecture, specifically an RF mesh based architecture, under cyber attacks. We analyze the resiliency of the communication architecture by studying the performance of high-level smart grid functions such as metering, and demand response which depend on communication. Disrupting the operation of these functions impacts the operational resiliency of the smart grid. Our analysis shows that it takes an attacker only a small fraction of meters to compromisemore » the communication resiliency of the smart grid. We discuss the implications of our result to critical smart grid functions and to the overall security of the smart grid.« less
  • OpenSHMEM is an effort to standardize the well-known SHMEM parallel programming library. The project aims to produce an open-source and portable SHMEM API and is led by ORNL and UH. In this paper, we optimize the current OpenSHMEM reference implementa- tion, based on GASNet, to achieve higher performance characteristics. To achieve these desired performance characteristics, we have redesigned an important component of the OpenSHMEM implementation, the network layer, to leverage a low-level communication library designed for imple- menting parallel programming models called UCCS. In particular, UCCS provides an interface and semantics such as native atomic operations and remote memory operationsmore » to better support PGAS programming models, including OpenSHMEM. Through the use of microbenchmarks, we evaluate this new OpenSHMEM implementation on various network metrics, including the latency of point-to-point and collective operations. Furthermore, we compare the performance of our OpenSHMEM imple- mentation with the state-of-the-art SGI SHMEM. Our results show that the atomic operations of our OpenSHMEM implementation outperform SGI s SHMEM implementation by 3%. Its RMA operations outperform both SGI s SHMEM and the original OpenSHMEM reference implemen- tation by as much as 18% and 12% for gets, and as much as 83% and 53% for puts.« less