skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: On Data Transfers Over Wide-Area Dedicated Connections

Abstract

Dedicated wide-area network connections are employed in big data and high-performance computing scenarios, since the absence of cross-traffic promises to make it easier to analyze and optimize data transfers over them. However, nonlinear transport dynamics and end-system complexity due to multi-core hosts and distributed file systems make these tasks surprisingly challenging. We present an overview of methods to analyze memory and disk file transfers using extensive measurements over 10 Gbps physical and emulated connections with 0–366 ms round trip times (RTTs). For memory transfers, we derive performance profiles of TCP and UDT throughput as a function of RTT, which show concave regions in contrast to entirely convex regions predicted by previous models. These highly desirable concave regions can be expanded by utilizing large buffers and more parallel flows. We also present Poincar´e maps and Lyapunov exponents of TCP and UDT throughput traces that indicate complex throughput dynamics. For disk file transfers, we show that throughput can be optimized using a combination of parallel I/O and network threads under direct I/O mode. Our initial throughput measurements of Lustre filesystems mounted over long-haul connections using LNet routers show convex profiles indicative of I/O limits.

Authors:
ORCiD logo [1]; ORCiD logo [1]
  1. ORNL
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1399399
DOE Contract Number:
AC05-00OR22725
Resource Type:
Book
Country of Publication:
United States
Language:
English

Citation Formats

Rao, Nageswara S., and Liu, Qiang. On Data Transfers Over Wide-Area Dedicated Connections. United States: N. p., 2018. Web.
Rao, Nageswara S., & Liu, Qiang. On Data Transfers Over Wide-Area Dedicated Connections. United States.
Rao, Nageswara S., and Liu, Qiang. 2018. "On Data Transfers Over Wide-Area Dedicated Connections". United States. doi:.
@article{osti_1399399,
title = {On Data Transfers Over Wide-Area Dedicated Connections},
author = {Rao, Nageswara S. and Liu, Qiang},
abstractNote = {Dedicated wide-area network connections are employed in big data and high-performance computing scenarios, since the absence of cross-traffic promises to make it easier to analyze and optimize data transfers over them. However, nonlinear transport dynamics and end-system complexity due to multi-core hosts and distributed file systems make these tasks surprisingly challenging. We present an overview of methods to analyze memory and disk file transfers using extensive measurements over 10 Gbps physical and emulated connections with 0–366 ms round trip times (RTTs). For memory transfers, we derive performance profiles of TCP and UDT throughput as a function of RTT, which show concave regions in contrast to entirely convex regions predicted by previous models. These highly desirable concave regions can be expanded by utilizing large buffers and more parallel flows. We also present Poincar´e maps and Lyapunov exponents of TCP and UDT throughput traces that indicate complex throughput dynamics. For disk file transfers, we show that throughput can be optimized using a combination of parallel I/O and network threads under direct I/O mode. Our initial throughput measurements of Lustre filesystems mounted over long-haul connections using LNet routers show convex profiles indicative of I/O limits.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = 2018,
month = 1
}

Book:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this book.

Save / Share:
  • Long haul data transfers require the optimization and balancing of performances of host and storage systems as well as network transport. An assessment of such transport methods requires a systematic generation of throughput profiles from measurements collected over different system parameters and connection lengths. We describe host and storage systems to support wide-area I/O transfers at 10 Gbps, and present measurements of memory and disk transfer throughputs over suites of physical and emulated connections of several thousands of miles. The physical connections are limited by the infrastructure and incur significant costs. The emulated connections could be of arbitrary lengths atmore » significantly lower costs but only approximate the physical connections. We present a differential regression method to estimate the differences between the performance profiles of physical and emulated connections, and then to estimate 'physical' profiles from emulated measurements. We present a systematic analysis of wide-area memory and disk transfer throughput measurements, and establish that robust estimates of physical profiles can be generated using much less expensive emulated connections.« less
  • Large distributed systems such as Computational/Data Grids require large amounts of data to be co-located with the computing facilities for processing. Ensuring that the data is there in time for the computation in today's Internet is a massive problem. From our work developing a scalable distributed network cache, we have gained experience with techniques necessary to achieve high data throughput over high bandwidth Wide Area Networks (WAN). In this paper, we discuss several hardware and software design techniques and issues, and then describe their application to an implementation of an enhanced FTP protocol called GridFTP. We also describe results frommore » two applications using these techniques, which were obtained at the Supercomputing 2000 conference.« less
  • No abstract prepared.
  • The authors have analyzed H + N{sub 2}O by considering formation of chemically activated adducts with a master equation formalism to account for collisional stabilization. They consider four product channels for the addition of H atoms to N{sub 2}O: H + N{sub 2}O = HNNO; H + N{sub 2}O = N{sub 2} + OH; H + N{sub 2}O = NH + NO; and H + N{sub 2}O = NNH + O. The calculations are in good agreement with the available data. At higher temperatures, the NH + NO channel is predicted to be comparable to the N{sub 2} + OHmore » channel, while the NNH + O channel is less important but still significant. An interesting feature of the calculations is the suggestion that HNNO formation could be important at lower temperatures. This same potential energy surface is also used to analyze the reaction between NH and NO. There are three principal product channels: NH + NO = N{sub 2} + OH; NH + NO = N{sub 2}O + H; and NH + NO = NNH + O. Predictions are in good agreement with the available data. Below 3,000 K, H + N{sub 2}O is predicted to be the major channel for the NH + NO reaction. However, the NNH + O channel is predicted to be important at combustion temperatures and should be considered for inclusion in combustion models. Rate constants are given for each reaction.« less