skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Measurements and analytics of wide-area file transfers over dedicated connections

Abstract

Distributed scientific and big-data computations are becoming increasingly dependent on access to remote files. Wide-area file transfers are supported by two basic schemes: (i) application-level tools, such as GridFTP, that provide transport services between file systems housed at geographically separated sites, and (ii) file systems mounted over wide-area networks, using mechanisms such as LNet routers that make them transparently available. In both cases, the file transfer performance critically depends on the configuration consisting of host, file, IO, and disk subsystems, which are complex by themselves, as well as on their complex compositions implemented using buffers and IO-network data transitions. We present extensive file transfer rate measurements collected over dedicated 10 Gbps connections with 0-366 ms round-trip times, using GridFTP and XDD file transfer tools, and Lustre file system extended over wide-area networks using LNet routers. Our test configurations are composed of: three types of host systems; XFS, Lustre, and ext3 file systems; and Ethernet and SONET wide-area connections. We present analytics based on the convexity-concavity of throughput profiles which provide insights into throughput and its superior or inferior trend compared to linear interpolations. We propose the utilization-concavity coefficient, a scalar metric that characterizes the overall performance of any file transfermore » method consisting of specific configuration and scheme. Our results enable performance optimizations by highlighting the significant roles of (i) buffer sizes and parallelism in GridFTP and XDD, and (ii) buffer utilization and credit mechanism in LNet routers.« less

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [2];  [2];  [2]
  1. ORNL
  2. Argonne National Laboratory (ANL)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1509527
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: 20th International Conference on Distributed Computing and Networking (ICDCN 2019) - Bangalore, , India - 1/4/2019 5:00:00 AM-4/7/2019 4:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Liu, Zhengchun, Kettimuthu, Raj, and Foster, Ian. Measurements and analytics of wide-area file transfers over dedicated connections. United States: N. p., 2019. Web. doi:10.1145/3288599.3288641.
Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Liu, Zhengchun, Kettimuthu, Raj, & Foster, Ian. Measurements and analytics of wide-area file transfers over dedicated connections. United States. doi:10.1145/3288599.3288641.
Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Liu, Zhengchun, Kettimuthu, Raj, and Foster, Ian. Tue . "Measurements and analytics of wide-area file transfers over dedicated connections". United States. doi:10.1145/3288599.3288641. https://www.osti.gov/servlets/purl/1509527.
@article{osti_1509527,
title = {Measurements and analytics of wide-area file transfers over dedicated connections},
author = {Rao, Nageswara S. and Liu, Qiang and Sen, Satyabrata and Liu, Zhengchun and Kettimuthu, Raj and Foster, Ian},
abstractNote = {Distributed scientific and big-data computations are becoming increasingly dependent on access to remote files. Wide-area file transfers are supported by two basic schemes: (i) application-level tools, such as GridFTP, that provide transport services between file systems housed at geographically separated sites, and (ii) file systems mounted over wide-area networks, using mechanisms such as LNet routers that make them transparently available. In both cases, the file transfer performance critically depends on the configuration consisting of host, file, IO, and disk subsystems, which are complex by themselves, as well as on their complex compositions implemented using buffers and IO-network data transitions. We present extensive file transfer rate measurements collected over dedicated 10 Gbps connections with 0-366 ms round-trip times, using GridFTP and XDD file transfer tools, and Lustre file system extended over wide-area networks using LNet routers. Our test configurations are composed of: three types of host systems; XFS, Lustre, and ext3 file systems; and Ethernet and SONET wide-area connections. We present analytics based on the convexity-concavity of throughput profiles which provide insights into throughput and its superior or inferior trend compared to linear interpolations. We propose the utilization-concavity coefficient, a scalar metric that characterizes the overall performance of any file transfer method consisting of specific configuration and scheme. Our results enable performance optimizations by highlighting the significant roles of (i) buffer sizes and parallelism in GridFTP and XDD, and (ii) buffer utilization and credit mechanism in LNet routers.},
doi = {10.1145/3288599.3288641},
journal = {},
number = ,
volume = ,
place = {United States},
year = {2019},
month = {1}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: