skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections

Abstract

File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.

Authors:
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1];  [1]; ORCiD logo [1];  [2];  [3];  [4];  [5];  [6]
  1. ORNL
  2. University of Chicago
  3. Argonne National Laboratory (ANL)
  4. Los Alamos National Laboratory (LANL)
  5. University of Memphis
  6. Harrisburg University
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1399536
DOE Contract Number:  
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: 18th IEEE International Conference on High Performance Computing and Communications (HPCC) - Sydney, , Australia - 12/12/2016 5:00:00 AM-12/14/2016 5:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Hinkel, Gregory Carl, Imam, Neena, Foster, Ian, Kettimuthu, R., Settlemyer, Bradley, Wu, Qishi, and Yun, Daqing. Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections. United States: N. p., 2016. Web. doi:10.1109/HPCC-SmartCity-DSS.2016.0038.
Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Hinkel, Gregory Carl, Imam, Neena, Foster, Ian, Kettimuthu, R., Settlemyer, Bradley, Wu, Qishi, & Yun, Daqing. Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections. United States. https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0038
Rao, Nageswara S., Liu, Qiang, Sen, Satyabrata, Hinkel, Gregory Carl, Imam, Neena, Foster, Ian, Kettimuthu, R., Settlemyer, Bradley, Wu, Qishi, and Yun, Daqing. Thu . "Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections". United States. https://doi.org/10.1109/HPCC-SmartCity-DSS.2016.0038. https://www.osti.gov/servlets/purl/1399536.
@article{osti_1399536,
title = {Experimental Analysis of File Transfer Rates over Wide-Area Dedicated Connections},
author = {Rao, Nageswara S. and Liu, Qiang and Sen, Satyabrata and Hinkel, Gregory Carl and Imam, Neena and Foster, Ian and Kettimuthu, R. and Settlemyer, Bradley and Wu, Qishi and Yun, Daqing},
abstractNote = {File transfers over dedicated connections, supported by large parallel file systems, have become increasingly important in high-performance computing and big data workflows. It remains a challenge to achieve peak rates for such transfers due to the complexities of file I/O, host, and network transport subsystems, and equally importantly, their interactions. We present extensive measurements of disk-to-disk file transfers using Lustre and XFS file systems mounted on multi-core servers over a suite of 10 Gbps emulated connections with 0-366 ms round trip times. Our results indicate that large buffer sizes and many parallel flows do not always guarantee high transfer rates. Furthermore, large variations in the measured rates necessitate repeated measurements to ensure confidence in inferences based on them. We propose a new method to efficiently identify the optimal joint file I/O and network transport parameters using a small number of measurements. We show that for XFS and Lustre with direct I/O, this method identifies configurations achieving 97% of the peak transfer rate while probing only 12% of the parameter space.},
doi = {10.1109/HPCC-SmartCity-DSS.2016.0038},
url = {https://www.osti.gov/biblio/1399536}, journal = {},
number = ,
volume = ,
place = {United States},
year = {2016},
month = {12}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: