Measurements of file transfer rates over dedicated long-haul connections
- ORNL
Wide-area file transfers are an integral part of several High-Performance Computing (HPC) scenarios. Dedicated network connections with high capacity, low loss rate and low competing traffic, are increasingly being provisioned over current HPC infrastructures to support such transfers. To gain insights into these file transfers, we collected transfer rate measurements for Lustre and xfs file systems between dedicated multi-core servers over emulated 10 Gbps connections with round trip times (rtt) in 0-366 ms range. Memory transfer throughput over these connections is measured using iperf, and file IO throughput on host systems is measured using xddprof. We consider two file system configurations: Lustre over IB network and xfs over SSD connected to PCI bus. Files are transferred using xdd across these connections, and the transfer rates are measured, which indicate the need to jointly optimize the connection and host file IO parameters to achieve peak transfer rates. In particular, these measurements indicate that (i) peak file transfer rate is lower than peak connection and host IO throughput, in some cases by as much as 50% or lower, (ii) xdd request sizes that achieve peak throughput for host file IO do not necessarily lead to peak file transfer rates, and (iii) parallelism in host IO and TCP transport does not always improve the file transfer rates.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- Work for Others (WFO)
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 1327686
- Resource Relation:
- Conference: 2nd International Workshop on the Lustre Ecosystem: Enhancing Lustre Support for Diverse Workloads, Baltimore, MD, USA, 20160308, 20160309
- Country of Publication:
- United States
- Language:
- English
Similar Records
Measurements and analytics of wide-area file transfers over dedicated connections
On Analytics of File Transfer Rates over Dedicated Wide-Area Connections