skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Data Transfer Advisor with Transport Profiling Optimization

Abstract

The network infrastructures have been rapidly upgraded in many high-performance networks (HPNs). However, such infrastructure investment has not led to corresponding performance improvement in big data transfer, especially at the application layer, largely due to the complexity of optimizing transport control on end hosts. We design and implement ProbData, a PRofiling Optimization Based DAta Transfer Advisor, to help users determine the most effective data transfer method with the most appropriate control parameter values to achieve the best data transfer performance. ProbData employs a profiling optimization based approach to exploit the optimal operational zone of various data transfer methods in support of big data transfer in extreme scale scientific applications. We present a theoretical framework of the optimized profiling approach employed in ProbData as wellas its detailed design and implementation. The advising procedure and performance benefits of ProbData are illustrated and evaluated by proof-of-concept experiments in real-life networks.

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2];  [3];  [4];  [4]
  1. ORNL
  2. Harrisburg University
  3. New Jersey Institute of Technology
  4. Argonne National Laboratory (ANL)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1399418
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: 42nd Annual IEEE Conference on Local Computer Networks - Singapore, , Singapore - 11/9/2017 10:00:00 AM-11/12/2017 10:00:00 AM
Country of Publication:
United States
Language:
English

Citation Formats

Rao, Nageswara S., Liu, Qiang, Yun, Daqing, Wu, Chase Qishi, Kettimuthu, R., and Jung, Eun-sung. Data Transfer Advisor with Transport Profiling Optimization. United States: N. p., 2017. Web.
Rao, Nageswara S., Liu, Qiang, Yun, Daqing, Wu, Chase Qishi, Kettimuthu, R., & Jung, Eun-sung. Data Transfer Advisor with Transport Profiling Optimization. United States.
Rao, Nageswara S., Liu, Qiang, Yun, Daqing, Wu, Chase Qishi, Kettimuthu, R., and Jung, Eun-sung. Wed . "Data Transfer Advisor with Transport Profiling Optimization". United States. doi:. https://www.osti.gov/servlets/purl/1399418.
@article{osti_1399418,
title = {Data Transfer Advisor with Transport Profiling Optimization},
author = {Rao, Nageswara S. and Liu, Qiang and Yun, Daqing and Wu, Chase Qishi and Kettimuthu, R. and Jung, Eun-sung},
abstractNote = {The network infrastructures have been rapidly upgraded in many high-performance networks (HPNs). However, such infrastructure investment has not led to corresponding performance improvement in big data transfer, especially at the application layer, largely due to the complexity of optimizing transport control on end hosts. We design and implement ProbData, a PRofiling Optimization Based DAta Transfer Advisor, to help users determine the most effective data transfer method with the most appropriate control parameter values to achieve the best data transfer performance. ProbData employs a profiling optimization based approach to exploit the optimal operational zone of various data transfer methods in support of big data transfer in extreme scale scientific applications. We present a theoretical framework of the optimized profiling approach employed in ProbData as wellas its detailed design and implementation. The advising procedure and performance benefits of ProbData are illustrated and evaluated by proof-of-concept experiments in real-life networks.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Nov 01 00:00:00 EDT 2017},
month = {Wed Nov 01 00:00:00 EDT 2017}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share: