skip to main content

Title: High-Performance data flows using analytical models and measurements

The combination of analytical models and measurements provide practical configurations and parameters to achieve high data transport rates: (a) buffer sizes and number of parallel streams for improved memory and file transfer rates, (b) Hamilton and Scalable TCP congestion control modules for memory transfers in place of default CUBIC, and (c) direct IO mode for Lustre file systems for wide-area transfers. Conventional parameter selection using full sweeps is impractical in many cases since it takes months. By exploiting the unimodality of throughput profiles, we developed the d-w method that significantly reduces the number of measurements needed for parameter identification. This heuristic method was effective in practice in reducing the measurements by about 90% for Lustre and XFS file transfers.
Authors:
 [1] ;  [2] ;  [2] ;  [3] ;  [3] ;  [4]
  1. ORNL
  2. University of Massachusetts
  3. Argonne National Laboratory (ANL)
  4. Los Alamos National Laboratory (LANL)
Publication Date:
OSTI Identifier:
1327697
DOE Contract Number:
AC05-00OR22725
Resource Type:
Conference
Resource Relation:
Conference: Workshop on Modeling & Simulation of Systems and Applications, Seattle, WA, USA, 20160810, 20160812
Research Org:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States
Language:
English