skip to main content

SciTech ConnectSciTech Connect

Title: High-Performance data flows using analytical models and measurements

The combination of analytical models and measurements provide practical configurations and parameters to achieve high data transport rates: (a) buffer sizes and number of parallel streams for improved memory and file transfer rates, (b) Hamilton and Scalable TCP congestion control modules for memory transfers in place of default CUBIC, and (c) direct IO mode for Lustre file systems for wide-area transfers. Conventional parameter selection using full sweeps is impractical in many cases since it takes months. By exploiting the unimodality of throughput profiles, we developed the d-w method that significantly reduces the number of measurements needed for parameter identification. This heuristic method was effective in practice in reducing the measurements by about 90% for Lustre and XFS file transfers.
 [1] ;  [2] ;  [2] ;  [3] ;  [3] ;  [4]
  1. ORNL
  2. University of Massachusetts
  3. Argonne National Laboratory (ANL)
  4. Los Alamos National Laboratory (LANL)
Publication Date:
OSTI Identifier:
DOE Contract Number:
Resource Type:
Resource Relation:
Conference: Workshop on Modeling & Simulation of Systems and Applications, Seattle, WA, USA, 20160810, 20160812
Research Org:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org:
USDOE Office of Science (SC)
Country of Publication:
United States