High-Performance data flows using analytical models and measurements

Conference · Fri Jan 01 00:00:00 EST 2016

OSTI ID:1327697

Rao, Nageswara S ^[1]; Towlsey, D. ^[2]; Vardoyan, G. ^[2]; Kettimuthu, R. ^[3]; Foster, I. ^[3]; Settlemyer, Bradley ^[4]

---

1. ORNL
2. University of Massachusetts
3. Argonne National Laboratory (ANL)
4. Los Alamos National Laboratory (LANL)

The combination of analytical models and measurements provide practical configurations and parameters to achieve high data transport rates: (a) buffer sizes and number of parallel streams for improved memory and file transfer rates, (b) Hamilton and Scalable TCP congestion control modules for memory transfers in place of default CUBIC, and (c) direct IO mode for Lustre file systems for wide-area transfers. Conventional parameter selection using full sweeps is impractical in many cases since it takes months. By exploiting the unimodality of throughput profiles, we developed the d-w method that significantly reduces the number of measurements needed for parameter identification. This heuristic method was effective in practice in reducing the measurements by about 90% for Lustre and XFS file transfers.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1327697
• Resource Relation: Conference: Workshop on Modeling & Simulation of Systems and Applications, Seattle, WA, USA, 20160810, 20160812
• Country of Publication: United States
• Language: English