Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows, In: 2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2)

Conference · · PROCEEDINGS OF SECOND INTERNATIONAL WORKSHOP ON EXTREME SCALE PROGRAMMING MODELS AND MIDDLEWARE (ESPM2 2016)
 [1];  [1];  [1];  [2];  [3];  [4];  [5]
  1. Rutgers University, Piscataway NJ (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
  3. Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
  4. Sci. Comput. & Imaging Inst., University of Utah, Salt Lake City, UT (United States)
  5. Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN (United States)
+ Show Author Affiliations
Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.
Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Rutgers Univ., Piscataway, NJ (United States); UT-Battelle LLC/ORNL, Oak Ridge, TN (Unted States)
Sponsoring Organization:
USDOE Office of Science (SC)
DOE Contract Number:
FC02-06ER54857; AC05-00OR22725; SC0007455
OSTI ID:
1567424
Conference Information:
Journal Name: PROCEEDINGS OF SECOND INTERNATIONAL WORKSHOP ON EXTREME SCALE PROGRAMMING MODELS AND MIDDLEWARE (ESPM2 2016)
Country of Publication:
United States
Language:
English

Similar Records

Enabling In-situ Execution of Coupled Scientific Workflow on Multi-core Platform
Conference · Thu Aug 16 00:00:00 EDT 2012 · 2012 IEEE 26TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM (IPDPS) · OSTI ID:1567312
Asynchronous Execution of Heterogeneous Tasks in ML-Driven HPC Workflows
Conference · Fri May 19 00:00:00 EDT 2023 · OSTI ID:2333668
Addressing Global Data Dependencies in Heterogeneous Asynchronous Runtime Systems on GPUs
Conference · Wed Nov 01 00:00:00 EDT 2017 · Proceedings of the 3rd International IEEE Workshop on Extreme Scale Programming Models and Middleware · OSTI ID:1582428

Related Subjects

Computer Science