Enabling Structured Exploration of Workflow Performance Variability in Extreme-Scale Environments
Workflows are taking an Workflows are taking an increasingly important role in orchestrating complex scientific processes in extreme scale and highly heterogeneous environments. However, to date we cannot reliably predict, understand, and optimize workflow performance. Sources of performance variability and in particular the interdependencies of workflow design, execution environment and system architecture are not well understood. While there is a rich portfolio of tools for performance analysis, modeling and prediction for single applications in homogenous computing environments, these are not applicable to workflows, due to the number and heterogeneity of the involved workflow and system components and their strong interdependencies. In this paper, we investigate workflow performance goals and identify factors that could have a relevant impact. Based on our analysis, we propose a new workflow performance provenance ontology, the Open Provenance Model-based WorkFlow Performance Provenance, or OPM-WFPP, that will enable the empirical study of workflow performance characteristics and variability including complex source attribution.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1339919
- Report Number(s):
- PNNL-SA-120941; KJ0404000
- Resource Relation:
- Conference: 8th Workshop on Many-Task Computing on Clouds, Grids, and Supercomputers (MTAGS) 2015, November 15, 2015, Austin, Texas
- Country of Publication:
- United States
- Language:
- English
Similar Records
Capturing provenance as a diagnostic tool for workflow performance evaluation and optimization
Capturing provenance as a diagnostic tool for workflow performance evaluation and optimization