Data Provenance Hybridization Supporting Extreme-Scale Scientific WorkflowApplications
As high performance computing (HPC) infrastructures continue to grow in capability and complexity, so do the applications that they serve. HPC and distributed-area computing (DAC) (e.g. grid and cloud) users are looking increasingly toward workflow solutions to orchestrate their complex application coupling, pre- and post-processing needs To gain insight and a more quantitative understanding of a workflow’s performance our method includes not only the capture of traditional provenance information, but also the capture and integration of system environment metrics helping to give context and explanation for a workflow’s execution. In this paper, we describe IPPD’s provenance management solution (ProvEn) and its hybrid data store combining both of these data provenance perspectives.
- Research Organization:
- Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1344661
- Report Number(s):
- PNNL-SA-119959; KJ0404000
- Resource Relation:
- Conference: New York Scientific Data Summit (NYSDS 2016), August 14-17, 2016, New York
- Country of Publication:
- United States
- Language:
- English
Similar Records
Integrated End-to-end Performance Prediction and Diagnosis for Extreme Scientific Workflows
ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum