Data Provenance Hybridization Supporting Extreme-Scale Scientific WorkflowApplications

Conference · Mon Nov 21 04:00:00 EST 2016

DOI:https://doi.org/10.1109/NYSDS.2016.7747819· OSTI ID:1344661

Elsethagen, Todd O.; Stephan, Eric G.; Raju, Bibi; Schram, Malachi; Macduff, Matt C.; Kerbyson, Darren J.; Kleese-Van Dam, Kerstin; Singh, Alok; Altintas, Ilkay

As high performance computing (HPC) infrastructures continue to grow in capability and complexity, so do the applications that they serve. HPC and distributed-area computing (DAC) (e.g. grid and cloud) users are looking increasingly toward workflow solutions to orchestrate their complex application coupling, pre- and post-processing needs To gain insight and a more quantitative understanding of a workflow’s performance our method includes not only the capture of traditional provenance information, but also the capture and integration of system environment metrics helping to give context and explanation for a workflow’s execution. In this paper, we describe IPPD’s provenance management solution (ProvEn) and its hybrid data store combining both of these data provenance perspectives.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (US)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1344661
• Report Number(s): PNNL-SA-119959; KJ0404000
• Country of Publication: United States
• Language: English

Related Subjects

provenance