Computational Reproducibility of Scientific Workflows at Extreme Scales

Pouchard, Line C.; Baldwin, Sterling A.; Elsethagen, Todd O.; Jha, Shantenu; Raju, Bibi; Stephan, Eric G.; Tang, Li; Kleese van Dam, Kerstin

doi:10.1177/1094342019839124

Computational Reproducibility of Scientific Workflows at Extreme Scales

Journal Article · Sun Sep 01 04:00:00 EDT 2019 · International Journal of High Performance Computing Applications

DOI:https://doi.org/10.1177/1094342019839124· OSTI ID:1577046

Pouchard, Line C. ^[1]; Baldwin, Sterling A. ^[2]; Elsethagen, Todd O. ^[3]; Jha, Shantenu ^[4]; ^[3]; ^[3]; Tang, Li ^[5]; Kleese van Dam, Kerstin ^[4]

Oark Ridge National Laboratory
LAWRENCE LIVERMORE
BATTELLE (PACIFIC NW LAB)
Brookhaven National Laboratory
Los Alamos National Laboratory

We propose an approach for improved reproducibility that includes capturing and relating provenance characteristics and performance metrics, in a hybrid queriable system, the ProvEn server. The system capabilities are illustrated on two use cases: scientific reproducibility of results in the ACME climate simulations and performance reproducibility in molecular dynamics workflows on HPC computing platforms.

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-76RL01830

OSTI ID:: 1577046

Report Number(s):: PNNL-SA-137882

Journal Information:: International Journal of High Performance Computing Applications, Journal Name: International Journal of High Performance Computing Applications Journal Issue: 5 Vol. 33

Country of Publication:: United States

Language:: English

References (29)

The Earth System Grid Federation: An open infrastructure for access to distributed geospatial data Cinquini, Luca; Crichton, Daniel; Mattmann, Chris Future Generation Computer Systems, Vol. 36 https://doi.org/10.1016/j.future.2013.07.002	journal	July 2014
Enhancing reproducibility for computational methods Stodden, V.; McNutt, M.; Bailey, D. H. Science, Vol. 354, Issue 6317 https://doi.org/10.1126/science.aah6168	journal	December 2016
Data provenance hybridization supporting extreme-scale scientific workflow applications Elsethagen, Todd; Stephan, Eric; Raju, Bibi 2016 New York Scientific Data Summit (NYSDS) https://doi.org/10.1109/NYSDS.2016.7747819	conference	August 2016
R3: repeatability, reproducibility and rigor Vitek, Jan; Kalibera, Tomas ACM SIGPLAN Notices, Vol. 47, Issue 4a https://doi.org/10.1145/2442776.2442781	journal	March 2012
Ten Simple Rules for Reproducible Computational Research Sandve, Geir Kjetil; Nekrutenko, Anton; Taylor, James PLoS Computational Biology, Vol. 9, Issue 10 https://doi.org/10.1371/journal.pcbi.1003285	journal	October 2013
Numerical Reproducibility and Accuracy at ExaScale Demmel, J. 2013 IEEE 21st Symposium on Computer Arithmetic (ARITH) https://doi.org/10.1109/ARITH.2013.43	conference	April 2013
High Performance MPI Library for Container-Based HPC Cloud on InfiniBand Clusters Zhang, Jie; Lu, Xiaoyi; Panda, Dhabaleswar K. 2016 45th International Conference on Parallel Processing (ICPP) https://doi.org/10.1109/ICPP.2016.38	conference	August 2016
Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks: HELLO ADIOS Liu, Qing; Logan, Jeremy; Tian, Yuan Concurrency and Computation: Practice and Experience, Vol. 26, Issue 7 https://doi.org/10.1002/cpe.3125	journal	August 2013
Fast Parallel Algorithms for Short-Range Molecular Dynamics Plimpton, Steve Journal of Computational Physics, Vol. 117, Issue 1 https://doi.org/10.1006/jcph.1995.1039	journal	March 1995
Capturing provenance as a diagnostic tool for workflow performance evaluation and optimization Pouchard, Line; Malik, Abid; Dam, Huub Van 2017 New York Scientific Data Summit (NYSDS) https://doi.org/10.1109/NYSDS.2017.8085043	conference	August 2017
Performance Visualization for TAU Instrumented Scientific Workflows [Performance Visualization for TAU Instrumented Scientific Workflows] Xie, Cong; Xu, Wei; Ha, Sungsoo International Conference on Information Visualization Theory and Applications, Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications https://doi.org/10.5220/0006646803330340	conference	January 2018
An introduction to Docker for reproducible research Boettiger, Carl ACM SIGOPS Operating Systems Review, Vol. 49, Issue 1 https://doi.org/10.1145/2723872.2723882	journal	January 2015
Prescriptive provenance for streaming analysis of workflows at scale Pouchard, Line; Huck, Kevin; Matyasfalvi, Gyorgy 2018 New York Scientific Data Summit (NYSDS) https://doi.org/10.1109/NYSDS.2018.8538951	conference	August 2018
Towards trustworthy testbeds thanks to throughout testing Nussbaum, Lucas 2017 IEEE International Parallel and Distributed Processing Symposium: Workshops (IPDPSW), 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) https://doi.org/10.1109/IPDPSW.2017.101	conference	May 2017
Testbeds Support for Reproducible Research Nussbaum, Lucas Proceedings of the Reproducibility Workshop on ZZZ - Reproducibility '17 https://doi.org/10.1145/3097766.3097773	conference	January 2017
Toward the Geoscience Paper of the Future: Best practices for documenting and sharing research from data to software to provenance Gil, Yolanda; David, Cédric H.; Demir, Ibrahim Earth and Space Science, Vol. 3, Issue 10 https://doi.org/10.1002/2015EA000136	journal	October 2016
A web interface for XALT log data analysis Huang, Ruizhu; Xu, Weijia; McLay, Robert Proceedings of the XSEDE16 on Diversity, Big Data, and Science at Scale - XSEDE16 https://doi.org/10.1145/2949550.2949560	conference	January 2016
Streaming spectral clustering Yoo, Shinjae; Huang, Hao; Kasiviswanathan, Shiva Prasad 2016 IEEE 32nd International Conference on Data Engineering (ICDE) https://doi.org/10.1109/ICDE.2016.7498277	conference	May 2016
User Environment Tracking and Problem Detection with XALT Agrawal, Kapil; Fahey, Mark R.; McLay, Robert 2014 First International Workshop on HPC User Support Tools (HUST) https://doi.org/10.1109/HUST.2014.6	conference	November 2014
Guidelines for evaluating and expressing the uncertainty of NIST measurement results Taylor, B. N. NIST Technical Note 1297 1994 Edition https://doi.org/10.6028/NIST.TN.1297	report	January 1994
Data storage and sharing for the long tail of science Zhang, Boyu; Pouchard, Line C.; Smith, Preston M. 2016 New York Scientific Data Summit (NYSDS) https://doi.org/10.1109/NYSDS.2016.7747811	conference	August 2016
Packing experiments for sharing and publication Chirigati, Fernando; Shasha, Dennis; Freire, Juliana Proceedings of the 2013 international conference on Management of data - SIGMOD '13 https://doi.org/10.1145/2463676.2465269	conference	January 2013
Leveraging large sensor streams for robust cloud control Singh, Alok; Stephan, Eric; Elsethagen, Todd 2016 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData.2016.7840839	conference	December 2016
The Spack package manager: bringing order to HPC software chaos Gamblin, Todd; LeGendre, Matthew; Collette, Michael R. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15 https://doi.org/10.1145/2807591.2807623	conference	January 2015
Reproducible Research in Computational Science Peng, R. D. Science, Vol. 334, Issue 6060 https://doi.org/10.1126/science.1213847	journal	December 2011
Provenance: An Introduction to PROV Moreau, Luc; Groth, Paul Synthesis Lectures on the Semantic Web: Theory and Technology, Vol. 3, Issue 4 https://doi.org/10.2200/S00528ED1V01Y201308WBE007	journal	September 2013
Extreme Heterogeneity 2018 - Productive Computational Science in the Era of Extreme Heterogeneity: Report for DOE ASCR Workshop on Extreme Heterogeneity Vetter, Jeffrey S.; Brightwell, Ron; Gokhale, Maya https://doi.org/10.2172/1473756	report	December 2018
Numerical reproducibility for the parallel reduction on multi- and many-core architectures Collange, Sylvain; Defour, David; Graillat, Stef Parallel Computing, Vol. 49 https://doi.org/10.1016/j.parco.2015.09.001	journal	November 2015
Automated Capture of Experiment Context for Easier Reproducibility in Computational Research Davison, Andrew Computing in Science & Engineering, Vol. 14, Issue 4 https://doi.org/10.1109/MCSE.2012.41	journal	July 2012

Similar Records

Computational reproducibility of scientific workflows at extreme scales

Journal Article · Mon Apr 08 00:00:00 EDT 2019 · International Journal of High Performance Computing Applications · OSTI ID:1542776

Data Provenance Hybridization Supporting Extreme-Scale Scientific WorkflowApplications

Conference · Sun Nov 20 23:00:00 EST 2016 · OSTI ID:1344661

Enabling HPC Scientific Workflows for Serverless

Conference · Fri Nov 01 00:00:00 EDT 2024 · OSTI ID:2538241

Related Subjects

provenance
Reproducibility
Climate & Earth System Models
-High Performance Computing

Computational Reproducibility of Scientific Workflows at Extreme Scales

Citation Formats

References (29)

Similar Records

Related Subjects