ACM TOMS replicated computational results initiative

Heroux, Michael Allen

doi:10.1145/2743015

Title: ACM TOMS replicated computational results initiative

Abstract

In this study, the scientific community relies on the peer review process for assuring the quality of published material, the goal of which is to build a body of work we can trust. Computational journals such as The ACM Transactions on Mathematical Software (TOMS) use this process for rigorously promoting the clarity and completeness of content, and citation of prior work. At the same time, it is unusual to independently confirm computational results.

Authors:

Heroux, Michael Allen ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Publication Date:: Wed Jun 03 00:00:00 EDT 2015

Research Org.:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1235340

Report Number(s):: SAND-2015-20790J
Journal ID: ISSN 0098-3500; 558227

Grant/Contract Number:: AC04-94AL85000

Resource Type:: Accepted Manuscript

Journal Name:: ACM Transactions on Mathematical Software

Additional Journal Information:: Journal Volume: 41; Journal Issue: 3; Journal ID: ISSN 0098-3500

Publisher:: ACM

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; replicated computational results; reproducibility; publication

Citation Formats


                    Heroux, Michael Allen. ACM TOMS replicated computational results initiative.  United States: N. p., 2015. 
Web.  doi:10.1145/2743015.

Copy to clipboard


                    Heroux, Michael Allen. ACM TOMS replicated computational results initiative.  United States.  https://doi.org/10.1145/2743015

Copy to clipboard


                    Heroux, Michael Allen. Wed .  
"ACM TOMS replicated computational results initiative".  United States.  https://doi.org/10.1145/2743015.  https://www.osti.gov/servlets/purl/1235340.

Copy to clipboard


                    
@article{osti_1235340,

  title        = {ACM TOMS replicated computational results initiative},

  author       = {Heroux, Michael Allen},

  abstractNote = {In this study, the scientific community relies on the peer review process for assuring the quality of published material, the goal of which is to build a body of work we can trust. Computational journals such as The ACM Transactions on Mathematical Software (TOMS) use this process for rigorously promoting the clarity and completeness of content, and citation of prior work. At the same time, it is unusual to independently confirm computational results.},

  doi          = {10.1145/2743015},

  journal      = {ACM Transactions on Mathematical Software},

  number       = 3,

  volume       = 41,

  place        = {United States},

  year         = {Wed Jun 03 00:00:00 EDT 2015},

  month        = {Wed Jun 03 00:00:00 EDT 2015}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1145/2743015

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 10 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Testing Scientific Programs
journal, July 2012

Dubois, Paul F.
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.84

Virtual Appliances, Cloud Computing, and Reproducible Research
journal, July 2012

Howe, Bill
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.62

Replicated Computational Results (RCR) Report for “BLIS: A Framework for Rapidly Instantiating BLAS Functionality”
journal, June 2015

Willenbring, James M.
ACM Transactions on Mathematical Software, Vol. 41, Issue 3
DOI: 10.1145/2738033

Making Computations and Publications Reproducible with VisTrails
journal, July 2012

Freire, Juliana; Silva, Claudio T.
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.76

Three Dream Applications of Verifiable Computational Results
journal, July 2012

Gavish, Matan; Donoho, David
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.65

Phase II failures: 2008–2010
journal, April 2011

Arrowsmith, John
Nature Reviews Drug Discovery, Vol. 10, Issue 5
DOI: 10.1038/nrd3439

Automated Capture of Experiment Context for Easier Reproducibility in Computational Research
journal, July 2012

Davison, Andrew
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.41

Code Sharing Is Associated with Research Impact in Image Processing
journal, July 2012

Vandewalle, Patrick
Computing in Science & Engineering, Vol. 14, Issue 4
DOI: 10.1109/MCSE.2012.63

Replicated Computational Results Certified BLIS: A Framework for Rapidly Instantiating BLAS Functionality
journal, June 2015

Zee, Field G. VAN; Geijn, Robert A. VAN DE
ACM Transactions on Mathematical Software, Vol. 41, Issue 3
DOI: 10.1145/2764454

Works referencing / citing this record:

“Reproducible” Research in Mathematical Sciences Requires Changes in our Peer Review Culture and Modernization of our Current Publication Approach
journal, September 2018

Schnell, Santiago
Bulletin of Mathematical Biology, Vol. 80, Issue 12
DOI: 10.1007/s11538-018-0500-9

50 Years of Data Science
journal, October 2017

Donoho, David
Journal of Computational and Graphical Statistics, Vol. 26, Issue 4
DOI: 10.1080/10618600.2017.1384734

Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge
journal, January 2019

Liu, David M.; Salganik, Matthew J.
Socius: Sociological Research for a Dynamic World, Vol. 5
DOI: 10.1177/2378023119849803

50 Years of Data Science
text, January 2018

Donoho, David
QUBES Educational Resources
DOI: 10.25334/q42b0d

Similar Records in DOE PAGES and OSTI.GOV collections:

Computational Biology Support: RECOMB Conference Series (Conference Support)

Other Waterman, Michael

This funding was support for student and postdoctoral attendance at the Annual Recomb Conference from 2001 to 2005. The RECOMB Conference series was founded in 1997 to provide a scientific forum for theoretical advances in computational biology and their applications in molecular biology and medicine. The conference series aims at attracting research contributions in all areas of computational molecular biology. Typical, but not exclusive, the topics of interest are: Genomics, Molecular sequence analysis, Recognition of genes and regulatory elements, Molecular evolution, Protein structure, Structural genomics, Gene Expression, Gene Networks, Drug Design, Combinatorial libraries, Computational proteomics, and Structural and functional genomics.more »« less
Full Text Available
Mentorship: The Education-Research Continuum

Journal Article Correll, D - Journal of Undergraduate Research, VIII, VIII, January 1, 2009, pp. 4-5

Mentoring of science students stems naturally from the intertwined link between science education and science research. In fact, the mentoring relationship between a student and a scientist may be thought of analogically as a type of double helix forming the 'DNA' that defines the blueprint for the next generation of scientists. Although this analogy would not meet the rigorous tests commonly used for exploring the natural laws of the universe, the image depicted does capture how creating and sustaining the future science workforce benefits greatly from the continuum between education and research. The path science students pursue from their educationmore »« less
Full Text Available
Proceedings of the seventh annual ACM-SIAM symposium on discrete algorithms

Conference

The papers in this volume were presented at the Seventh Annual ACM-SIAM Symposium on Discrete Algorithms, held January 28-30, 1996 in Atlanta, Georgia. The Symposium was jointly sponsored by the ACM Special Interest Group on Algorithms and Computation Theory and the SIAM Activity Group on Discrete Mathematics. The Program Committee met September 10-11, 1995 and selected the papers in this volume from 232 abstracts submitted in response to the call for papers. The selection was based on originality, quality, and relevance to the study of algorithms. The submissions were not refereed, and many of them represent reports of continuing research.more »« less
Office of the Chief Financial Officer Annual Report 2010

Technical Report Fernandez, Jeffrey

In March, a review team consisting of CFOs from other national laboratories, industry, and members of the University of California Office of the President (UCOP) convened for three days to conduct a comprehensive peer review of the OCFO. This was the first time in almost a decade that the financial operations of the Laboratory had been reviewed. The Committee relayed their observations on our strengths, and their very thoughtful recommendations for improvement, which we are actively pursuing. These improvements, when implemented, will benefit the entire Laboratory for many years to come. The complete report is available on the OCFO websitemore »« less
https://doi.org/10.2172/1003827

Full Text Available
Accomplishments in the Trident Laser Facility

Technical Report Fernandez, Juan Carlos

Trident has been an extremely productive laser facility, despite its modest size and operating cost in the firmament of high-energy, high-power laser facilities worldwide. More than 150 peer-reviewed journal articles (in 39 different journals) have been published using Trident experimental data, many in high-impact journals such as Nature, Nature Physics, Nature Communications, and Physical Review Letters. More than 230 oral presentations involving research at Trident have been presented at national and international conferences. Trident publications have over 5000 citations in the literature with an h-index of 38. AT least 23 Los Alamos postdoctoral researchers have worked on Trident. In themore »« less
https://doi.org/10.2172/1312642

Full Text Available

Similar Records

Title: ACM TOMS replicated computational results initiative

Abstract

Citation Formats

Testing Scientific Programs journal, July 2012

Virtual Appliances, Cloud Computing, and Reproducible Research journal, July 2012

Replicated Computational Results (RCR) Report for “BLIS: A Framework for Rapidly Instantiating BLAS Functionality” journal, June 2015

Making Computations and Publications Reproducible with VisTrails journal, July 2012

Three Dream Applications of Verifiable Computational Results journal, July 2012

Phase II failures: 2008–2010 journal, April 2011

Automated Capture of Experiment Context for Easier Reproducibility in Computational Research journal, July 2012

Code Sharing Is Associated with Research Impact in Image Processing journal, July 2012

Replicated Computational Results Certified BLIS: A Framework for Rapidly Instantiating BLAS Functionality journal, June 2015

“Reproducible” Research in Mathematical Sciences Requires Changes in our Peer Review Culture and Modernization of our Current Publication Approach journal, September 2018

50 Years of Data Science journal, October 2017

Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge journal, January 2019

50 Years of Data Science text, January 2018

Testing Scientific Programs
journal, July 2012

Virtual Appliances, Cloud Computing, and Reproducible Research
journal, July 2012

Replicated Computational Results (RCR) Report for “BLIS: A Framework for Rapidly Instantiating BLAS Functionality”
journal, June 2015

Making Computations and Publications Reproducible with VisTrails
journal, July 2012

Three Dream Applications of Verifiable Computational Results
journal, July 2012

Phase II failures: 2008–2010
journal, April 2011

Automated Capture of Experiment Context for Easier Reproducibility in Computational Research
journal, July 2012

Code Sharing Is Associated with Research Impact in Image Processing
journal, July 2012

Replicated Computational Results Certified BLIS: A Framework for Rapidly Instantiating BLAS Functionality
journal, June 2015

“Reproducible” Research in Mathematical Sciences Requires Changes in our Peer Review Culture and Modernization of our Current Publication Approach
journal, September 2018

50 Years of Data Science
journal, October 2017

Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge
journal, January 2019

50 Years of Data Science
text, January 2018