DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The Resource Identification Initiative: A cultural shift in publishing

Abstract

A central tenet in support of research reproducibility is the ability to uniquely identify research resources, i.e., reagents, tools, and materials that are used to perform experiments. However, current reporting practices for research resources are insufficient to allow humans and algorithms to identify the exact resources that are reported or answer basic questions such as “What other studies used resource X?” To address this issue, the Resource Identification Initiative was launched as a pilot project to improve the reporting standards for research resources in the methods sections of papers and thereby improve identifiability and reproducibility. The pilot engaged over 25 biomedical journal editors from most major publishers, as well as scientists and funding officials. Authors were asked to include Research Resource Identifiers (RRIDs) in their manuscripts prior to publication for three resource types: antibodies, model organisms, and tools (including software and databases). RRIDs represent accession numbers assigned by an authoritative database, e.g., the model organism databases, for each type of resource. To make it easier for authors to obtain RRIDs, resources were aggregated from the appropriate databases and their RRIDs made available in a central web portal (www.scicrunch.org/resources). RRIDs meet three key criteria: they are machine readable, free to generatemore » and access, and are consistent across publishers and journals. The pilot was launched in February of 2014 and over 300 papers have appeared that report RRIDs. The number of journals participating has expanded from the original 25 to more than 40. Here, we present an overview of the pilot project and its outcomes to date. We show that authors are generally accurate in performing the task of identifying resources and supportive of the goals of the project. We also show that identifiability of the resources pre- and post-pilot showed a dramatic improvement for all three resource types, suggesting that the project has had a significant impact on reproducibility relating to research resources.« less

Authors:
 [1];  [2];  [1];  [2];  [3];  [4];  [5];  [1];  [6];  [7];  [8];  [9];  [2]
  1. Univ. of California, Lo Jolla, CA (United States). Center for Research in Biological Systems
  2. Oregon Health and Science Univ., Portland, OR (United States). Dept. of Medical Informatics and Clinical Epidemiology
  3. Department of Psychiatry, University of Massachusetts Medical School, Worcester, MA (United States)
  4. Karolinska Institutet, Stockholm (Sweden)
  5. Leon and Norma Hess Center for Science and Medicine, Icahn School of Medicine at Mount Sinai, New York, NY (United States)
  6. Scientific Outreach, Faculty of 1000 Ltd, London (United Kingdom)
  7. John Wiley and Sons, Hoboken, NJ (United States)
  8. Lawrence Berkeley Lab., CA (United States)
  9. Elsevier, Amsterdam (Netherlands)
Publication Date:
Research Org.:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE
Contributing Org.:
Resource Identification Initiative Members are listed here: https://www.force11.org/node/4463/members
OSTI Identifier:
1209491
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
F1000Research
Additional Journal Information:
Journal Volume: 4; Journal Issue: 134; Journal ID: ISSN 2046-1402
Publisher:
F1000Research
Country of Publication:
United States
Language:
English
Subject:
96 KNOWLEDGE MANAGEMENT AND PRESERVATION; 59 BASIC BIOLOGICAL SCIENCES

Citation Formats

Bandrowski, Anita, Brush, Matthew, Grethe, Jeffery S., Haendel, Melissa A., Kennedy, David N., Hill, Sean, Hof, Patrick R., Martone, Maryann E., Pols, Maaike, Tan, Serena, Washington, Nicole, Zudilova-Seinstra, Elena, and Vasilevsky, Nicole. The Resource Identification Initiative: A cultural shift in publishing. United States: N. p., 2015. Web. doi:10.12688/f1000research.6555.1.
Bandrowski, Anita, Brush, Matthew, Grethe, Jeffery S., Haendel, Melissa A., Kennedy, David N., Hill, Sean, Hof, Patrick R., Martone, Maryann E., Pols, Maaike, Tan, Serena, Washington, Nicole, Zudilova-Seinstra, Elena, & Vasilevsky, Nicole. The Resource Identification Initiative: A cultural shift in publishing. United States. https://doi.org/10.12688/f1000research.6555.1
Bandrowski, Anita, Brush, Matthew, Grethe, Jeffery S., Haendel, Melissa A., Kennedy, David N., Hill, Sean, Hof, Patrick R., Martone, Maryann E., Pols, Maaike, Tan, Serena, Washington, Nicole, Zudilova-Seinstra, Elena, and Vasilevsky, Nicole. Fri . "The Resource Identification Initiative: A cultural shift in publishing". United States. https://doi.org/10.12688/f1000research.6555.1. https://www.osti.gov/servlets/purl/1209491.
@article{osti_1209491,
title = {The Resource Identification Initiative: A cultural shift in publishing},
author = {Bandrowski, Anita and Brush, Matthew and Grethe, Jeffery S. and Haendel, Melissa A. and Kennedy, David N. and Hill, Sean and Hof, Patrick R. and Martone, Maryann E. and Pols, Maaike and Tan, Serena and Washington, Nicole and Zudilova-Seinstra, Elena and Vasilevsky, Nicole},
abstractNote = {A central tenet in support of research reproducibility is the ability to uniquely identify research resources, i.e., reagents, tools, and materials that are used to perform experiments. However, current reporting practices for research resources are insufficient to allow humans and algorithms to identify the exact resources that are reported or answer basic questions such as “What other studies used resource X?” To address this issue, the Resource Identification Initiative was launched as a pilot project to improve the reporting standards for research resources in the methods sections of papers and thereby improve identifiability and reproducibility. The pilot engaged over 25 biomedical journal editors from most major publishers, as well as scientists and funding officials. Authors were asked to include Research Resource Identifiers (RRIDs) in their manuscripts prior to publication for three resource types: antibodies, model organisms, and tools (including software and databases). RRIDs represent accession numbers assigned by an authoritative database, e.g., the model organism databases, for each type of resource. To make it easier for authors to obtain RRIDs, resources were aggregated from the appropriate databases and their RRIDs made available in a central web portal (www.scicrunch.org/resources). RRIDs meet three key criteria: they are machine readable, free to generate and access, and are consistent across publishers and journals. The pilot was launched in February of 2014 and over 300 papers have appeared that report RRIDs. The number of journals participating has expanded from the original 25 to more than 40. Here, we present an overview of the pilot project and its outcomes to date. We show that authors are generally accurate in performing the task of identifying resources and supportive of the goals of the project. We also show that identifiability of the resources pre- and post-pilot showed a dramatic improvement for all three resource types, suggesting that the project has had a significant impact on reproducibility relating to research resources.},
doi = {10.12688/f1000research.6555.1},
journal = {F1000Research},
number = 134,
volume = 4,
place = {United States},
year = {Fri May 29 00:00:00 EDT 2015},
month = {Fri May 29 00:00:00 EDT 2015}
}

Works referenced in this record:

Cautionary notes on the use of NF-κB p65 and p50 antibodies for CNS studies
journal, January 2011

  • Herkenham, Miles; Rathore, Priyanka; Brown, Pierre
  • Journal of Neuroinflammation, Vol. 8, Issue 1
  • DOI: 10.1186/1742-2094-8-141

On the Reproducibility of Science: Unique Identification of Research Resources in the Biomedical Literature
image, January 2014


A Comparative Antibody Analysis of Pannexin1 Expression in Four Rat Brain Regions Reveals Varying Subcellular Localizations
journal, January 2013


Cell line cross-contamination: WSU-CLL is a known derivative of REH and is unsuitable as a model for chronic lymphocytic leukaemia
journal, August 2014


Power failure: why small sample size undermines the reliability of neuroscience
journal, April 2013

  • Button, Katherine S.; Ioannidis, John P. A.; Mokrysz, Claire
  • Nature Reviews Neuroscience, Vol. 14, Issue 5
  • DOI: 10.1038/nrn3475

Minimal NF-κB activity in neurons
journal, October 2013


Promoting research resource identification at JCN
journal, April 2014

  • Bandrowski, Anita; Tan, Serena; Hof, Patrick R.
  • Journal of Comparative Neurology, Vol. 522, Issue 8
  • DOI: 10.1002/cne.23585

The Neuroscience Information Framework: A Data and Knowledge Environment for Neuroscience
journal, September 2008


Statistical inferences under the Null hypothesis: common mistakes and pitfalls in neuroimaging studies
journal, February 2015


An Investigation of the Specificity of Research Antibodies against NF-κB-subunit p65
journal, November 2013

  • Slotta, Carsten; Müller, Janine; Tran, Loan
  • Journal of Histochemistry & Cytochemistry, Vol. 62, Issue 2
  • DOI: 10.1369/0022155413515484

Enhanced consumption of salient solutions following pedunculopontine tegmental lesions
journal, January 2015


On the reproducibility of science: unique identification of research resources in the biomedical literature
journal, January 2013

  • Vasilevsky, Nicole A.; Brush, Matthew H.; Paddock, Holly
  • PeerJ, Vol. 1
  • DOI: 10.7717/peerj.148

Promoting research resource identification at JCN
journal, April 2014

  • Bandrowski, Anita; Tan, Serena; Hof, Patrick R.
  • Journal of Comparative Neurology, Vol. 522, Issue 8
  • DOI: 10.1002/cne.23585

The Neuroscience Information Framework: A Data and Knowledge Environment for Neuroscience
journal, September 2008


Minimal NF-κB activity in neurons
journal, October 2013


Enhanced consumption of salient solutions following pedunculopontine tegmental lesions
journal, January 2015


Power failure: why small sample size undermines the reliability of neuroscience
journal, April 2013

  • Button, Katherine S.; Ioannidis, John P. A.; Mokrysz, Claire
  • Nature Reviews Neuroscience, Vol. 14, Issue 5
  • DOI: 10.1038/nrn3475

Cautionary notes on the use of NF-κB p65 and p50 antibodies for CNS studies
journal, January 2011

  • Herkenham, Miles; Rathore, Priyanka; Brown, Pierre
  • Journal of Neuroinflammation, Vol. 8, Issue 1
  • DOI: 10.1186/1742-2094-8-141

An Investigation of the Specificity of Research Antibodies against NF-κB-subunit p65
journal, November 2013

  • Slotta, Carsten; Müller, Janine; Tran, Loan
  • Journal of Histochemistry & Cytochemistry, Vol. 62, Issue 2
  • DOI: 10.1369/0022155413515484

A Comparative Antibody Analysis of Pannexin1 Expression in Four Rat Brain Regions Reveals Varying Subcellular Localizations
journal, January 2013


Treatment needs and diagnosis awareness in primary care patients with chronic kidney disease
journal, April 2012

  • McIntyre, Natasha J.; Fluck, Richard; McIntyre, Chris
  • British Journal of General Practice, Vol. 62, Issue 597
  • DOI: 10.3399/bjgp12x636047

On the reproducibility of science: unique identification of research resources in the biomedical literature
journal, January 2013

  • Vasilevsky, Nicole A.; Brush, Matthew H.; Paddock, Holly
  • PeerJ, Vol. 1
  • DOI: 10.7717/peerj.148

Works referencing / citing this record:

DAISY: A Data Information System for accountability under the General Data Protection Regulation
journal, December 2019


Engineered Ovarian Cancer Cell Lines for Validation of CAR T Cell Function
journal, November 2019

  • Repellin, Claire E.; Ganesan, Priya; Alcudia, Javier F.
  • Advanced Biosystems, Vol. 4, Issue 1
  • DOI: 10.1002/adbi.201900224

A proposal for validation of antibodies
journal, September 2016

  • Uhlen, Mathias; Bandrowski, Anita; Carr, Steven
  • Nature Methods, Vol. 13, Issue 10
  • DOI: 10.1038/nmeth.3995

A Strategy Based on Technological Maps for the Identification of the State-of-the-Art Techniques in Software Development Projects: Virtual Judge Projects as a Case Study
book, January 2018

  • Hidalgo Suarez, Carlos G.; Bucheli, Vıctor A.; Restrepo-Calle, Felipe
  • Advances in Computing. Communications in Computer and Information Science
  • DOI: 10.1007/978-3-319-98998-3_27

Key components of data publishing: using current best practices to develop a reference model for data publishing
journal, June 2016

  • Austin, Claire C.; Bloom, Theodora; Dallmeier-Tiessen, Sünje
  • International Journal on Digital Libraries, Vol. 18, Issue 2
  • DOI: 10.1007/s00799-016-0178-2

Rigor and Transparency Index, a new metric of quality for assessing biological and medical science methods
posted_content, March 2020


Incidences of problematic cell lines are lower in papers that use RRIDs to identify cell lines
journal, January 2019

  • Babic, Zeljana; Capes-Davis, Amanda; Martone, Maryann E.
  • eLife, Vol. 8
  • DOI: 10.7554/elife.41676

Improving Transparency and Scientific Rigor in Academic Publishing
posted_content, February 2018


Improving transparency and scientific rigor in academic publishing
journal, December 2018

  • Prager, Eric M.; Chambers, Karen E.; Plotkin, Joshua L.
  • Brain and Behavior, Vol. 9, Issue 1
  • DOI: 10.1002/brb3.1141

Biomedical text mining for research rigor and integrity: tasks, challenges, directions
journal, June 2017


Tyrosinase and nestin immunohistochemical expression in melanocytic nevi as a histopathologic pattern to trace melanocyte differentiation and nevogenesis
journal, September 2018

  • Murtas, Daniela; Pilloni, Luca; Diana, Andrea
  • Histochemistry and Cell Biology, Vol. 151, Issue 2
  • DOI: 10.1007/s00418-018-1730-5

Theory and practice of data citation
journal, September 2017

  • Silvello, Gianmaria
  • Journal of the Association for Information Science and Technology, Vol. 69, Issue 1
  • DOI: 10.1002/asi.23917

Foundry: a message-oriented, horizontally scalable ETL system for scientific data integration and enhancement
journal, January 2018


Key Components Of Data Publishing: Using Current Best Practices To Develop A Reference Model For Data Publishing
text, January 2015


How to avoid pitfalls in antibody use
journal, January 2015


Reproducibility2020: Progress and priorities
journal, January 2017


Role of epithelial–mesenchymal transition involved molecules in the progression of cutaneous melanoma
journal, August 2017

  • Murtas, Daniela; Maxia, Cristina; Diana, Andrea
  • Histochemistry and Cell Biology, Vol. 148, Issue 6
  • DOI: 10.1007/s00418-017-1606-0

Continuous cuprizone intoxication allows active experimental autoimmune encephalomyelitis induction in C57BL/6 mice
journal, April 2019

  • Yakimov, Vladislav; Schweiger, Felix; Zhan, Jiangshan
  • Histochemistry and Cell Biology, Vol. 152, Issue 2
  • DOI: 10.1007/s00418-019-01786-4

Revisiting the scientific method to improve rigor and reproducibility of immunohistochemistry in reproductive science†
journal, April 2018

  • Manuel, Sharrón L.; Johnson, Brian W.; Frevert, Charles W.
  • Biology of Reproduction, Vol. 99, Issue 4
  • DOI: 10.1093/biolre/ioy094

Enhanced validation of antibodies for research applications
journal, October 2018


Reproducibility2020: Progress and Priorities
posted_content, February 2017

  • Freedman, Leonard P.; Venugopalan, Gautham; Wisman, Rosann
  • bioRxiv
  • DOI: 10.1101/109017

TechMiner: Extracting Technologies from Academic Publications
book, January 2016

  • Osborne, Francesco; de Ribaupierre, Hélène; Motta, Enrico
  • Knowledge Engineering and Knowledge Management
  • DOI: 10.1007/978-3-319-49004-5_30

Robust Cross-Platform Workflows: How Technical and Scientific Communities Collaborate to Develop, Test and Share Best Practices for Data Analysis
journal, September 2017

  • Möller, Steffen; Prescott, Stuart W.; Wirzenius, Lars
  • Data Science and Engineering, Vol. 2, Issue 3
  • DOI: 10.1007/s41019-017-0050-4

Spatial proteomics: a powerful discovery tool for cell biology
journal, January 2019


Reporting animal research: Explanation and Elaboration for the ARRIVE guidelines 2019
posted_content, July 2019

  • Percie du Sert, Nathalie; Ahluwalia, Amrita; Alam, Sabina
  • bioRxiv
  • DOI: 10.1101/703355

Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool
journal, March 2019


Improving transparency and scientific rigor in academic publishing
journal, December 2018

  • Prager, Eric M.; Chambers, Karen E.; Plotkin, Joshua L.
  • Cancer Reports, Vol. 2, Issue 1
  • DOI: 10.1002/cnr2.1150

Robust Cross-Platform Workflows: How Technical and Scientific Communities Collaborate to Develop, Test and Share Best Practices for Data Analysis
journal, September 2017

  • Möller, Steffen; Prescott, Stuart W.; Wirzenius, Lars
  • Data Science and Engineering, Vol. 2, Issue 3
  • DOI: 10.1007/s41019-017-0050-4

The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods
journal, November 2020


Enhanced validation of antibodies for research applications
journal, October 2018


Integrative biological simulation praxis: Considerations from physics, philosophy, and data/model curation practices
journal, October 2017


A snapshot of 3649 Web-based services published between 1994 and 2017 shows a decrease in availability after 2 years
journal, December 2017

  • Ősz, Ágnes; Pongor, Lőrinc Sándor; Szirmai, Danuta
  • Briefings in Bioinformatics, Vol. 20, Issue 3
  • DOI: 10.1093/bib/bbx159

On expert curation and scalability: UniProtKB/Swiss-Prot as a case study
journal, July 2017


DAISY: A Data Information System for accountability under the General Data Protection Regulation
journal, December 2019


How to avoid pitfalls in antibody use
journal, January 2015


Antibody Watch: Text mining antibody specificity from the literature
journal, May 2021


Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool
journal, March 2019


Antibody validation for protein expression on tissue slides: a protocol for immunohistochemistry
journal, December 2020

  • MacNeil, Tyler; Vathiotis, Ioannis A.; Martinez-Morilla, Sandra
  • BioTechniques, Vol. 69, Issue 6
  • DOI: 10.2144/btn-2020-0095

On the Necessity of Validating Antibodies in the Immunohistochemistry Literature
journal, April 2019


Data Citation in Neuroimaging: Proposed Best Practices for Data Identification and Attribution
journal, August 2016

  • Honor, Leah B.; Haselgrove, Christian; Frazier, Jean A.
  • Frontiers in Neuroinformatics, Vol. 10
  • DOI: 10.3389/fninf.2016.00034

ATPP: A Pipeline for Automatic Tractography-Based Brain Parcellation
journal, May 2017


Data Publications Correlate with Citation Impact
journal, September 2016


Fast Confirmation of Antibody Identity by MALDI-TOF MS Fingerprints
journal, March 2020

  • Tscheuschner, Georg; Schwaar, Timm; Weller, Michael G.
  • Antibodies, Vol. 9, Issue 2
  • DOI: 10.3390/antib9020008

Incidences of problematic cell lines are lower in papers that use RRIDs to identify cell lines
journal, January 2019

  • Babic, Zeljana; Capes-Davis, Amanda; Martone, Maryann E.
  • eLife, Vol. 8
  • DOI: 10.7554/elife.41676