Does repeat synthesis in materials chemistry obey a power law?
Abstract
Finding examples where experimental measurements have been repeated is a powerful strategy for assessing reproducibility of scientific data. Here, we collect quantitative data to assess how often synthesis of a newly reported material is repeated in the scientific literature. We present a simple power-law model for the frequency of repeat syntheses and assess the validity of this model using a specific class of materials, metal-organic frameworks (MOFs). Our data suggest that a power law describes the frequency of repeat synthesis of many MOFs but that a small number of “supermaterials” exist that have been replicated many times more than a power law would predict. Our results also hint that there are many repeat syntheses that have been performed but not reported in the literature, which suggests simple steps that could be taken to greatly increase the number of reports of replicate experiments in materials chemistry.
- Authors:
-
[1];
[1]
- Georgia Institute of Technology, Atlanta, GA (United States). School of Chemical & Biomolecular Engineering
- Publication Date:
- Research Org.:
- Energy Frontier Research Centers (EFRC) (United States). Center for Understanding and Control of Acid Gas-induced Evolution of Materials for Energy (UNCAGE-ME); Georgia Institute of Technology, Atlanta, GA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC)
- OSTI Identifier:
- 1625062
- Grant/Contract Number:
- SC0012577
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Proceedings of the National Academy of Sciences of the United States of America
- Additional Journal Information:
- Journal Volume: 117; Journal Issue: 2; Journal ID: ISSN 0027-8424
- Publisher:
- National Academy of Sciences
- Country of Publication:
- United States
- Language:
- English
- Subject:
- Science & Technology - Other Topics
Citation Formats
Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, and Sholl, David S. Does repeat synthesis in materials chemistry obey a power law?. United States: N. p., 2019.
Web. doi:10.1073/pnas.1918484117.
Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, & Sholl, David S. Does repeat synthesis in materials chemistry obey a power law?. United States. https://doi.org/10.1073/pnas.1918484117
Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, and Sholl, David S. Thu .
"Does repeat synthesis in materials chemistry obey a power law?". United States. https://doi.org/10.1073/pnas.1918484117. https://www.osti.gov/servlets/purl/1625062.
@article{osti_1625062,
title = {Does repeat synthesis in materials chemistry obey a power law?},
author = {Agrawal, Mayank and Han, Rebecca and Herath, Dinushka and Sholl, David S.},
abstractNote = {Finding examples where experimental measurements have been repeated is a powerful strategy for assessing reproducibility of scientific data. Here, we collect quantitative data to assess how often synthesis of a newly reported material is repeated in the scientific literature. We present a simple power-law model for the frequency of repeat syntheses and assess the validity of this model using a specific class of materials, metal-organic frameworks (MOFs). Our data suggest that a power law describes the frequency of repeat synthesis of many MOFs but that a small number of “supermaterials” exist that have been replicated many times more than a power law would predict. Our results also hint that there are many repeat syntheses that have been performed but not reported in the literature, which suggests simple steps that could be taken to greatly increase the number of reports of replicate experiments in materials chemistry.},
doi = {10.1073/pnas.1918484117},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 2,
volume = 117,
place = {United States},
year = {Thu Dec 26 00:00:00 EST 2019},
month = {Thu Dec 26 00:00:00 EST 2019}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journalFigures / Tables:
Fig. 1: Fraction of MOFs whose synthesis has been reported exactly n times among the group of 130 MOFs described in the text. Blue (red) symbols show results for all reports (results when only resynthesis by authors distinct from the original paper are counted). The solid curve shows the powermore »
All figures and tables
(3 total)
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Believe it or not: how much can we rely on published data on potential drug targets?
journal, August 2011
- Prinz, Florian; Schlange, Thomas; Asadullah, Khusru
- Nature Reviews Drug Discovery, Vol. 10, Issue 9
Why Most Published Research Findings Are False
journal, August 2005
- Ioannidis, John P. A.
- PLoS Medicine, Vol. 2, Issue 8
Opinion: Is science really facing a reproducibility crisis, and do we need it to?
journal, March 2018
- Fanelli, Daniele
- Proceedings of the National Academy of Sciences, Vol. 115, Issue 11
Opinion: Reproducibility failures are essential to scientific inquiry
journal, May 2018
- Redish, A. David; Kummerfeld, Erich; Morris, Rebecca Lea
- Proceedings of the National Academy of Sciences, Vol. 115, Issue 20
Open is not enough
journal, November 2018
- Chen, Xiaoli; Dallmeier-Tiessen, Sünje; Dasler, Robin
- Nature Physics, Vol. 15, Issue 2
Before reproducibility must come preproducibility
journal, May 2018
- Stark, Philip B.
- Nature, Vol. 557, Issue 7707
Responding to the growing issue of research reproducibility
journal, November 2018
- Baer, Donald R.; Gilmore, Ian S.
- Journal of Vacuum Science & Technology A, Vol. 36, Issue 6
Reproducibility of research: Issues and proposed remedies
journal, March 2018
- Allison, David B.; Shiffrin, Richard M.; Stodden, Victoria
- Proceedings of the National Academy of Sciences, Vol. 115, Issue 11
How Reproducible Are Isotherm Measurements in Metal–Organic Frameworks?
journal, November 2017
- Park, Jongwoo; Howe, Joshua D.; Sholl, David S.
- Chemistry of Materials, Vol. 29, Issue 24
Power-Law Distributions in Empirical Data
journal, November 2009
- Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J.
- SIAM Review, Vol. 51, Issue 4
The Cambridge Structural Database
journal, April 2016
- Groom, Colin R.; Bruno, Ian J.; Lightfoot, Matthew P.
- Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials, Vol. 72, Issue 2, p. 171-179
Computation-Ready, Experimental Metal–Organic Frameworks: A Tool To Enable High-Throughput Screening of Nanoporous Crystals
journal, October 2014
- Chung, Yongchul G.; Camp, Jeffrey; Haranczyk, Maciej
- Chemistry of Materials, Vol. 26, Issue 21
Metal-adeninate vertices for the construction of an exceptionally porous metal-organic framework
journal, January 2012
- An, Jihyun; Farha, Omar K.; Hupp, Joseph T.
- Nature Communications, Vol. 3, Issue 1
Photoreactivity examined through incorporation in metal−organic frameworks
journal, May 2010
- Blake, Alexander J.; Champness, Neil R.; Easun, Timothy L.
- Nature Chemistry, Vol. 2, Issue 8
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties
journal, February 2010
- Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A.
- Angewandte Chemie International Edition, Vol. 49, Issue 6
Guest-Induced Chirality in the Ferrimagnetic Nanoporous Diamond Framework Mn 3 (HCOO) 6
journal, January 2007
- Zhang, B.; Wang, Z. -M.; Kurmoo, M.
- Advanced Functional Materials, Vol. 17, Issue 4
A New Zirconium Inorganic Building Brick Forming Metal Organic Frameworks with Exceptional Stability
journal, October 2008
- Cavka, Jasmina Hafizovic; Jakobsen, Søren; Olsbye, Unni
- Journal of the American Chemical Society, Vol. 130, Issue 42, p. 13850-13851
Exceptional chemical and thermal stability of zeolitic imidazolate frameworks
journal, June 2006
- Park, K. S.; Ni, Z.; Cote, A. P.
- Proceedings of the National Academy of Sciences, Vol. 103, Issue 27, p. 10186-10191
A Chemically Functionalizable Nanoporous Material [Cu3(TMA)2(H2O)3]n
journal, February 1999
- Chui, Stephen S.-Y.; Lo, Samuel M.-F.; Charmant, Jonathan P. H.
- Science, Vol. 283, Issue 5405, p. 1148-1150
A Chromium Terephthalate-Based Solid with Unusually Large Pore Volumes and Surface Area
journal, September 2005
- Ferey, G.; Mellot-Draznieks, C.; Serre, C.
- Science, Vol. 309, Issue 5743, p. 2040-2042
Design and synthesis of an exceptionally stable and highly porous metal-organic framework
journal, November 1999
- Li, Hailian; Eddaoudi, Mohamed; M., O'Keeffe
- Nature, Vol. 402, Issue 6759, p. 276-279
A route to high surface area, porosity and inclusion of large molecules in crystals
journal, February 2004
- Chae, Hee K.; Siberio-Pérez, Diana Y.; Kim, Jaheon
- Nature, Vol. 427, Issue 6974, p. 523-527
Exploration of the chemical space and its three historical regimes
journal, June 2019
- Llanos, Eugenio J.; Leal, Wilmer; Luu, Duc H.
- Proceedings of the National Academy of Sciences, Vol. 116, Issue 26
Does Chemical Engineering Research Have a Reproducibility Problem?
journal, June 2019
- Han, Rebecca; Walton, Krista S.; Sholl, David S.
- Annual Review of Chemical and Biomolecular Engineering, Vol. 10, Issue 1
Applicability of the BET Method for Determining Surface Areas of Microporous Metal−Organic Frameworks
journal, July 2007
- Walton, Krista S.; Snurr, Randall Q.
- Journal of the American Chemical Society, Vol. 129, Issue 27
Algorithms and tools for high-throughput geometry-based analysis of crystalline porous materials
journal, February 2012
- Willems, Thomas F.; Rycroft, Chris H.; Kazi, Michaeel
- Microporous and Mesoporous Materials, Vol. 149, Issue 1
Water Stability and Adsorption in Metal–Organic Frameworks
journal, September 2014
- Burtch, Nicholas C.; Jasuja, Himanshu; Walton, Krista S.
- Chemical Reviews, Vol. 114, Issue 20, p. 10575-10612
Activation of metal–organic framework materials
journal, January 2013
- Mondloch, Joseph E.; Karagiaridi, Olga; Farha, Omar K.
- CrystEngComm, Vol. 15, Issue 45
The Metal–Organic Framework Collapse Continuum: Insights from Two-Dimensional Powder X-ray Diffraction
journal, September 2018
- Dodson, Ryan A.; Wong-Foy, Antek G.; Matzger, Adam J.
- Chemistry of Materials, Vol. 30, Issue 18
Structural Complexity in Metal–Organic Frameworks: Simultaneous Modification of Open Metal Sites and Hierarchical Porosity by Systematic Doping with Defective Linkers
journal, June 2014
- Fang, Zhenlan; Dürholt, Johannes P.; Kauer, Max
- Journal of the American Chemical Society, Vol. 136, Issue 27
Unusual and Highly Tunable Missing-Linker Defects in Zirconium Metal–Organic Framework UiO-66 and Their Important Effects on Gas Adsorption
journal, May 2013
- Wu, Hui; Chua, Yong Shen; Krungleviciute, Vaiva
- Journal of the American Chemical Society, Vol. 135, Issue 28
Defects in Metal–Organic Frameworks: Challenge or Opportunity?
journal, August 2015
- Sholl, David S.; Lively, Ryan P.
- The Journal of Physical Chemistry Letters, Vol. 6, Issue 17
Heat-Treatment of Defective UiO-66 from Modulated Synthesis: Adsorption and Stability Studies
journal, October 2017
- Jiao, Yang; Liu, Yang; Zhu, Guanghui
- The Journal of Physical Chemistry C, Vol. 121, Issue 42
Best Practices for the Synthesis, Activation, and Characterization of Metal–Organic Frameworks
journal, September 2016
- Howarth, Ashlee J.; Peters, Aaron W.; Vermeulen, Nicolaas A.
- Chemistry of Materials, Vol. 29, Issue 1
Supercritical Processing as a Route to High Internal Surface Areas and Permanent Microporosity in Metal−Organic Framework Materials
journal, January 2009
- Nelson, Andrew P.; Farha, Omar K.; Mulfort, Karen L.
- Journal of the American Chemical Society, Vol. 131, Issue 2, p. 458-460
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties
journal, January 2010
- Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A.
- Angewandte Chemie, Vol. 122, Issue 6
Why most published research findings are false
journal, September 2008
- Brooks, James
- Urologic Oncology: Seminars and Original Investigations, Vol. 26, Issue 5
Power-law distributions in empirical data
text, January 2018
- Clauset, Aaron; Shalizi, Cosma; Newman, M. E. J.
- Figshare
Why Most Published Research Findings Are False
journal, September 2005
- Ioannidis, John P. A.
- CHANCE, Vol. 18, Issue 4
Why Most Published Research Findings Are False
journal, January 2019
- Ioannidis, John P. A.
- CHANCE, Vol. 32, Issue 1
Power-law distributions in empirical data
text, January 2018
- Clauset, Aaron; Shalizi, Cosma; Newman, M. E. J.
- Figshare
Power-law distributions in empirical data
text, January 2007
- Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J.
- arXiv
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties
journal, February 2010
- Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A.
- Angewandte Chemie International Edition, Vol. 49, Issue 6
Defects in Metal–Organic Frameworks: Challenge or Opportunity?
journal, August 2015
- Sholl, David S.; Lively, Ryan P.
- The Journal of Physical Chemistry Letters, Vol. 6, Issue 17
Water Stability and Adsorption in Metal–Organic Frameworks
journal, September 2014
- Burtch, Nicholas C.; Jasuja, Himanshu; Walton, Krista S.
- Chemical Reviews, Vol. 114, Issue 20, p. 10575-10612
Applicability of the BET Method for Determining Surface Areas of Microporous Metal−Organic Frameworks
journal, July 2007
- Walton, Krista S.; Snurr, Randall Q.
- Journal of the American Chemical Society, Vol. 129, Issue 27
Unusual and Highly Tunable Missing-Linker Defects in Zirconium Metal–Organic Framework UiO-66 and Their Important Effects on Gas Adsorption
journal, May 2013
- Wu, Hui; Chua, Yong Shen; Krungleviciute, Vaiva
- Journal of the American Chemical Society, Vol. 135, Issue 28
Structural Complexity in Metal–Organic Frameworks: Simultaneous Modification of Open Metal Sites and Hierarchical Porosity by Systematic Doping with Defective Linkers
journal, June 2014
- Fang, Zhenlan; Dürholt, Johannes P.; Kauer, Max
- Journal of the American Chemical Society, Vol. 136, Issue 27
A New Zirconium Inorganic Building Brick Forming Metal Organic Frameworks with Exceptional Stability
journal, October 2008
- Cavka, Jasmina Hafizovic; Jakobsen, Søren; Olsbye, Unni
- Journal of the American Chemical Society, Vol. 130, Issue 42, p. 13850-13851
Before reproducibility must come preproducibility
journal, May 2018
- Stark, Philip B.
- Nature, Vol. 557, Issue 7707
A route to high surface area, porosity and inclusion of large molecules in crystals
journal, February 2004
- Chae, Hee K.; Siberio-Pérez, Diana Y.; Kim, Jaheon
- Nature, Vol. 427, Issue 6974, p. 523-527
Photoreactivity examined through incorporation in metal−organic frameworks
journal, May 2010
- Blake, Alexander J.; Champness, Neil R.; Easun, Timothy L.
- Nature Chemistry, Vol. 2, Issue 8
Believe it or not: how much can we rely on published data on potential drug targets?
journal, August 2011
- Prinz, Florian; Schlange, Thomas; Asadullah, Khusru
- Nature Reviews Drug Discovery, Vol. 10, Issue 9
Reproducibility of research: Issues and proposed remedies
journal, March 2018
- Allison, David B.; Shiffrin, Richard M.; Stodden, Victoria
- Proceedings of the National Academy of Sciences, Vol. 115, Issue 11
Does Chemical Engineering Research Have a Reproducibility Problem?
journal, June 2019
- Han, Rebecca; Walton, Krista S.; Sholl, David S.
- Annual Review of Chemical and Biomolecular Engineering, Vol. 10, Issue 1
Why Most Published Research Findings Are False
journal, August 2005
- Ioannidis, John P. A.
- PLoS Medicine, Vol. 2, Issue 8
Investigating Variation in Replicability: A “Many Labs” Replication Project
text, January 2022
- Klein, Richard; Ratliff, Kate; Vianello, Michelangelo
- Open Science Framework
Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.