skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Does repeat synthesis in materials chemistry obey a power law?

Abstract

Finding examples where experimental measurements have been repeated is a powerful strategy for assessing reproducibility of scientific data. Here, we collect quantitative data to assess how often synthesis of a newly reported material is repeated in the scientific literature. We present a simple power-law model for the frequency of repeat syntheses and assess the validity of this model using a specific class of materials, metal-organic frameworks (MOFs). Our data suggest that a power law describes the frequency of repeat synthesis of many MOFs but that a small number of “supermaterials” exist that have been replicated many times more than a power law would predict. Our results also hint that there are many repeat syntheses that have been performed but not reported in the literature, which suggests simple steps that could be taken to greatly increase the number of reports of replicate experiments in materials chemistry.

Authors:
ORCiD logo [1];  [1];  [1]; ORCiD logo [1]
  1. Georgia Institute of Technology, Atlanta, GA (United States). School of Chemical & Biomolecular Engineering
Publication Date:
Research Org.:
Georgia Inst. of Technology, Atlanta, GA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1625062
Grant/Contract Number:  
SC0012577
Resource Type:
Accepted Manuscript
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Volume: 117; Journal Issue: 2; Journal ID: ISSN 0027-8424
Publisher:
National Academy of Sciences
Country of Publication:
United States
Language:
English
Subject:
Science & Technology - Other Topics

Citation Formats

Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, and Sholl, David S. Does repeat synthesis in materials chemistry obey a power law?. United States: N. p., 2019. Web. doi:10.1073/pnas.1918484117.
Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, & Sholl, David S. Does repeat synthesis in materials chemistry obey a power law?. United States. doi:10.1073/pnas.1918484117.
Agrawal, Mayank, Han, Rebecca, Herath, Dinushka, and Sholl, David S. Thu . "Does repeat synthesis in materials chemistry obey a power law?". United States. doi:10.1073/pnas.1918484117. https://www.osti.gov/servlets/purl/1625062.
@article{osti_1625062,
title = {Does repeat synthesis in materials chemistry obey a power law?},
author = {Agrawal, Mayank and Han, Rebecca and Herath, Dinushka and Sholl, David S.},
abstractNote = {Finding examples where experimental measurements have been repeated is a powerful strategy for assessing reproducibility of scientific data. Here, we collect quantitative data to assess how often synthesis of a newly reported material is repeated in the scientific literature. We present a simple power-law model for the frequency of repeat syntheses and assess the validity of this model using a specific class of materials, metal-organic frameworks (MOFs). Our data suggest that a power law describes the frequency of repeat synthesis of many MOFs but that a small number of “supermaterials” exist that have been replicated many times more than a power law would predict. Our results also hint that there are many repeat syntheses that have been performed but not reported in the literature, which suggests simple steps that could be taken to greatly increase the number of reports of replicate experiments in materials chemistry.},
doi = {10.1073/pnas.1918484117},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 2,
volume = 117,
place = {United States},
year = {2019},
month = {12}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Believe it or not: how much can we rely on published data on potential drug targets?
journal, August 2011

  • Prinz, Florian; Schlange, Thomas; Asadullah, Khusru
  • Nature Reviews Drug Discovery, Vol. 10, Issue 9
  • DOI: 10.1038/nrd3439-c1

Why Most Published Research Findings Are False
journal, August 2005


Opinion: Is science really facing a reproducibility crisis, and do we need it to?
journal, March 2018

  • Fanelli, Daniele
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 11
  • DOI: 10.1073/pnas.1708272114

Opinion: Reproducibility failures are essential to scientific inquiry
journal, May 2018

  • Redish, A. David; Kummerfeld, Erich; Morris, Rebecca Lea
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 20
  • DOI: 10.1073/pnas.1806370115

Open is not enough
journal, November 2018


Before reproducibility must come preproducibility
journal, May 2018


Responding to the growing issue of research reproducibility
journal, November 2018

  • Baer, Donald R.; Gilmore, Ian S.
  • Journal of Vacuum Science & Technology A, Vol. 36, Issue 6
  • DOI: 10.1116/1.5049141

Reproducibility of research: Issues and proposed remedies
journal, March 2018

  • Allison, David B.; Shiffrin, Richard M.; Stodden, Victoria
  • Proceedings of the National Academy of Sciences, Vol. 115, Issue 11
  • DOI: 10.1073/pnas.1802324115

How Reproducible Are Isotherm Measurements in Metal–Organic Frameworks?
journal, November 2017


Power-Law Distributions in Empirical Data
journal, November 2009

  • Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J.
  • SIAM Review, Vol. 51, Issue 4
  • DOI: 10.1137/070710111

The Cambridge Structural Database
journal, April 2016

  • Groom, Colin R.; Bruno, Ian J.; Lightfoot, Matthew P.
  • Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials, Vol. 72, Issue 2, p. 171-179
  • DOI: 10.1107/S2052520616003954

Computation-Ready, Experimental Metal–Organic Frameworks: A Tool To Enable High-Throughput Screening of Nanoporous Crystals
journal, October 2014

  • Chung, Yongchul G.; Camp, Jeffrey; Haranczyk, Maciej
  • Chemistry of Materials, Vol. 26, Issue 21
  • DOI: 10.1021/cm502594j

Metal-adeninate vertices for the construction of an exceptionally porous metal-organic framework
journal, January 2012

  • An, Jihyun; Farha, Omar K.; Hupp, Joseph T.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1618

Photoreactivity examined through incorporation in metal−organic frameworks
journal, May 2010

  • Blake, Alexander J.; Champness, Neil R.; Easun, Timothy L.
  • Nature Chemistry, Vol. 2, Issue 8
  • DOI: 10.1038/nchem.681

A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties
journal, February 2010

  • Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A.
  • Angewandte Chemie International Edition, Vol. 49, Issue 6
  • DOI: 10.1002/anie.200906322

Guest-Induced Chirality in the Ferrimagnetic Nanoporous Diamond Framework Mn 3 (HCOO) 6
journal, January 2007

  • Zhang, B.; Wang, Z. -M.; Kurmoo, M.
  • Advanced Functional Materials, Vol. 17, Issue 4
  • DOI: 10.1002/adfm.200600265

A New Zirconium Inorganic Building Brick Forming Metal Organic Frameworks with Exceptional Stability
journal, October 2008

  • Cavka, Jasmina Hafizovic; Jakobsen, Søren; Olsbye, Unni
  • Journal of the American Chemical Society, Vol. 130, Issue 42, p. 13850-13851
  • DOI: 10.1021/ja8057953

Exceptional chemical and thermal stability of zeolitic imidazolate frameworks
journal, June 2006

  • Park, K. S.; Ni, Z.; Cote, A. P.
  • Proceedings of the National Academy of Sciences, Vol. 103, Issue 27, p. 10186-10191
  • DOI: 10.1073/pnas.0602439103

A Chemically Functionalizable Nanoporous Material [Cu3(TMA)2(H2O)3]n
journal, February 1999


A Chromium Terephthalate-Based Solid with Unusually Large Pore Volumes and Surface Area
journal, September 2005

  • Ferey, G.; Mellot-Draznieks, C.; Serre, C.
  • Science, Vol. 309, Issue 5743, p. 2040-2042
  • DOI: 10.1126/science.1116275

Design and synthesis of an exceptionally stable and highly porous metal-organic framework
journal, November 1999

  • Li, Hailian; Eddaoudi, Mohamed; M., O'Keeffe
  • Nature, Vol. 402, Issue 6759, p. 276-279
  • DOI: 10.1038/46248

A route to high surface area, porosity and inclusion of large molecules in crystals
journal, February 2004

  • Chae, Hee K.; Siberio-Pérez, Diana Y.; Kim, Jaheon
  • Nature, Vol. 427, Issue 6974, p. 523-527
  • DOI: 10.1038/nature02311

Exploration of the chemical space and its three historical regimes
journal, June 2019

  • Llanos, Eugenio J.; Leal, Wilmer; Luu, Duc H.
  • Proceedings of the National Academy of Sciences, Vol. 116, Issue 26
  • DOI: 10.1073/pnas.1816039116

Does Chemical Engineering Research Have a Reproducibility Problem?
journal, June 2019


Applicability of the BET Method for Determining Surface Areas of Microporous Metal−Organic Frameworks
journal, July 2007

  • Walton, Krista S.; Snurr, Randall Q.
  • Journal of the American Chemical Society, Vol. 129, Issue 27
  • DOI: 10.1021/ja071174k

Algorithms and tools for high-throughput geometry-based analysis of crystalline porous materials
journal, February 2012


Water Stability and Adsorption in Metal–Organic Frameworks
journal, September 2014

  • Burtch, Nicholas C.; Jasuja, Himanshu; Walton, Krista S.
  • Chemical Reviews, Vol. 114, Issue 20, p. 10575-10612
  • DOI: 10.1021/cr5002589

Activation of metal–organic framework materials
journal, January 2013

  • Mondloch, Joseph E.; Karagiaridi, Olga; Farha, Omar K.
  • CrystEngComm, Vol. 15, Issue 45
  • DOI: 10.1039/c3ce41232f

The Metal–Organic Framework Collapse Continuum: Insights from Two-Dimensional Powder X-ray Diffraction
journal, September 2018


Structural Complexity in Metal–Organic Frameworks: Simultaneous Modification of Open Metal Sites and Hierarchical Porosity by Systematic Doping with Defective Linkers
journal, June 2014

  • Fang, Zhenlan; Dürholt, Johannes P.; Kauer, Max
  • Journal of the American Chemical Society, Vol. 136, Issue 27
  • DOI: 10.1021/ja503218j

Unusual and Highly Tunable Missing-Linker Defects in Zirconium Metal–Organic Framework UiO-66 and Their Important Effects on Gas Adsorption
journal, May 2013

  • Wu, Hui; Chua, Yong Shen; Krungleviciute, Vaiva
  • Journal of the American Chemical Society, Vol. 135, Issue 28
  • DOI: 10.1021/ja404514r

Defects in Metal–Organic Frameworks: Challenge or Opportunity?
journal, August 2015


Heat-Treatment of Defective UiO-66 from Modulated Synthesis: Adsorption and Stability Studies
journal, October 2017

  • Jiao, Yang; Liu, Yang; Zhu, Guanghui
  • The Journal of Physical Chemistry C, Vol. 121, Issue 42
  • DOI: 10.1021/acs.jpcc.7b07772

Best Practices for the Synthesis, Activation, and Characterization of Metal–Organic Frameworks
journal, September 2016


Supercritical Processing as a Route to High Internal Surface Areas and Permanent Microporosity in Metal−Organic Framework Materials
journal, January 2009

  • Nelson, Andrew P.; Farha, Omar K.; Mulfort, Karen L.
  • Journal of the American Chemical Society, Vol. 131, Issue 2, p. 458-460
  • DOI: 10.1021/ja808853q