Does repeat synthesis in materials chemistry obey a power law?

Agrawal, Mayank; Han, Rebecca; Herath, Dinushka; Sholl, David S.

doi:10.1073/pnas.1918484117

Title: Does repeat synthesis in materials chemistry obey a power law?

Journal Article · 25 December 2019 · Proceedings of the National Academy of Sciences of the United States of America

DOI: https://doi.org/10.1073/pnas.1918484117 · OSTI ID:1625062

^[1]; Han, Rebecca ^[2]; Herath, Dinushka ^[2];

^[2]

Georgia Institute of Technology, Atlanta, GA (United States). School of Chemical & Biomolecular Engineering; DOE/OSTI
Georgia Institute of Technology, Atlanta, GA (United States). School of Chemical & Biomolecular Engineering

Finding examples where experimental measurements have been repeated is a powerful strategy for assessing reproducibility of scientific data. Here, we collect quantitative data to assess how often synthesis of a newly reported material is repeated in the scientific literature. We present a simple power-law model for the frequency of repeat syntheses and assess the validity of this model using a specific class of materials, metal-organic frameworks (MOFs). Our data suggest that a power law describes the frequency of repeat synthesis of many MOFs but that a small number of “supermaterials” exist that have been replicated many times more than a power law would predict. Our results also hint that there are many repeat syntheses that have been performed but not reported in the literature, which suggests simple steps that could be taken to greatly increase the number of reports of replicate experiments in materials chemistry.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Energy Frontier Research Centers (EFRC) (United States). Center for Understanding and Control of Acid Gas-induced Evolution of Materials for Energy (UNCAGE-ME); Georgia Inst. of Technology, Atlanta, GA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

Grant/Contract Number:: SC0012577

OSTI ID:: 1625062

Journal Information:: Proceedings of the National Academy of Sciences of the United States of America, Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Issue: 2 Vol. 117; ISSN 0027-8424

Publisher:: National Academy of SciencesCopyright Statement

Country of Publication:: United States

Language:: English

References (58)

Algorithms and tools for high-throughput geometry-based analysis of crystalline porous materials Willems, Thomas F.; Rycroft, Chris H.; Kazi, Michaeel Microporous and Mesoporous Materials, Vol. 149, Issue 1 https://doi.org/10.1016/j.micromeso.2011.08.020	journal	February 2012
Heat-Treatment of Defective UiO-66 from Modulated Synthesis: Adsorption and Stability Studies Jiao, Yang; Liu, Yang; Zhu, Guanghui The Journal of Physical Chemistry C, Vol. 121, Issue 42 https://doi.org/10.1021/acs.jpcc.7b07772	journal	October 2017
Guest-Induced Chirality in the Ferrimagnetic Nanoporous Diamond Framework Mn ₃ (HCOO) ₆ Zhang, B.; Wang, Z. -M.; Kurmoo, M. Advanced Functional Materials, Vol. 17, Issue 4 https://doi.org/10.1002/adfm.200600265	journal	January 2007
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A. Angewandte Chemie International Edition, Vol. 49, Issue 6 https://doi.org/10.1002/anie.200906322	journal	February 2010
The Cambridge Structural Database Groom, Colin R.; Bruno, Ian J.; Lightfoot, Matthew P. Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials, Vol. 72, Issue 2, p. 171-179 https://doi.org/10.1107/s2052520616003954	journal	April 2016
Believe it or not: how much can we rely on published data on potential drug targets? Prinz, Florian; Schlange, Thomas; Asadullah, Khusru Nature Reviews Drug Discovery, Vol. 10, Issue 9 https://doi.org/10.1038/nrd3439-c1	journal	August 2011
Before reproducibility must come preproducibility Stark, Philip B. Nature, Vol. 557, Issue 7707 https://doi.org/10.1038/d41586-018-05256-0	journal	May 2018
Why Most Published Research Findings Are False Ioannidis, John P. A. PLoS Medicine, Vol. 2, Issue 8 https://doi.org/10.1371/journal.pmed.0020124	journal	August 2005
Exploration of the chemical space and its three historical regimes Llanos, Eugenio J.; Leal, Wilmer; Luu, Duc H. Proceedings of the National Academy of Sciences, Vol. 116, Issue 26 https://doi.org/10.1073/pnas.1816039116	journal	June 2019
Why Most Published Research Findings Are False Ioannidis, John P. A. CHANCE, Vol. 32, Issue 1 https://doi.org/10.1080/09332480.2019.1579573	journal	January 2019
Best Practices for the Synthesis, Activation, and Characterization of Metal–Organic Frameworks Howarth, Ashlee J.; Peters, Aaron W.; Vermeulen, Nicolaas A. Chemistry of Materials, Vol. 29, Issue 1 https://doi.org/10.1021/acs.chemmater.6b02626	journal	September 2016
Does Chemical Engineering Research Have a Reproducibility Problem? Han, Rebecca; Walton, Krista S.; Sholl, David S. Annual Review of Chemical and Biomolecular Engineering, Vol. 10, Issue 1 https://doi.org/10.1146/annurev-chembioeng-060718-030323	journal	June 2019
Why most published research findings are false Brooks, James Urologic Oncology: Seminars and Original Investigations, Vol. 26, Issue 5 https://doi.org/10.1016/j.urolonc.2008.07.015	journal	September 2008
Power-law distributions in empirical data Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J. arXiv https://doi.org/10.48550/arxiv.0706.1062	text	January 2007
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A. Angewandte Chemie, Vol. 122, Issue 6 https://doi.org/10.1002/ange.200906322	journal	January 2010
Unusual and Highly Tunable Missing-Linker Defects in Zirconium Metal–Organic Framework UiO-66 and Their Important Effects on Gas Adsorption Wu, Hui; Chua, Yong Shen; Krungleviciute, Vaiva Journal of the American Chemical Society, Vol. 135, Issue 28 https://doi.org/10.1021/ja404514r	journal	May 2013
A Chromium Terephthalate-Based Solid with Unusually Large Pore Volumes and Surface Area Ferey, G.; Mellot-Draznieks, C.; Serre, C. Science, Vol. 309, Issue 5743, p. 2040-2042 https://doi.org/10.1126/science.1116275	journal	September 2005
Power-law distributions in empirical data Clauset, Aaron; Shalizi, Cosma; Newman, M. E. J. Figshare https://doi.org/10.1184/r1/6586835	text	January 2018
A Chemically Functionalizable Nanoporous Material [Cu₃(TMA)₂(H₂O)₃]_n Chui, Stephen S.-Y.; Lo, Samuel M.-F.; Charmant, Jonathan P. H. Science, Vol. 283, Issue 5405, p. 1148-1150 https://doi.org/10.1126/science.283.5405.1148	journal	February 1999
A New Zirconium Inorganic Building Brick Forming Metal Organic Frameworks with Exceptional Stability Cavka, Jasmina Hafizovic; Jakobsen, Søren; Olsbye, Unni Journal of the American Chemical Society, Vol. 130, Issue 42, p. 13850-13851 https://doi.org/10.1021/ja8057953	journal	October 2008
Why Most Published Research Findings Are False Ioannidis, John P. A. CHANCE, Vol. 18, Issue 4 https://doi.org/10.1080/09332480.2005.10722754	journal	September 2005
Responding to the growing issue of research reproducibility Baer, Donald R.; Gilmore, Ian S. Journal of Vacuum Science & Technology A, Vol. 36, Issue 6 https://doi.org/10.1116/1.5049141	journal	November 2018
Unusual and Highly Tunable Missing-Linker Defects in Zirconium Metal–Organic Framework UiO-66 and Their Important Effects on Gas Adsorption Wu, Hui; Chua, Yong Shen; Krungleviciute, Vaiva Journal of the American Chemical Society, Vol. 135, Issue 28 https://doi.org/10.1021/ja404514r	journal	May 2013
Water Stability and Adsorption in Metal–Organic Frameworks Burtch, Nicholas C.; Jasuja, Himanshu; Walton, Krista S. Chemical Reviews, Vol. 114, Issue 20, p. 10575-10612 https://doi.org/10.1021/cr5002589	journal	September 2014
Does Chemical Engineering Research Have a Reproducibility Problem? Han, Rebecca; Walton, Krista S.; Sholl, David S. Annual Review of Chemical and Biomolecular Engineering, Vol. 10, Issue 1 https://doi.org/10.1146/annurev-chembioeng-060718-030323	journal	June 2019
Power-Law Distributions in Empirical Data Clauset, Aaron; Shalizi, Cosma Rohilla; Newman, M. E. J. SIAM Review, Vol. 51, Issue 4 https://doi.org/10.1137/070710111	journal	November 2009
Reproducibility of research: Issues and proposed remedies Allison, David B.; Shiffrin, Richard M.; Stodden, Victoria Proceedings of the National Academy of Sciences, Vol. 115, Issue 11 https://doi.org/10.1073/pnas.1802324115	journal	March 2018
A New Zirconium Inorganic Building Brick Forming Metal Organic Frameworks with Exceptional Stability Cavka, Jasmina Hafizovic; Jakobsen, Søren; Olsbye, Unni Journal of the American Chemical Society, Vol. 130, Issue 42, p. 13850-13851 https://doi.org/10.1021/ja8057953	journal	October 2008
Metal-adeninate vertices for the construction of an exceptionally porous metal-organic framework An, Jihyun; Farha, Omar K.; Hupp, Joseph T. Nature Communications, Vol. 3, Issue 1 https://doi.org/10.1038/ncomms1618	journal	January 2012
Believe it or not: how much can we rely on published data on potential drug targets? Prinz, Florian; Schlange, Thomas; Asadullah, Khusru Nature Reviews Drug Discovery, Vol. 10, Issue 9 https://doi.org/10.1038/nrd3439-c1	journal	August 2011
A route to high surface area, porosity and inclusion of large molecules in crystals Chae, Hee K.; Siberio-Pérez, Diana Y.; Kim, Jaheon Nature, Vol. 427, Issue 6974, p. 523-527 https://doi.org/10.1038/nature02311	journal	February 2004
Investigating Variation in Replicability: A “Many Labs” Replication Project Klein, Richard; Ratliff, Kate; Vianello, Michelangelo Open Science Framework https://doi.org/10.17605/osf.io/wx7ck	text	January 2022
Structural Complexity in Metal–Organic Frameworks: Simultaneous Modification of Open Metal Sites and Hierarchical Porosity by Systematic Doping with Defective Linkers Fang, Zhenlan; Dürholt, Johannes P.; Kauer, Max Journal of the American Chemical Society, Vol. 136, Issue 27 https://doi.org/10.1021/ja503218j	journal	June 2014
Why Most Published Research Findings Are False Ioannidis, John P. A. PLoS Medicine, Vol. 2, Issue 8 https://doi.org/10.1371/journal.pmed.0020124	journal	August 2005
Structural Complexity in Metal–Organic Frameworks: Simultaneous Modification of Open Metal Sites and Hierarchical Porosity by Systematic Doping with Defective Linkers Fang, Zhenlan; Dürholt, Johannes P.; Kauer, Max Journal of the American Chemical Society, Vol. 136, Issue 27 https://doi.org/10.1021/ja503218j	journal	June 2014
Water Stability and Adsorption in Metal–Organic Frameworks Burtch, Nicholas C.; Jasuja, Himanshu; Walton, Krista S. Chemical Reviews, Vol. 114, Issue 20, p. 10575-10612 https://doi.org/10.1021/cr5002589	journal	September 2014
Applicability of the BET Method for Determining Surface Areas of Microporous Metal−Organic Frameworks Walton, Krista S.; Snurr, Randall Q. Journal of the American Chemical Society, Vol. 129, Issue 27 https://doi.org/10.1021/ja071174k	journal	July 2007
The Cambridge Structural Database Groom, Colin R.; Bruno, Ian J.; Lightfoot, Matthew P. Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials, Vol. 72, Issue 2, p. 171-179 https://doi.org/10.1107/S2052520616003954	journal	April 2016
A Simple Lithium(I) Salt with a Microporous Structure and Its Gas Sorption Properties Abrahams, Brendan F.; Grannas, Martin J.; Hudson, Timothy A. Angewandte Chemie International Edition, Vol. 49, Issue 6 https://doi.org/10.1002/anie.200906322	journal	February 2010
Opinion: Is science really facing a reproducibility crisis, and do we need it to? Fanelli, Daniele Proceedings of the National Academy of Sciences, Vol. 115, Issue 11 https://doi.org/10.1073/pnas.1708272114	journal	March 2018
How Reproducible Are Isotherm Measurements in Metal–Organic Frameworks? Park, Jongwoo; Howe, Joshua D.; Sholl, David S. Chemistry of Materials, Vol. 29, Issue 24 https://doi.org/10.1021/acs.chemmater.7b04287	journal	November 2017
Defects in Metal–Organic Frameworks: Challenge or Opportunity? Sholl, David S.; Lively, Ryan P. The Journal of Physical Chemistry Letters, Vol. 6, Issue 17 https://doi.org/10.1021/acs.jpclett.5b01135	journal	August 2015
Activation of metal–organic framework materials Mondloch, Joseph E.; Karagiaridi, Olga; Farha, Omar K. CrystEngComm, Vol. 15, Issue 45 https://doi.org/10.1039/c3ce41232f	journal	January 2013
Reproducibility of research: Issues and proposed remedies Allison, David B.; Shiffrin, Richard M.; Stodden, Victoria Proceedings of the National Academy of Sciences, Vol. 115, Issue 11 https://doi.org/10.1073/pnas.1802324115	journal	March 2018
Applicability of the BET Method for Determining Surface Areas of Microporous Metal−Organic Frameworks Walton, Krista S.; Snurr, Randall Q. Journal of the American Chemical Society, Vol. 129, Issue 27 https://doi.org/10.1021/ja071174k	journal	July 2007
The Metal–Organic Framework Collapse Continuum: Insights from Two-Dimensional Powder X-ray Diffraction Dodson, Ryan A.; Wong-Foy, Antek G.; Matzger, Adam J. Chemistry of Materials, Vol. 30, Issue 18 https://doi.org/10.1021/acs.chemmater.8b03378	journal	September 2018
Before reproducibility must come preproducibility Stark, Philip B. Nature, Vol. 557, Issue 7707 https://doi.org/10.1038/d41586-018-05256-0	journal	May 2018
Computation-Ready, Experimental Metal–Organic Frameworks: A Tool To Enable High-Throughput Screening of Nanoporous Crystals Chung, Yongchul G.; Camp, Jeffrey; Haranczyk, Maciej Chemistry of Materials, Vol. 26, Issue 21 https://doi.org/10.1021/cm502594j	journal	October 2014
Open is not enough Chen, Xiaoli; Dallmeier-Tiessen, Sünje; Dasler, Robin Nature Physics, Vol. 15, Issue 2 https://doi.org/10.1038/s41567-018-0342-2	journal	November 2018
Exceptional chemical and thermal stability of zeolitic imidazolate frameworks Park, K. S.; Ni, Z.; Cote, A. P. Proceedings of the National Academy of Sciences, Vol. 103, Issue 27, p. 10186-10191 https://doi.org/10.1073/pnas.0602439103	journal	June 2006
A route to high surface area, porosity and inclusion of large molecules in crystals Chae, Hee K.; Siberio-Pérez, Diana Y.; Kim, Jaheon Nature, Vol. 427, Issue 6974, p. 523-527 https://doi.org/10.1038/nature02311	journal	February 2004
Opinion: Reproducibility failures are essential to scientific inquiry Redish, A. David; Kummerfeld, Erich; Morris, Rebecca Lea Proceedings of the National Academy of Sciences, Vol. 115, Issue 20 https://doi.org/10.1073/pnas.1806370115	journal	May 2018
Power-law distributions in empirical data Clauset, Aaron; Shalizi, Cosma; Newman, M. E. J. Figshare https://doi.org/10.1184/r1/6586835.v1	text	January 2018
Defects in Metal–Organic Frameworks: Challenge or Opportunity? Sholl, David S.; Lively, Ryan P. The Journal of Physical Chemistry Letters, Vol. 6, Issue 17 https://doi.org/10.1021/acs.jpclett.5b01135	journal	August 2015
Photoreactivity examined through incorporation in metal−organic frameworks Blake, Alexander J.; Champness, Neil R.; Easun, Timothy L. Nature Chemistry, Vol. 2, Issue 8 https://doi.org/10.1038/nchem.681	journal	May 2010
Design and synthesis of an exceptionally stable and highly porous metal-organic framework Li, Hailian; Eddaoudi, Mohamed; M., O'Keeffe Nature, Vol. 402, Issue 6759, p. 276-279 https://doi.org/10.1038/46248	journal	November 1999
Supercritical Processing as a Route to High Internal Surface Areas and Permanent Microporosity in Metal−Organic Framework Materials Nelson, Andrew P.; Farha, Omar K.; Mulfort, Karen L. Journal of the American Chemical Society, Vol. 131, Issue 2, p. 458-460 https://doi.org/10.1021/ja808853q	journal	January 2009
Photoreactivity examined through incorporation in metal−organic frameworks Blake, Alexander J.; Champness, Neil R.; Easun, Timothy L. Nature Chemistry, Vol. 2, Issue 8 https://doi.org/10.1038/nchem.681	journal	May 2010

Similar Records

Capture of volatile fission products: present status and future perspectives - 15680

Conference · 2015 · OSTI ID:22824507

De Angelis, Giorgio; Fedeli, Carlo; Capone, Mauro; +3 more

Challenges at the Frontiers of Matter and Energy: Transformative Opportunities for Discovery Science

Program Document · 2015 · OSTI ID:1283188

Hemminger, John C.; Sarrao, John; Crabtree, George; +2 more

Computational Materials Science and Chemistry: Accelerating Discovery and Innovation through Simulation-Based Engineering and Science

Program Document · 2010 · OSTI ID:1294275

Crabtree, George; Glotzer, Sharon; McCurdy, Bill; +1 more

Related Subjects

Science & Technology - Other Topics

Title: Does repeat synthesis in materials chemistry obey a power law?

Citation Formats

References (58)

Similar Records

Related Subjects