Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities

Bligaard, Thomas; Bullock, R. Morris; Campbell, Charles T.; Chen, Jingguang G.; Gates, Bruce C.; Gorte, Raymond J.; Jones, Christopher W.; Jones, William D.; Kitchin, John R.; Scott, Susannah L.

doi:10.1021/acscatal.6b00183

Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities

Journal Article · Fri Apr 01 00:00:00 UTC 2016 · ACS Catalysis

DOI:https://doi.org/10.1021/acscatal.6b00183· OSTI ID:1243133

Bligaard, Thomas; Bullock, R. Morris; Campbell, Charles T.; Chen, Jingguang G.; Gates, Bruce C.; Gorte, Raymond J.; Jones, Christopher W.; Jones, William D.; Kitchin, John R.; Scott, Susannah L.

Not Available

View Journal Article

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES); USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22)

Grant/Contract Number:: AC02-76SF00515; AC05-76RL01830; SC0012704

OSTI ID:: 1243133

Alternate ID(s):: OSTI ID: 1337213
OSTI ID: 1351981
OSTI ID: 1245000
OSTI ID: 1773108
OSTI ID: 1257959

Report Number(s):: BNL--112274-2016-JA; PNNL-SA--116548

Journal Information:: ACS Catalysis, Journal Name: ACS Catalysis Vol. 6; ISSN 2155-5435

Publisher:: American Chemical SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (77)

Reaction Progress Kinetic Analysis: A Powerful Methodology for Mechanistic Studies of Complex Catalytic Reactions Blackmond, Donna G. Angewandte Chemie International Edition, Vol. 44, Issue 28 https://doi.org/10.1002/anie.200462544	journal	July 2005
Multielectron, Multistep Molecular Catalysis of Electrochemical Reactions: Benchmarking of Homogeneous Catalysts Costentin, Cyrille; Savéant, Jean-Michel ChemElectroChem, Vol. 1, Issue 7 https://doi.org/10.1002/celc.201300263	journal	June 2014
Benchmarking the Stability of Oxygen Evolution Reaction Catalysts: The Importance of Monitoring Mass Losses Frydendal, Rasmus; Paoli, Elisa A.; Knudsen, Brian P. ChemElectroChem, Vol. 1, Issue 12 https://doi.org/10.1002/celc.201402262	journal	October 2014
Experimental Aspects in Benchmarking of the Electrocatalytic Activity Čolić, Viktor; Tymoczko, Jakub; Maljusch, Artjom ChemElectroChem, Vol. 2, Issue 1 https://doi.org/10.1002/celc.201402295	journal	November 2014
A balanced procedure for the treatment of cluster-ligand interactions on gold phosphine systems in catalysis Mollenhauer, Doreen; Gaston, Nicola Journal of Computational Chemistry, Vol. 35, Issue 13 https://doi.org/10.1002/jcc.23578	journal	March 2014
Temperature-Programmed Desorption of Hydrogen from Platinum Particles on γ-Al2O3: Evidence of Platinum-Catalyzed Dehydroxylation of γ-Al2O3 Alexeev, Oleg; Kim, Do-Woan; Graham, George W. Journal of Catalysis, Vol. 185, Issue 1 https://doi.org/10.1006/jcat.1999.2489	journal	July 1999
Finding the Rate-Determining Step in a Mechanism Campbell, Charles T. Journal of Catalysis, Vol. 204, Issue 2 https://doi.org/10.1006/jcat.2001.3396	journal	December 2001
Gold reference catalysts Hashmi, A. Stephen K. Gold Bulletin, Vol. 36, Issue 1, p. 24-24 https://doi.org/10.1007/BF03214864	journal	March 2003
Chemical accuracy in ab initio thermochemistry and spectroscopy: current strategies and future challenges Peterson, Kirk A.; Feller, David; Dixon, David A. Theoretical Chemistry Accounts, Vol. 131, Issue 1 https://doi.org/10.1007/s00214-011-1079-5	journal	January 2012
On the Stability and Recyclability of Supported Metal–Ligand Complex Catalysts: Myths, Misconceptions and Critical Research Needs Jones, Christopher W. Topics in Catalysis, Vol. 53, Issue 13-14 https://doi.org/10.1007/s11244-010-9513-9	journal	May 2010
Analysis of Catalyst Surface Structure by Physical Sorption Hammond, Karl D.; Conner, Wm. Curtis Advances in Catalysis https://doi.org/10.1016/B978-0-12-420173-6.00001-2	book	January 2013
Activity benchmarks and requirements for Pt, Pt-alloy, and non-Pt oxygen reduction catalysts for PEMFCs Gasteiger, Hubert A.; Kocha, Shyam S.; Sompalli, Bhaskar Applied Catalysis B: Environmental, Vol. 56, Issue 1-2, p. 9-35 https://doi.org/10.1016/j.apcatb.2004.06.021	journal	March 2005
A benchmark database for adsorption bond energies to transition metal surfaces and comparison to selected DFT functionals Wellendorff, Jess; Silbaugh, Trent L.; Garcia-Pintos, Delfina Surface Science, Vol. 640 https://doi.org/10.1016/j.susc.2015.03.023	journal	October 2015
Striving for Reproducible Science Sweedler, Jonathan V. Analytical Chemistry, Vol. 87, Issue 23 https://doi.org/10.1021/acs.analchem.5b04300	journal	November 2015
How Benchmarking in Science Can Lead to a Reversal of Priorities Farlin, Julien; Majewsky, Marius Environmental Science & Technology, Vol. 49, Issue 5 https://doi.org/10.1021/acs.est.5b00389	journal	February 2015
Do Practical Standard Coupled Cluster Calculations Agree Better than Kohn–Sham Calculations with Currently Available Functionals When Compared to the Best Available Experimental Data for Dissociation Energies of Bonds to 3 d Transition Metals? Xu, Xuefei; Zhang, Wenjing; Tang, Mingsheng Journal of Chemical Theory and Computation, Vol. 11, Issue 5 https://doi.org/10.1021/acs.jctc.5b00081	journal	April 2015
Toward a Database of Chemically Accurate Barrier Heights for Reactions of Molecules with Metal Surfaces Kroes, Geert-Jan The Journal of Physical Chemistry Letters, Vol. 6, Issue 20 https://doi.org/10.1021/acs.jpclett.5b01344	journal	September 2015
Editorial: Introducing Tutorials Chirik, Paul J. Organometallics, Vol. 34, Issue 20 https://doi.org/10.1021/acs.organomet.5b00815	journal	July 2015
Examples of Effective Data Sharing in Scientific Publishing Kitchin, John R. ACS Catalysis, Vol. 5, Issue 6 https://doi.org/10.1021/acscatal.5b00538	journal	May 2015
Electron Microscopy of Solid Catalysts—Transforming from a Challenge to a Toolbox Su, Dang Sheng; Zhang, Bingsen; Schlögl, Robert Chemical Reviews, Vol. 115, Issue 8 https://doi.org/10.1021/cr500084c	journal	March 2015
Recent Advances in Asymmetric Catalytic Metal Carbene Transformations Doyle, Michael P.; Forbes, David C. Chemical Reviews, Vol. 98, Issue 2 https://doi.org/10.1021/cr940066a	journal	April 1998
Selectivity in Propene Polymerization with Metallocene Catalysts Resconi, Luigi; Cavallo, Luigi; Fait, Anna Chemical Reviews, Vol. 100, Issue 4 https://doi.org/10.1021/cr9804691	journal	April 2000
“Turning Over” Definitions in Catalytic Cycles Kozuch, Sebastian; Martin, Jan M. L. ACS Catalysis, Vol. 2, Issue 12 https://doi.org/10.1021/cs3005264	journal	November 2012
Comment on “‘Turning Over’ Definitions in Catalytic Cycles” Lente, Gábor ACS Catalysis, Vol. 3, Issue 3 https://doi.org/10.1021/cs300846b	journal	February 2013
Catalysts under Controlled Atmospheres in the Transmission Electron Microscope Hansen, Thomas W.; Wagner, Jakob B. ACS Catalysis, Vol. 4, Issue 6 https://doi.org/10.1021/cs401148d	journal	April 2014
Preparing Your Manuscript for Submission to ACS Catalysis Jones, Christopher W.; Smith, David J.; Gunnoe, T. Brent ACS Catalysis, Vol. 4, Issue 8 https://doi.org/10.1021/cs501043q	journal	July 2014
Dispersion Interactions with Density-Functional Theory: Benchmarking Semiempirical and Interatomic Pairwise Corrected Density Functionals Marom, Noa; Tkatchenko, Alexandre; Rossi, Mariana Journal of Chemical Theory and Computation, Vol. 7, Issue 12 https://doi.org/10.1021/ct2005616	journal	October 2011
Comprehensive Benchmarking of a Density-Dependent Dispersion Correction Steinmann, Stephan N.; Corminboeuf, Clemence Journal of Chemical Theory and Computation, Vol. 7, Issue 11 https://doi.org/10.1021/ct200602x	journal	October 2011
Quantum Chemical Benchmarking, Validation, and Prediction of Acidity Constants for Substituted Pyridinium Ions and Pyridinyl Radicals Keith, John A.; Carter, Emily A. Journal of Chemical Theory and Computation, Vol. 8, Issue 9 https://doi.org/10.1021/ct300295g	journal	August 2012
Benchmark Energetic Data in a Model System for Grubbs II Metathesis Catalysis and Their Use for the Development, Assessment, and Validation of Electronic Structure Methods Zhao, Yan; Truhlar, Donald G. Journal of Chemical Theory and Computation, Vol. 5, Issue 2 https://doi.org/10.1021/ct800386d	journal	January 2009
“Mindless” DFT Benchmarking Korth, Martin; Grimme, Stefan Journal of Chemical Theory and Computation, Vol. 5, Issue 4 https://doi.org/10.1021/ct800511q	journal	March 2009
Experimental criterion for the absence of artifacts in the measurement of rates of heterogeneous catalytic reactions Madon, Rostam J.; Boudart, Michel Industrial & Engineering Chemistry Fundamentals, Vol. 21, Issue 4, p. 438-447 https://doi.org/10.1021/i100008a022	journal	November 1982
Water-Catalyzed Activation of H ₂ O ₂ by Methyltrioxorhenium: A Combined Computational–Experimental Study Hwang, Taeho; Goldsmith, Bryan R.; Peters, Baron Inorganic Chemistry, Vol. 52, Issue 24 https://doi.org/10.1021/ic401343m	journal	November 2013
Benchmarking Heterogeneous Electrocatalysts for the Oxygen Evolution Reaction McCrory, Charles C. L.; Jung, Suho; Peters, Jonas C. Journal of the American Chemical Society, Vol. 135, Issue 45 https://doi.org/10.1021/ja407115p	journal	October 2013
Benchmarking Hydrogen Evolving Reaction and Oxygen Evolving Reaction Electrocatalysts for Solar Water Splitting Devices McCrory, Charles C. L.; Jung, Suho; Ferrer, Ivonne M. Journal of the American Chemical Society, Vol. 137, Issue 13 https://doi.org/10.1021/ja510442p	journal	March 2015
A Case Study of the Mechanism of Alcohol-Mediated Morita Baylis–Hillman Reactions. The Importance of Experimental Observations Plata, R. Erik; Singleton, Daniel A. Journal of the American Chemical Society, Vol. 137, Issue 11 https://doi.org/10.1021/ja5111392	journal	March 2015
Degree of Rate Control: How Much the Energies of Intermediates and Transition States Control Rates Stegelmann, Carsten; Andreasen, Anders; Campbell, Charles T. Journal of the American Chemical Society, Vol. 131, Issue 23 https://doi.org/10.1021/ja9000097	journal	June 2009
Benchmarking of Homogeneous Electrocatalysts: Overpotential, Turnover Frequency, Limiting Turnover Number Costentin, Cyrille; Passard, Guillaume; Savéant, Jean-Michel Journal of the American Chemical Society, Vol. 137, Issue 16 https://doi.org/10.1021/jacs.5b00914	journal	April 2015
Rate-Enhancing Roles of Water Molecules in Methyltrioxorhenium-Catalyzed Olefin Epoxidation by Hydrogen Peroxide Goldsmith, Bryan R.; Hwang, Taeho; Seritan, Stefan Journal of the American Chemical Society, Vol. 137, Issue 30 https://doi.org/10.1021/jacs.5b03750	journal	July 2015
Can Contemporary Density Functional Theory Predict Energy Spans in Molecular Catalysis Accurately Enough To Be Applicable for in Silico Catalyst Design? A Computational/Experimental Case Study for the Ruthenium-Catalyzed Hydrogenation of Olefins Rohmann, Kai; Hölscher, Markus; Leitner, Walter Journal of the American Chemical Society, Vol. 138, Issue 1 https://doi.org/10.1021/jacs.5b11997	journal	December 2015
A Standard Set of Pericyclic Reactions of Hydrocarbons for the Benchmarking of Computational Methods: The Performance of ab Initio, Density Functional, CASSCF, CASPT2, and CBS-QB3 Methods for the Prediction of Activation Barriers, Reaction Energetics, and Transition State Geometries Guner, Vildan; Khuong, Kelli S.; Leach, Andrew G. The Journal of Physical Chemistry A, Vol. 107, Issue 51 https://doi.org/10.1021/jp035501w	journal	December 2003
Homogeneous Ni Catalysts for H ₂ Oxidation and Production: An Assessment of Theoretical Methods, from Density Functional Theory to Post Hartree−Fock Correlated Wave-Function Theory Chen, Shentan; Raugei, Simone; Rousseau, Roger The Journal of Physical Chemistry A, Vol. 114, Issue 48 https://doi.org/10.1021/jp106800n	journal	December 2010
Quantitative Assay for the Direct Comparison of Platinum Catalysts in Benzene H/D Exchange Hickman, Amanda J.; Villalobos, Janette M.; Sanford, Melanie S. Organometallics, Vol. 28, Issue 18 https://doi.org/10.1021/om900495n	journal	September 2009
Raise standards for preclinical cancer research Begley, C. Glenn; Ellis, Lee M. Nature, Vol. 483, Issue 7391 https://doi.org/10.1038/483531a	journal	March 2012
Towards the computational design of solid catalysts Nørskov, J.; Bligaard, T.; Rossmeisl, J. Nature Chemistry, Vol. 1, Issue 1, p. 37-46 https://doi.org/10.1038/nchem.121	journal	April 2009
Introducing structural sensitivity into adsorption–energy scaling relations by means of coordination numbers Calle-Vallejo, Federico; Loffreda, David; Koper, Marc T. M. Nature Chemistry, Vol. 7, Issue 5 https://doi.org/10.1038/nchem.2226	journal	April 2015
Enhancing reproducibility Doerr, Allison; Vogt, Nina Nature Methods, Vol. 10, Issue 5, p. 367-367 https://doi.org/10.1038/nmeth.2471	journal	April 2013
Believe it or not: how much can we rely on published data on potential drug targets? Prinz, Florian; Schlange, Thomas; Asadullah, Khusru Nature Reviews Drug Discovery, Vol. 10, Issue 9 https://doi.org/10.1038/nrd3439-c1	journal	August 2011
Toward the rational benchmarking of homogeneous H ₂ -evolving catalysts Artero, Vincent; Saveant, Jean-Michel Energy Environ. Sci., Vol. 7, Issue 11 https://doi.org/10.1039/C4EE01709A	journal	January 2014
Predicting accurate absolute binding energies in aqueous solution: thermodynamic considerations for electronic structure methods Jensen, Jan H. Physical Chemistry Chemical Physics, Vol. 17, Issue 19 https://doi.org/10.1039/C5CP00628G	journal	January 2015
Origin of highly active metal–organic framework catalysts: defects? Defects! Canivet, J.; Vandichel, M.; Farrusseng, D. Dalton Transactions, Vol. 45, Issue 10 https://doi.org/10.1039/C5DT03522H	journal	January 2016
Highly active metallocene catalysts for olefin polymerization Kaminsky, Walter Journal of the Chemical Society, Dalton Transactions, Issue 9 https://doi.org/10.1039/a800056e	journal	January 1998
Continuous flow organometallic catalysis: new wind in old sails Hintermair, Ulrich; Franciò, Giancarlo; Leitner, Walter Chemical Communications, Vol. 47, Issue 13 https://doi.org/10.1039/c0cc04958a	journal	January 2011
On the accuracy of density-functional theory exchange-correlation functionals for H bonds in small water clusters: Benchmarks approaching the complete basis set limit Santra, Biswajit; Michaelides, Angelos; Scheffler, Matthias The Journal of Chemical Physics, Vol. 127, Issue 18 https://doi.org/10.1063/1.2790009	journal	November 2007
Tests of the RPBE, revPBE, τ-HCTHhyb, ωB97X-D, and MOHLYP density functional approximations and 29 others against representative databases for diverse bond energies and barrier heights in catalysis Yang, Ke; Zheng, Jingjing; Zhao, Yan The Journal of Chemical Physics, Vol. 132, Issue 16 https://doi.org/10.1063/1.3382342	journal	April 2010
Density functional theory in surface chemistry and catalysis Norskov, J. K.; Abild-Pedersen, F.; Studt, F. Proceedings of the National Academy of Sciences, Vol. 108, Issue 3 https://doi.org/10.1073/pnas.1006652108	journal	January 2011
Using scaling relations to understand trends in the catalytic activity of transition metals Jones, G.; Bligaard, T.; Abild-Pedersen, F. Journal of Physics: Condensed Matter, Vol. 20, Issue 6 https://doi.org/10.1088/0953-8984/20/6/064239	journal	January 2008
Not so loosely bound rare gas atoms: finite-temperature vibrational fingerprints of neutral gold-cluster complexes Ghiringhelli, Luca M.; Gruene, Philipp; Lyon, Jonathan T. New Journal of Physics, Vol. 15, Issue 8 https://doi.org/10.1088/1367-2630/15/8/083003	journal	August 2013
An invitation to reproducible computational research Donoho, D. L. Biostatistics, Vol. 11, Issue 3 https://doi.org/10.1093/biostatistics/kxq028	journal	June 2010
Au N clusters ( N = 1 -- 6 ) supported on MgO(100) surfaces: Effect of exact exchange and dispersion interactions on adhesion energies Paz-Borbón, Lauro Oliver; Barcaro, Giovanni; Fortunelli, Alessandro Physical Review B, Vol. 85, Issue 15 https://doi.org/10.1103/PhysRevB.85.155409	journal	April 2012
Stability and Metastability of Clusters in a Reactive Atmosphere: Theoretical Evidence for Unexpected Stoichiometries of Mg M O x Bhattacharya, Saswata; Levchenko, Sergey V.; Ghiringhelli, Luca M. Physical Review Letters, Vol. 111, Issue 13 https://doi.org/10.1103/PhysRevLett.111.135501	journal	September 2013
Making scientific computations reproducible Schwab, M.; Karrenbach, N.; Claerbout, J. Computing in Science & Engineering, Vol. 2, Issue 6 https://doi.org/10.1109/5992.881708	journal	January 2000
Using benchmarking to advance research: a challenge to software engineering Sim, S. E.; Easterbrook, S.; Holt, R. C. 25th International Conference on Software Engineering, 2003. Proceedings. https://doi.org/10.1109/ICSE.2003.1201189	conference	January 2003
Activity of CeOx and TiOx Nanoparticles Grown on Au(111) in the Water-Gas Shift Reaction Rodriguez, J. A.; Ma, S.; Liu, P. Science, Vol. 318, Issue 5857 https://doi.org/10.1126/science.1150038	journal	December 2007
Control of Metal Nanocrystal Size Reveals Metal-Support Interface Role for Ceria Catalysts Cargnello, M.; Doan-Nguyen, V. V. T.; Gordon, T. R. Science, Vol. 341, Issue 6147 https://doi.org/10.1126/science.1240148	journal	July 2013
Promoting an open research culture Nosek, B. A.; Alter, G.; Banks, G. C. Science, Vol. 348, Issue 6242 https://doi.org/10.1126/science.aab2374	journal	June 2015
Self-correction in science at work Alberts, B.; Cicerone, R. J.; Fienberg, S. E. Science, Vol. 348, Issue 6242 https://doi.org/10.1126/science.aab3847	journal	June 2015
Estimating the reproducibility of psychological science No authors listed Science, Vol. 349, Issue 6251 https://doi.org/10.1126/science.aac4716	journal	August 2015
Solving reproducibility Buck, S. Science, Vol. 348, Issue 6242 https://doi.org/10.1126/science.aac8041	journal	June 2015
The Open Knowledge Foundation: Open Data Means Better Science Molloy, Jennifer C. PLoS Biology, Vol. 9, Issue 12 https://doi.org/10.1371/journal.pbio.1001195	journal	December 2011
The Economics of Reproducibility in Preclinical Research Freedman, Leonard P.; Cockburn, Iain M.; Simcoe, Timothy S. PLOS Biology, Vol. 13, Issue 6 https://doi.org/10.1371/journal.pbio.1002165	journal	June 2015
Why Most Published Research Findings Are False Ioannidis, John P. A. PLoS Medicine, Vol. 2, Issue 8 https://doi.org/10.1371/journal.pmed.0020124	journal	August 2005
Sharing Detailed Research Data Is Associated with Increased Citation Rate Piwowar, Heather A.; Day, Roger S.; Fridsma, Douglas B. PLoS ONE, Vol. 2, Issue 3 https://doi.org/10.1371/journal.pone.0000308	journal	March 2007
Priming Intelligent Behavior: An Elusive Phenomenon Shanks, David R.; Newell, Ben R.; Lee, Eun Hee PLoS ONE, Vol. 8, Issue 4 https://doi.org/10.1371/journal.pone.0056515	journal	April 2013
What Drives Academic Data Sharing? Fecher, Benedikt; Friesike, Sascha; Hebing, Marcel PLOS ONE, Vol. 10, Issue 2 https://doi.org/10.1371/journal.pone.0118053	journal	February 2015
A Standard Platinum/Silica Catalyst Wells, P. B. Platinum Metals Review, Vol. 29, Issue 4 https://doi.org/10.1595/003214085X294168174	journal	October 1985
Bringing Digital Science Deep Inside the Scientific Article: the Elsevier Article of the Future Project Aalbersberg, IJsbrand Jan; Atzeni, Sophia; Koers, Hylke LIBER Quarterly, Vol. 23, Issue 4 https://doi.org/10.18352/lq.8446	journal	April 2014

Similar Records

Related Subjects

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY
benchmarking
catalytic performance
computational catalysis
electrocatalysis
heterogeneous catalysis
molecular catalysis

Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities

Citation Formats

References (77)

Similar Records

Related Subjects