DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities

Abstract

Benchmarking is a community-based and (preferably) community-driven activity involving consensus-based decisions on how to make reproducible, fair, and relevant assessments. In catalysis science, important catalyst performance metrics include activity, selectivity, and the deactivation profile, which enable comparisons between new and standard catalysts. Benchmarking also requires careful documentation, archiving, and sharing of methods and measurements, to ensure that the full value of research data can be realized. Beyond these goals, benchmarking presents unique opportunities to advance and accelerate understanding of complex reaction systems by combining and comparing experimental information from multiple, in situ and operando techniques with theoretical insights derived from calculations characterizing model systems. This Perspective describes the origins and uses of benchmarking and its applications in computational catalysis, heterogeneous catalysis, molecular catalysis, and electrocatalysis. As a result, it also discusses opportunities and challenges for future developments in these fields.

Authors:
; ; ; ; ; ; ; ; ;
Publication Date:
Research Org.:
SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States); Brookhaven National Laboratory (BNL), Upton, NY (United States); Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
OSTI Identifier:
1243133
Alternate Identifier(s):
OSTI ID: 1245000; OSTI ID: 1257959; OSTI ID: 1337213; OSTI ID: 1351981; OSTI ID: 1773108
Report Number(s):
BNL-112274-2016-JA; PNNL-SA-116548
Journal ID: ISSN 2155-5435
Grant/Contract Number:  
AC02-76SF00515; AC05-76RL01830; SC0012704
Resource Type:
Published Article
Journal Name:
ACS Catalysis
Additional Journal Information:
Journal Name: ACS Catalysis Journal Volume: 6; Journal ID: ISSN 2155-5435
Publisher:
American Chemical Society
Country of Publication:
United States
Language:
English
Subject:
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; benchmarking; catalytic performance; computational catalysis; heterogeneous catalysis; molecular catalysis; electrocatalysis

Citation Formats

Bligaard, Thomas, Bullock, R. Morris, Campbell, Charles T., Chen, Jingguang G., Gates, Bruce C., Gorte, Raymond J., Jones, Christopher W., Jones, William D., Kitchin, John R., and Scott, Susannah L. Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities. United States: N. p., 2016. Web. doi:10.1021/acscatal.6b00183.
Bligaard, Thomas, Bullock, R. Morris, Campbell, Charles T., Chen, Jingguang G., Gates, Bruce C., Gorte, Raymond J., Jones, Christopher W., Jones, William D., Kitchin, John R., & Scott, Susannah L. Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities. United States. https://doi.org/10.1021/acscatal.6b00183
Bligaard, Thomas, Bullock, R. Morris, Campbell, Charles T., Chen, Jingguang G., Gates, Bruce C., Gorte, Raymond J., Jones, Christopher W., Jones, William D., Kitchin, John R., and Scott, Susannah L. Fri . "Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities". United States. https://doi.org/10.1021/acscatal.6b00183.
@article{osti_1243133,
title = {Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities},
author = {Bligaard, Thomas and Bullock, R. Morris and Campbell, Charles T. and Chen, Jingguang G. and Gates, Bruce C. and Gorte, Raymond J. and Jones, Christopher W. and Jones, William D. and Kitchin, John R. and Scott, Susannah L.},
abstractNote = {Benchmarking is a community-based and (preferably) community-driven activity involving consensus-based decisions on how to make reproducible, fair, and relevant assessments. In catalysis science, important catalyst performance metrics include activity, selectivity, and the deactivation profile, which enable comparisons between new and standard catalysts. Benchmarking also requires careful documentation, archiving, and sharing of methods and measurements, to ensure that the full value of research data can be realized. Beyond these goals, benchmarking presents unique opportunities to advance and accelerate understanding of complex reaction systems by combining and comparing experimental information from multiple, in situ and operando techniques with theoretical insights derived from calculations characterizing model systems. This Perspective describes the origins and uses of benchmarking and its applications in computational catalysis, heterogeneous catalysis, molecular catalysis, and electrocatalysis. As a result, it also discusses opportunities and challenges for future developments in these fields.},
doi = {10.1021/acscatal.6b00183},
journal = {ACS Catalysis},
number = ,
volume = 6,
place = {United States},
year = {Fri Apr 01 00:00:00 EDT 2016},
month = {Fri Apr 01 00:00:00 EDT 2016}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1021/acscatal.6b00183

Citation Metrics:
Cited by: 174 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Benchmarking Hydrogen Evolving Reaction and Oxygen Evolving Reaction Electrocatalysts for Solar Water Splitting Devices
journal, March 2015

  • McCrory, Charles C. L.; Jung, Suho; Ferrer, Ivonne M.
  • Journal of the American Chemical Society, Vol. 137, Issue 13
  • DOI: 10.1021/ja510442p

Origin of highly active metal–organic framework catalysts: defects? Defects!
journal, January 2016

  • Canivet, J.; Vandichel, M.; Farrusseng, D.
  • Dalton Transactions, Vol. 45, Issue 10
  • DOI: 10.1039/C5DT03522H

Electron Microscopy of Solid Catalysts—Transforming from a Challenge to a Toolbox
journal, March 2015

  • Su, Dang Sheng; Zhang, Bingsen; Schlögl, Robert
  • Chemical Reviews, Vol. 115, Issue 8
  • DOI: 10.1021/cr500084c

Sharing Detailed Research Data Is Associated with Increased Citation Rate
journal, March 2007


Promoting an open research culture
journal, June 2015


Comprehensive Benchmarking of a Density-Dependent Dispersion Correction
journal, October 2011

  • Steinmann, Stephan N.; Corminboeuf, Clemence
  • Journal of Chemical Theory and Computation, Vol. 7, Issue 11
  • DOI: 10.1021/ct200602x

Priming Intelligent Behavior: An Elusive Phenomenon
journal, April 2013


Stability and Metastability of Clusters in a Reactive Atmosphere: Theoretical Evidence for Unexpected Stoichiometries of Mg M O x
journal, September 2013


Activity of CeOx and TiOx Nanoparticles Grown on Au(111) in the Water-Gas Shift Reaction
journal, December 2007


Catalysts under Controlled Atmospheres in the Transmission Electron Microscope
journal, April 2014

  • Hansen, Thomas W.; Wagner, Jakob B.
  • ACS Catalysis, Vol. 4, Issue 6
  • DOI: 10.1021/cs401148d

Activity benchmarks and requirements for Pt, Pt-alloy, and non-Pt oxygen reduction catalysts for PEMFCs
journal, March 2005

  • Gasteiger, Hubert A.; Kocha, Shyam S.; Sompalli, Bhaskar
  • Applied Catalysis B: Environmental, Vol. 56, Issue 1-2, p. 9-35
  • DOI: 10.1016/j.apcatb.2004.06.021

Experimental Aspects in Benchmarking of the Electrocatalytic Activity
journal, November 2014

  • Čolić, Viktor; Tymoczko, Jakub; Maljusch, Artjom
  • ChemElectroChem, Vol. 2, Issue 1
  • DOI: 10.1002/celc.201402295

Raise standards for preclinical cancer research
journal, March 2012

  • Begley, C. Glenn; Ellis, Lee M.
  • Nature, Vol. 483, Issue 7391
  • DOI: 10.1038/483531a

What Drives Academic Data Sharing?
journal, February 2015


A benchmark database for adsorption bond energies to transition metal surfaces and comparison to selected DFT functionals
journal, October 2015


Believe it or not: how much can we rely on published data on potential drug targets?
journal, August 2011

  • Prinz, Florian; Schlange, Thomas; Asadullah, Khusru
  • Nature Reviews Drug Discovery, Vol. 10, Issue 9
  • DOI: 10.1038/nrd3439-c1

Why Most Published Research Findings Are False
journal, August 2005


Predicting accurate absolute binding energies in aqueous solution: thermodynamic considerations for electronic structure methods
journal, January 2015

  • Jensen, Jan H.
  • Physical Chemistry Chemical Physics, Vol. 17, Issue 19
  • DOI: 10.1039/C5CP00628G

Analysis of Catalyst Surface Structure by Physical Sorption
book, January 2013


Continuous flow organometallic catalysis: new wind in old sails
journal, January 2011

  • Hintermair, Ulrich; Franciò, Giancarlo; Leitner, Walter
  • Chemical Communications, Vol. 47, Issue 13
  • DOI: 10.1039/c0cc04958a

“Turning Over” Definitions in Catalytic Cycles
journal, November 2012

  • Kozuch, Sebastian; Martin, Jan M. L.
  • ACS Catalysis, Vol. 2, Issue 12
  • DOI: 10.1021/cs3005264

Quantum Chemical Benchmarking, Validation, and Prediction of Acidity Constants for Substituted Pyridinium Ions and Pyridinyl Radicals
journal, August 2012

  • Keith, John A.; Carter, Emily A.
  • Journal of Chemical Theory and Computation, Vol. 8, Issue 9
  • DOI: 10.1021/ct300295g

Density functional theory in surface chemistry and catalysis
journal, January 2011

  • Norskov, J. K.; Abild-Pedersen, F.; Studt, F.
  • Proceedings of the National Academy of Sciences, Vol. 108, Issue 3
  • DOI: 10.1073/pnas.1006652108

Control of Metal Nanocrystal Size Reveals Metal-Support Interface Role for Ceria Catalysts
journal, July 2013


Experimental criterion for the absence of artifacts in the measurement of rates of heterogeneous catalytic reactions
journal, November 1982

  • Madon, Rostam J.; Boudart, Michel
  • Industrial & Engineering Chemistry Fundamentals, Vol. 21, Issue 4, p. 438-447
  • DOI: 10.1021/i100008a022

Not so loosely bound rare gas atoms: finite-temperature vibrational fingerprints of neutral gold-cluster complexes
journal, August 2013


Rate-Enhancing Roles of Water Molecules in Methyltrioxorhenium-Catalyzed Olefin Epoxidation by Hydrogen Peroxide
journal, July 2015

  • Goldsmith, Bryan R.; Hwang, Taeho; Seritan, Stefan
  • Journal of the American Chemical Society, Vol. 137, Issue 30
  • DOI: 10.1021/jacs.5b03750

Using scaling relations to understand trends in the catalytic activity of transition metals
journal, January 2008


Finding the Rate-Determining Step in a Mechanism
journal, December 2001


Examples of Effective Data Sharing in Scientific Publishing
journal, May 2015


Bringing Digital Science Deep Inside the Scientific Article: the Elsevier Article of the Future Project
journal, April 2014

  • Aalbersberg, IJsbrand Jan; Atzeni, Sophia; Koers, Hylke
  • LIBER Quarterly, Vol. 23, Issue 4
  • DOI: 10.18352/lq.8446

The Open Knowledge Foundation: Open Data Means Better Science
journal, December 2011


Dispersion Interactions with Density-Functional Theory: Benchmarking Semiempirical and Interatomic Pairwise Corrected Density Functionals
journal, October 2011

  • Marom, Noa; Tkatchenko, Alexandre; Rossi, Mariana
  • Journal of Chemical Theory and Computation, Vol. 7, Issue 12
  • DOI: 10.1021/ct2005616

The Economics of Reproducibility in Preclinical Research
journal, June 2015


A balanced procedure for the treatment of cluster-ligand interactions on gold phosphine systems in catalysis
journal, March 2014

  • Mollenhauer, Doreen; Gaston, Nicola
  • Journal of Computational Chemistry, Vol. 35, Issue 13
  • DOI: 10.1002/jcc.23578

Temperature-Programmed Desorption of Hydrogen from Platinum Particles on γ-Al2O3: Evidence of Platinum-Catalyzed Dehydroxylation of γ-Al2O3
journal, July 1999

  • Alexeev, Oleg; Kim, Do-Woan; Graham, George W.
  • Journal of Catalysis, Vol. 185, Issue 1
  • DOI: 10.1006/jcat.1999.2489

Highly active metallocene catalysts for olefin polymerization
journal, January 1998

  • Kaminsky, Walter
  • Journal of the Chemical Society, Dalton Transactions, Issue 9
  • DOI: 10.1039/a800056e

A Case Study of the Mechanism of Alcohol-Mediated Morita Baylis–Hillman Reactions. The Importance of Experimental Observations
journal, March 2015

  • Plata, R. Erik; Singleton, Daniel A.
  • Journal of the American Chemical Society, Vol. 137, Issue 11
  • DOI: 10.1021/ja5111392

Chemical accuracy in ab initio thermochemistry and spectroscopy: current strategies and future challenges
journal, January 2012

  • Peterson, Kirk A.; Feller, David; Dixon, David A.
  • Theoretical Chemistry Accounts, Vol. 131, Issue 1
  • DOI: 10.1007/s00214-011-1079-5

Introducing structural sensitivity into adsorption–energy scaling relations by means of coordination numbers
journal, April 2015

  • Calle-Vallejo, Federico; Loffreda, David; Koper, Marc T. M.
  • Nature Chemistry, Vol. 7, Issue 5
  • DOI: 10.1038/nchem.2226

Self-correction in science at work
journal, June 2015


Gold reference catalysts
journal, March 2003


Benchmarking of Homogeneous Electrocatalysts: Overpotential, Turnover Frequency, Limiting Turnover Number
journal, April 2015

  • Costentin, Cyrille; Passard, Guillaume; Savéant, Jean-Michel
  • Journal of the American Chemical Society, Vol. 137, Issue 16
  • DOI: 10.1021/jacs.5b00914

“Mindless” DFT Benchmarking
journal, March 2009

  • Korth, Martin; Grimme, Stefan
  • Journal of Chemical Theory and Computation, Vol. 5, Issue 4
  • DOI: 10.1021/ct800511q

Water-Catalyzed Activation of H 2 O 2 by Methyltrioxorhenium: A Combined Computational–Experimental Study
journal, November 2013

  • Hwang, Taeho; Goldsmith, Bryan R.; Peters, Baron
  • Inorganic Chemistry, Vol. 52, Issue 24
  • DOI: 10.1021/ic401343m

Toward a Database of Chemically Accurate Barrier Heights for Reactions of Molecules with Metal Surfaces
journal, September 2015


Quantitative Assay for the Direct Comparison of Platinum Catalysts in Benzene H/D Exchange
journal, September 2009

  • Hickman, Amanda J.; Villalobos, Janette M.; Sanford, Melanie S.
  • Organometallics, Vol. 28, Issue 18
  • DOI: 10.1021/om900495n

Selectivity in Propene Polymerization with Metallocene Catalysts
journal, April 2000

  • Resconi, Luigi; Cavallo, Luigi; Fait, Anna
  • Chemical Reviews, Vol. 100, Issue 4
  • DOI: 10.1021/cr9804691

Preparing Your Manuscript for Submission to ACS Catalysis
journal, July 2014

  • Jones, Christopher W.; Smith, David J.; Gunnoe, T. Brent
  • ACS Catalysis, Vol. 4, Issue 8
  • DOI: 10.1021/cs501043q

Benchmarking Heterogeneous Electrocatalysts for the Oxygen Evolution Reaction
journal, October 2013

  • McCrory, Charles C. L.; Jung, Suho; Peters, Jonas C.
  • Journal of the American Chemical Society, Vol. 135, Issue 45
  • DOI: 10.1021/ja407115p

On the accuracy of density-functional theory exchange-correlation functionals for H bonds in small water clusters: Benchmarks approaching the complete basis set limit
journal, November 2007

  • Santra, Biswajit; Michaelides, Angelos; Scheffler, Matthias
  • The Journal of Chemical Physics, Vol. 127, Issue 18
  • DOI: 10.1063/1.2790009

Using benchmarking to advance research: a challenge to software engineering
conference, January 2003

  • Sim, S. E.; Easterbrook, S.; Holt, R. C.
  • 25th International Conference on Software Engineering, 2003. Proceedings.
  • DOI: 10.1109/ICSE.2003.1201189

Toward the rational benchmarking of homogeneous H 2 -evolving catalysts
journal, January 2014

  • Artero, Vincent; Saveant, Jean-Michel
  • Energy Environ. Sci., Vol. 7, Issue 11
  • DOI: 10.1039/C4EE01709A

Multielectron, Multistep Molecular Catalysis of Electrochemical Reactions: Benchmarking of Homogeneous Catalysts
journal, June 2014


Enhancing reproducibility
journal, April 2013

  • Doerr, Allison; Vogt, Nina
  • Nature Methods, Vol. 10, Issue 5, p. 367-367
  • DOI: 10.1038/nmeth.2471

Recent Advances in Asymmetric Catalytic Metal Carbene Transformations
journal, April 1998

  • Doyle, Michael P.; Forbes, David C.
  • Chemical Reviews, Vol. 98, Issue 2
  • DOI: 10.1021/cr940066a

Making scientific computations reproducible
journal, January 2000

  • Schwab, M.; Karrenbach, N.; Claerbout, J.
  • Computing in Science & Engineering, Vol. 2, Issue 6
  • DOI: 10.1109/5992.881708

How Benchmarking in Science Can Lead to a Reversal of Priorities
journal, February 2015

  • Farlin, Julien; Majewsky, Marius
  • Environmental Science & Technology, Vol. 49, Issue 5
  • DOI: 10.1021/acs.est.5b00389

Benchmarking the Stability of Oxygen Evolution Reaction Catalysts: The Importance of Monitoring Mass Losses
journal, October 2014

  • Frydendal, Rasmus; Paoli, Elisa A.; Knudsen, Brian P.
  • ChemElectroChem, Vol. 1, Issue 12
  • DOI: 10.1002/celc.201402262

Towards the computational design of solid catalysts
journal, April 2009

  • Nørskov, J.; Bligaard, T.; Rossmeisl, J.
  • Nature Chemistry, Vol. 1, Issue 1, p. 37-46
  • DOI: 10.1038/nchem.121

Striving for Reproducible Science
journal, November 2015


Solving reproducibility
journal, June 2015


An invitation to reproducible computational research
journal, June 2010


Comment on “‘Turning Over’ Definitions in Catalytic Cycles”
journal, February 2013


Reaction Progress Kinetic Analysis: A Powerful Methodology for Mechanistic Studies of Complex Catalytic Reactions
journal, July 2005


Editorial: Introducing Tutorials
journal, July 2015


Degree of Rate Control: How Much the Energies of Intermediates and Transition States Control Rates
journal, June 2009

  • Stegelmann, Carsten; Andreasen, Anders; Campbell, Charles T.
  • Journal of the American Chemical Society, Vol. 131, Issue 23
  • DOI: 10.1021/ja9000097

Homogeneous Ni Catalysts for H 2 Oxidation and Production: An Assessment of Theoretical Methods, from Density Functional Theory to Post Hartree−Fock Correlated Wave-Function Theory
journal, December 2010

  • Chen, Shentan; Raugei, Simone; Rousseau, Roger
  • The Journal of Physical Chemistry A, Vol. 114, Issue 48
  • DOI: 10.1021/jp106800n