DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Benchmarking the acceleration of materials discovery by sequential learning

Abstract

Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any “good” material, discovery of all “good” materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 comparedmore » to random acquisition in specific scenarios. The results also show that certain choices of SL models are ill-suited for a given research goal resulting in substantial deceleration compared to random acquisition methods. The results provide quantitative guidance on how to tune an SL strategy for a given research goal and demonstrate the need for a new generation of materials-aware SL algorithms to further accelerate materials discovery.« less

Authors:
 [1]; ORCiD logo [2]; ORCiD logo [2]; ORCiD logo [2]; ORCiD logo [2]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [3]
  1. Accelerated Materials Design and Discovery, Toyota Research Institute, Los Altos, USA
  2. Joint Center for Artificial Photosynthesis, California Institute of Technology, Pasadena, USA
  3. Joint Center for Artificial Photosynthesis, California Institute of Technology, Pasadena, USA, Division of Engineering and Applied Science
Publication Date:
Research Org.:
California Institute of Technology (CalTech), Pasadena, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES)
OSTI Identifier:
1598712
Alternate Identifier(s):
OSTI ID: 1801646
Grant/Contract Number:  
SC0004993; SC0020383
Resource Type:
Published Article
Journal Name:
Chemical Science
Additional Journal Information:
Journal Name: Chemical Science Journal Volume: 11 Journal Issue: 10; Journal ID: ISSN 2041-6520
Publisher:
Royal Society of Chemistry (RSC)
Country of Publication:
United Kingdom
Language:
English
Subject:
36 MATERIALS SCIENCE

Citation Formats

Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., and Gregoire, John M. Benchmarking the acceleration of materials discovery by sequential learning. United Kingdom: N. p., 2020. Web. doi:10.1039/C9SC05999G.
Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., & Gregoire, John M. Benchmarking the acceleration of materials discovery by sequential learning. United Kingdom. https://doi.org/10.1039/C9SC05999G
Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., and Gregoire, John M. Wed . "Benchmarking the acceleration of materials discovery by sequential learning". United Kingdom. https://doi.org/10.1039/C9SC05999G.
@article{osti_1598712,
title = {Benchmarking the acceleration of materials discovery by sequential learning},
author = {Rohr, Brian and Stein, Helge S. and Guevarra, Dan and Wang, Yu and Haber, Joel A. and Aykol, Muratahan and Suram, Santosh K. and Gregoire, John M.},
abstractNote = {Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any “good” material, discovery of all “good” materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 compared to random acquisition in specific scenarios. The results also show that certain choices of SL models are ill-suited for a given research goal resulting in substantial deceleration compared to random acquisition methods. The results provide quantitative guidance on how to tune an SL strategy for a given research goal and demonstrate the need for a new generation of materials-aware SL algorithms to further accelerate materials discovery.},
doi = {10.1039/C9SC05999G},
journal = {Chemical Science},
number = 10,
volume = 11,
place = {United Kingdom},
year = {Wed Mar 11 00:00:00 EDT 2020},
month = {Wed Mar 11 00:00:00 EDT 2020}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1039/C9SC05999G

Citation Metrics:
Cited by: 47 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Active Learning with Statistical Models
journal, January 1996

  • Cohn, D. A.; Ghahramani, Z.; Jordan, M. I.
  • Journal of Artificial Intelligence Research, Vol. 4
  • DOI: 10.1613/jair.295

Functional mapping reveals mechanistic clusters for OER catalysis across (Cu–Mn–Ta–Co–Sn–Fe)O x composition and pH space
journal, January 2019

  • Stein, Helge S.; Guevarra, Dan; Shinde, Aniketa
  • Materials Horizons, Vol. 6, Issue 6
  • DOI: 10.1039/C8MH01641K

High-Dimensional Materials and Process Optimization Using Data-Driven Experimental Design with Well-Calibrated Uncertainty Estimates
journal, July 2017

  • Ling, Julia; Hutchinson, Maxwell; Antono, Erin
  • Integrating Materials and Manufacturing Innovation, Vol. 6, Issue 3
  • DOI: 10.1007/s40192-017-0098-z

Accelerated Discovery of Large Electrostrains in BaTiO 3 -Based Piezoelectrics Using Active Learning
journal, January 2018

  • Yuan, Ruihao; Liu, Zhen; Balachandran, Prasanna V.
  • Advanced Materials, Vol. 30, Issue 7
  • DOI: 10.1002/adma.201702884

Low-Scaling Algorithm for Nudged Elastic Band Calculations Using a Surrogate Machine Learning Model
journal, April 2019


Inkjet Printing Assisted Synthesis of Multicomponent Mesoporous Metal Oxides for Ultrafast Catalyst Exploration
journal, October 2012

  • Liu, Xiaonao; Shen, Yi; Yang, Ruoting
  • Nano Letters, Vol. 12, Issue 11
  • DOI: 10.1021/nl302992q

An Investigation of Thin-Film Ni–Fe Oxide Catalysts for the Electrochemical Evolution of Oxygen
journal, August 2013

  • Louie, Mary W.; Bell, Alexis T.
  • Journal of the American Chemical Society, Vol. 135, Issue 33
  • DOI: 10.1021/ja405351s

An Operando Investigation of (Ni–Fe–Co–Ce)O x System as Highly Efficient Electrocatalyst for Oxygen Evolution Reaction
journal, January 2017


Phoenics: A Bayesian Optimizer for Chemistry
journal, August 2018


100% clean and renewable wind, water, and sunlight (WWS) all-sector energy roadmaps for the 50 United States
journal, January 2015

  • Jacobson, Mark Z.; Delucchi, Mark A.; Bazouin, Guillaume
  • Energy & Environmental Science, Vol. 8, Issue 7
  • DOI: 10.1039/C5EE01283J

Prediction of Low-Thermal-Conductivity Compounds with First-Principles Anharmonic Lattice-Dynamics Calculations and Bayesian Optimization
journal, November 2015


Less is more: Sampling chemical space with active learning
journal, June 2018

  • Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas
  • The Journal of Chemical Physics, Vol. 148, Issue 24
  • DOI: 10.1063/1.5023802

Autonomous discovery in the chemical sciences part II: Outlook
journal, September 2019

  • Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F.
  • Angewandte Chemie International Edition
  • DOI: 10.1002/anie.201909989

The 2019 materials by design roadmap
journal, October 2018

  • Alberi, Kirstin; Nardelli, Marco Buongiorno; Zakutayev, Andriy
  • Journal of Physics D: Applied Physics, Vol. 52, Issue 1
  • DOI: 10.1088/1361-6463/aad926

Discovery of New Oxygen Evolution Reaction Electrocatalysts by Combinatorial Investigation of the Ni-La-Co-Ce Oxide Composition Space
journal, July 2014


Next-Generation Experimentation with Self-Driving Laboratories
journal, June 2019


Editorial: special issue on learning from imbalanced data sets
journal, June 2004

  • Chawla, Nitesh V.; Japkowicz, Nathalie; Kotcz, Aleksander
  • ACM SIGKDD Explorations Newsletter, Vol. 6, Issue 1
  • DOI: 10.1145/1007730.1007733

Autonomous discovery in the chemical sciences part I: Progress
journal, September 2019

  • Jensen, Klavs F.; Coley, Connor W.; Eyke, Natalie S.
  • Angewandte Chemie International Edition
  • DOI: 10.1002/anie.201909987

Adaptive Strategies for Materials Design using Uncertainties
journal, January 2016

  • Balachandran, Prasanna V.; Xue, Dezhen; Theiler, James
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep19660

Autonomy in materials research: a case study in carbon nanotube growth
journal, October 2016


Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach
journal, August 2016

  • Gómez-Bombarelli, Rafael; Aguilera-Iparraguirre, Jorge; Hirzel, Timothy D.
  • Nature Materials, Vol. 15, Issue 10
  • DOI: 10.1038/nmat4717

Tracking materials science data lineage to manage millions of materials experiments and analyses
journal, July 2019

  • Soedarmadji, Edwin; Stein, Helge S.; Suram, Santosh K.
  • npj Computational Materials, Vol. 5, Issue 1
  • DOI: 10.1038/s41524-019-0216-x

Active learning across intermetallics to guide discovery of electrocatalysts for CO2 reduction and H2 evolution
journal, September 2018


An autonomous organic reaction search engine for chemical reactivity
journal, June 2017

  • Dragone, Vincenza; Sans, Victor; Henson, Alon B.
  • Nature Communications, Vol. 8, Issue 1
  • DOI: 10.1038/ncomms15733

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
journal, January 2018

  • Gómez-Bombarelli, Rafael; Wei, Jennifer N.; Duvenaud, David
  • ACS Central Science, Vol. 4, Issue 2
  • DOI: 10.1021/acscentsci.7b00572

Discovery of Wall-Selective Carbon Nanotube Growth Conditions via Automated Experimentation
journal, October 2014

  • Nikolaev, Pavel; Hooper, Daylond; Perea-López, Nestor
  • ACS Nano, Vol. 8, Issue 10
  • DOI: 10.1021/nn503347a

Machine learning for molecular and materials science
journal, July 2018


Progress and prospects for accelerating materials science with automated and autonomous workflows
journal, January 2019

  • Stein, Helge S.; Gregoire, John M.
  • Chemical Science, Vol. 10, Issue 42
  • DOI: 10.1039/C9SC03766G

Parallel Electrochemical Treatment System and Application for Identifying Acid-Stable Oxygen Evolution Electrocatalysts
journal, January 2015

  • Jones, Ryan J. R.; Shinde, Aniketa; Guevarra, Dan
  • ACS Combinatorial Science, Vol. 17, Issue 2
  • DOI: 10.1021/co500148p