Benchmarking the acceleration of materials discovery by sequential learning
Abstract
Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any “good” material, discovery of all “good” materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 comparedmore »
- Authors:
-
- Accelerated Materials Design and Discovery, Toyota Research Institute, Los Altos, USA
- Joint Center for Artificial Photosynthesis, California Institute of Technology, Pasadena, USA
- Joint Center for Artificial Photosynthesis, California Institute of Technology, Pasadena, USA, Division of Engineering and Applied Science
- Publication Date:
- Research Org.:
- California Institute of Technology (CalTech), Pasadena, CA (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES)
- OSTI Identifier:
- 1598712
- Alternate Identifier(s):
- OSTI ID: 1801646
- Grant/Contract Number:
- SC0004993; SC0020383
- Resource Type:
- Published Article
- Journal Name:
- Chemical Science
- Additional Journal Information:
- Journal Name: Chemical Science Journal Volume: 11 Journal Issue: 10; Journal ID: ISSN 2041-6520
- Publisher:
- Royal Society of Chemistry (RSC)
- Country of Publication:
- United Kingdom
- Language:
- English
- Subject:
- 36 MATERIALS SCIENCE
Citation Formats
Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., and Gregoire, John M. Benchmarking the acceleration of materials discovery by sequential learning. United Kingdom: N. p., 2020.
Web. doi:10.1039/C9SC05999G.
Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., & Gregoire, John M. Benchmarking the acceleration of materials discovery by sequential learning. United Kingdom. https://doi.org/10.1039/C9SC05999G
Rohr, Brian, Stein, Helge S., Guevarra, Dan, Wang, Yu, Haber, Joel A., Aykol, Muratahan, Suram, Santosh K., and Gregoire, John M. Wed .
"Benchmarking the acceleration of materials discovery by sequential learning". United Kingdom. https://doi.org/10.1039/C9SC05999G.
@article{osti_1598712,
title = {Benchmarking the acceleration of materials discovery by sequential learning},
author = {Rohr, Brian and Stein, Helge S. and Guevarra, Dan and Wang, Yu and Haber, Joel A. and Aykol, Muratahan and Suram, Santosh K. and Gregoire, John M.},
abstractNote = {Sequential learning (SL) strategies, i.e. iteratively updating a machine learning model to guide experiments, have been proposed to significantly accelerate materials discovery and research. Applications on computational datasets and a handful of optimization experiments have demonstrated the promise of SL, motivating a quantitative evaluation of its ability to accelerate materials discovery, specifically in the case of physical experiments. The benchmarking effort in the present work quantifies the performance of SL algorithms with respect to a breadth of research goals: discovery of any “good” material, discovery of all “good” materials, and discovery of a model that accurately predicts the performance of new materials. To benchmark the effectiveness of different machine learning models against these goals, we use datasets in which the performance of all materials in the search space is known from high-throughput synthesis and electrochemistry experiments. Each dataset contains all pseudo-quaternary metal oxide combinations from a set of six elements (chemical space), the performance metric chosen is the electrocatalytic activity (overpotential) for the oxygen evolution reaction (OER). A diverse set of SL schemes is tested on four chemical spaces, each containing 2121 catalysts. The presented work suggests that research can be accelerated by up to a factor of 20 compared to random acquisition in specific scenarios. The results also show that certain choices of SL models are ill-suited for a given research goal resulting in substantial deceleration compared to random acquisition methods. The results provide quantitative guidance on how to tune an SL strategy for a given research goal and demonstrate the need for a new generation of materials-aware SL algorithms to further accelerate materials discovery.},
doi = {10.1039/C9SC05999G},
journal = {Chemical Science},
number = 10,
volume = 11,
place = {United Kingdom},
year = {Wed Mar 11 00:00:00 EDT 2020},
month = {Wed Mar 11 00:00:00 EDT 2020}
}
https://doi.org/10.1039/C9SC05999G
Web of Science
Works referenced in this record:
Active Learning with Statistical Models
journal, January 1996
- Cohn, D. A.; Ghahramani, Z.; Jordan, M. I.
- Journal of Artificial Intelligence Research, Vol. 4
Functional mapping reveals mechanistic clusters for OER catalysis across (Cu–Mn–Ta–Co–Sn–Fe)O x composition and pH space
journal, January 2019
- Stein, Helge S.; Guevarra, Dan; Shinde, Aniketa
- Materials Horizons, Vol. 6, Issue 6
High-Dimensional Materials and Process Optimization Using Data-Driven Experimental Design with Well-Calibrated Uncertainty Estimates
journal, July 2017
- Ling, Julia; Hutchinson, Maxwell; Antono, Erin
- Integrating Materials and Manufacturing Innovation, Vol. 6, Issue 3
Accelerated Discovery of Large Electrostrains in BaTiO 3 -Based Piezoelectrics Using Active Learning
journal, January 2018
- Yuan, Ruihao; Liu, Zhen; Balachandran, Prasanna V.
- Advanced Materials, Vol. 30, Issue 7
Low-Scaling Algorithm for Nudged Elastic Band Calculations Using a Surrogate Machine Learning Model
journal, April 2019
- Garrido Torres, José A.; Jennings, Paul C.; Hansen, Martin H.
- Physical Review Letters, Vol. 122, Issue 15
Inkjet Printing Assisted Synthesis of Multicomponent Mesoporous Metal Oxides for Ultrafast Catalyst Exploration
journal, October 2012
- Liu, Xiaonao; Shen, Yi; Yang, Ruoting
- Nano Letters, Vol. 12, Issue 11
An Investigation of Thin-Film Ni–Fe Oxide Catalysts for the Electrochemical Evolution of Oxygen
journal, August 2013
- Louie, Mary W.; Bell, Alexis T.
- Journal of the American Chemical Society, Vol. 135, Issue 33
An Operando Investigation of (Ni–Fe–Co–Ce)O x System as Highly Efficient Electrocatalyst for Oxygen Evolution Reaction
journal, January 2017
- Favaro, Marco; Drisdell, Walter S.; Marcus, Matthew A.
- ACS Catalysis, Vol. 7, Issue 2
Phoenics: A Bayesian Optimizer for Chemistry
journal, August 2018
- Häse, Florian; Roch, Loïc M.; Kreisbeck, Christoph
- ACS Central Science, Vol. 4, Issue 9
100% clean and renewable wind, water, and sunlight (WWS) all-sector energy roadmaps for the 50 United States
journal, January 2015
- Jacobson, Mark Z.; Delucchi, Mark A.; Bazouin, Guillaume
- Energy & Environmental Science, Vol. 8, Issue 7
Prediction of Low-Thermal-Conductivity Compounds with First-Principles Anharmonic Lattice-Dynamics Calculations and Bayesian Optimization
journal, November 2015
- Seko, Atsuto; Togo, Atsushi; Hayashi, Hiroyuki
- Physical Review Letters, Vol. 115, Issue 20
Less is more: Sampling chemical space with active learning
journal, June 2018
- Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas
- The Journal of Chemical Physics, Vol. 148, Issue 24
Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics
journal, July 2019
- Vasudevan, Rama K.; Choudhary, Kamal; Mehta, Apurva
- MRS Communications, Vol. 9, Issue 3
Autonomous discovery in the chemical sciences part II: Outlook
journal, September 2019
- Coley, Connor W.; Eyke, Natalie S.; Jensen, Klavs F.
- Angewandte Chemie International Edition
The 2019 materials by design roadmap
journal, October 2018
- Alberi, Kirstin; Nardelli, Marco Buongiorno; Zakutayev, Andriy
- Journal of Physics D: Applied Physics, Vol. 52, Issue 1
Discovery of New Oxygen Evolution Reaction Electrocatalysts by Combinatorial Investigation of the Ni-La-Co-Ce Oxide Composition Space
journal, July 2014
- Haber, Joel A.; Guevarra, Dan; Jung, Suho
- ChemElectroChem, Vol. 1, Issue 10
Next-Generation Experimentation with Self-Driving Laboratories
journal, June 2019
- Häse, Florian; Roch, Loïc M.; Aspuru-Guzik, Alán
- Trends in Chemistry, Vol. 1, Issue 3
Editorial: special issue on learning from imbalanced data sets
journal, June 2004
- Chawla, Nitesh V.; Japkowicz, Nathalie; Kotcz, Aleksander
- ACM SIGKDD Explorations Newsletter, Vol. 6, Issue 1
Autonomous discovery in the chemical sciences part I: Progress
journal, September 2019
- Jensen, Klavs F.; Coley, Connor W.; Eyke, Natalie S.
- Angewandte Chemie International Edition
Adaptive Strategies for Materials Design using Uncertainties
journal, January 2016
- Balachandran, Prasanna V.; Xue, Dezhen; Theiler, James
- Scientific Reports, Vol. 6, Issue 1
Autonomy in materials research: a case study in carbon nanotube growth
journal, October 2016
- Nikolaev, Pavel; Hooper, Daylond; Webber, Frederick
- npj Computational Materials, Vol. 2, Issue 1
Design of efficient molecular organic light-emitting diodes by a high-throughput virtual screening and experimental approach
journal, August 2016
- Gómez-Bombarelli, Rafael; Aguilera-Iparraguirre, Jorge; Hirzel, Timothy D.
- Nature Materials, Vol. 15, Issue 10
Tracking materials science data lineage to manage millions of materials experiments and analyses
journal, July 2019
- Soedarmadji, Edwin; Stein, Helge S.; Suram, Santosh K.
- npj Computational Materials, Vol. 5, Issue 1
Active learning across intermetallics to guide discovery of electrocatalysts for CO2 reduction and H2 evolution
journal, September 2018
- Tran, Kevin; Ulissi, Zachary W.
- Nature Catalysis, Vol. 1, Issue 9
An autonomous organic reaction search engine for chemical reactivity
journal, June 2017
- Dragone, Vincenza; Sans, Victor; Henson, Alon B.
- Nature Communications, Vol. 8, Issue 1
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules
journal, January 2018
- Gómez-Bombarelli, Rafael; Wei, Jennifer N.; Duvenaud, David
- ACS Central Science, Vol. 4, Issue 2
Discovery of Wall-Selective Carbon Nanotube Growth Conditions via Automated Experimentation
journal, October 2014
- Nikolaev, Pavel; Hooper, Daylond; Perea-López, Nestor
- ACS Nano, Vol. 8, Issue 10
Machine learning for molecular and materials science
journal, July 2018
- Butler, Keith T.; Davies, Daniel W.; Cartwright, Hugh
- Nature, Vol. 559, Issue 7715
Progress and prospects for accelerating materials science with automated and autonomous workflows
journal, January 2019
- Stein, Helge S.; Gregoire, John M.
- Chemical Science, Vol. 10, Issue 42
Parallel Electrochemical Treatment System and Application for Identifying Acid-Stable Oxygen Evolution Electrocatalysts
journal, January 2015
- Jones, Ryan J. R.; Shinde, Aniketa; Guevarra, Dan
- ACS Combinatorial Science, Vol. 17, Issue 2