Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification

Journal Article · · Machine Learning: Science and Technology
 [1];  [1];  [1];  [1]
  1. Univ. of Michigan, Ann Arbor, MI (United States)
+ Show Author Affiliations

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

Research Organization:
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF); National Science Foundation (NSF)
Grant/Contract Number:
AC02-05CH11231
OSTI ID:
2422892
Journal Information:
Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 2 Vol. 4; ISSN 2632-2153
Publisher:
IOP PublishingCopyright Statement
Country of Publication:
United States
Language:
English

References (49)

Multivariate Density Estimation book August 1992
Searching for an Optimal Multi‐Metallic Alloy Catalyst by Active Learning Combined with Experiments journal March 2022
Development and Recent Progress on Ammonia Synthesis Catalysts for Haber–Bosch Process journal December 2020
Machine learning for heterogeneous catalyst design and discovery journal May 2018
Bayesian Optimization of High‐Entropy Alloy Compositions for Electrocatalytic Oxygen Reduction** journal October 2021
Machine Learning for Computational Heterogeneous Catalysis journal June 2019
Ensemble Methods in Machine Learning book January 2000
Active Learning book August 2012
Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods journal March 2021
Progress in catalytic naphtha reforming process: A review journal September 2013
Response surface methodology using Gaussian processes: Towards optimizing the trans-stilbene epoxidation over Co2+–NaX catalysts journal January 2010
Leveraging Uncertainty in Machine Learning Accelerates Biological Discovery and Design journal November 2020
Bayesian neural networks for uncertainty quantification in data-driven materials modeling journal December 2021
Bayesian statistics in catalysis: a perspective journal June 2022
Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation journal February 2020
Thermal/catalytic cracking of liquid hydrocarbons for the production of olefins: A state-of-the-art review II: Catalytic cracking review journal June 2016
A review of uncertainty quantification in deep learning: Techniques, applications and challenges journal December 2021
Recent progress on earth abundant electrocatalysts for hydrogen evolution reaction (HER) in alkaline medium to achieve efficient water splitting – A review journal July 2019
Five High-Impact Research Areas in Machine Learning for Materials Science journal December 2019
Model-Specific to Model-General Uncertainty for Physical Properties journal February 2022
Uncertainty-Quantified Hybrid Machine Learning/Density Functional Theory High Throughput Screening Method for Crystals journal March 2020
Automated Discovery and Construction of Surface Phase Diagrams Using Machine Learning journal September 2016
Open Catalyst 2020 (OC20) Dataset and Community Challenges journal May 2021
Open Challenges in Developing Generalizable Large-Scale Machine-Learning Models for Catalyst Discovery journal July 2022
Evidential Deep Learning for Guided Molecular Property Prediction and Discovery journal July 2021
Machine Learning in Catalysis, From Proposal to Practicing journal December 2019
A Bayesian framework for adsorption energy prediction on bimetallic alloy catalysts journal November 2020
Accelerated discovery of CO2 electrocatalysts using active machine learning journal May 2020
Active learning across intermetallics to guide discovery of electrocatalysts for CO2 reduction and H2 evolution journal September 2018
A quantitative uncertainty metric controls error in neural network-driven chemical discovery journal January 2019
Catlas: an automated framework for catalyst discovery demonstrated for direct syngas conversion journal January 2022
SchNet – A deep learning architecture for molecules and materials journal June 2018
OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features journal September 2020
The long road to calibrated prediction uncertainty in computational chemistry journal March 2022
Screening of bimetallic electrocatalysts for water purification with machine learning journal August 2022
From DFT to machine learning: recent approaches to materials science–a review journal May 2019
Methods for comparing uncertainty quantifications for material property predictions journal May 2020
Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials journal December 2022
An algorithm with guaranteed convergence for finding a zero of a function journal April 1971
Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties journal April 2018
Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Control conference May 2020
When Gaussian Process Meets Big Data: A Review of Scalable GPs journal November 2020
Probabilistic forecasts, calibration and sharpness journal April 2007
Probabilistic Forecasting journal January 2014
Trends in the Exchange Current for Hydrogen Evolution journal January 2005
Strictly Proper Scoring Rules, Prediction, and Estimation journal March 2007
Evaluating and Calibrating Uncertainty Prediction in Regression Tasks journal July 2022
Probabilities for SV Machines book September 2000
Dataset Shift in Machine Learning book January 2008

Similar Records

Related Subjects

36 MATERIALS SCIENCE
97 MATHEMATICS AND COMPUTING
Computer Science
Science & Technology - Other Topics
calibration
computational catalysis
crystal graph convolutional neural networks
evidential regression
recalibration