Uncertainty quantification for neural network potential foundation models

Bilbrey, Jenna A.; Firoz, Jesun S.; Lee, Mal-Soon; Choudhury, Sutanay

doi:10.1038/s41524-025-01572-y

Uncertainty quantification for neural network potential foundation models

Journal Article · Thu Apr 24 00:00:00 EDT 2025 · npj Computational Materials

DOI:https://doi.org/10.1038/s41524-025-01572-y· OSTI ID:2562728

Bilbrey, Jenna A. ^[1]; Firoz, Jesun S. ^[1]; ^[1]; Choudhury, Sutanay ^[1]

Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

View Journal Article

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division (CSGB)

Grant/Contract Number:: AC05-76RL01830

OSTI ID:: 2562728

Alternate ID(s):: OSTI ID: 2563187

Report Number(s):: PNNL-SA--206884

Journal Information:: npj Computational Materials, Journal Name: npj Computational Materials Journal Issue: 1 Vol. 11; ISSN 2057-3960

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United States

Language:: English

References (38)

Uncertainty of exchange‐correlation functionals in density functional theory calculations for lithium‐based solid electrolytes on the case study of lithium phosphorus oxynitride Henkel, Pascal; Mollenhauer, Doreen Journal of Computational Chemistry, Vol. 42, Issue 18 https://doi.org/10.1002/jcc.26546	journal	May 2021
Systematic assessment of various universal machine‐learning interatomic potentials Yu, Haochen; Giantomassi, Matteo; Materzanini, Giuliana Materials Genome Engineering Advances, Vol. 2, Issue 3 https://doi.org/10.1002/mgea.58	journal	July 2024
A survey of uncertainty in deep neural networks Gawlikowski, Jakob; Tassi, Cedrique Rovile Njieutcheu; Ali, Mohsin Artificial Intelligence Review, Vol. 56, Issue S1 https://doi.org/10.1007/s10462-023-10562-9	journal	July 2023
Gradient descent algorithms for quantile regression with smooth approximation Zheng, Songfeng International Journal of Machine Learning and Cybernetics, Vol. 2, Issue 3 https://doi.org/10.1007/s13042-011-0031-2	journal	July 2011
Effects of aleatoric and epistemic errors in reference data on the learnability and quality of NN-based potential energy surfaces Goswami, Sugata; Käser, Silvan; Bemish, Raymond J. Artificial Intelligence Chemistry, Vol. 2, Issue 1 https://doi.org/10.1016/j.aichem.2023.100033	journal	June 2024
A review of uncertainty quantification in deep learning: Techniques, applications and challenges Abdar, Moloud; Pourpanah, Farhad; Hussain, Sadiq Information Fusion, Vol. 76 https://doi.org/10.1016/j.inffus.2021.05.008	journal	December 2021
Quantile Regression Koenker, Roger https://doi.org/10.1017/CBO9780511754098	book	January 2005
Fundamentals of Quantile Regression Koenker, Roger Quantile Regression https://doi.org/10.1017/CBO9780511754098.003	book	January 2005
TorchANI: A Free and Open Source PyTorch-Based Deep Learning Implementation of the ANI Neural Network Potentials Gao, Xiang; Ramezanghorbani, Farhad; Isayev, Olexandr Journal of Chemical Information and Modeling, Vol. 60, Issue 7 https://doi.org/10.1021/acs.jcim.0c00451	journal	June 2020
Ensembles Are Required to Handle Aleatoric and Parametric Uncertainty in Molecular Dynamics Simulation Vassaux, Maxime; Wan, Shunzhou; Edeling, Wouter Journal of Chemical Theory and Computation, Vol. 17, Issue 8 https://doi.org/10.1021/acs.jctc.1c00526	journal	July 2021
Scalable Bayesian Uncertainty Quantification for Neural Network Potentials: Promise and Pitfalls Thaler, Stephan; Doehner, Gregor; Zavadlav, Julija Journal of Chemical Theory and Computation, Vol. 19, Issue 14 https://doi.org/10.1021/acs.jctc.2c01267	journal	April 2023
Adapting OC20-Trained EquiformerV2 Models for High-Entropy Materials Clausen, Christian M.; Rossmeisl, Jan; Ulissi, Zachary W. The Journal of Physical Chemistry C, Vol. 128, Issue 27 https://doi.org/10.1021/acs.jpcc.4c01704	journal	July 2024
Efficient Atomic-Resolution Uncertainty Estimation for Neural Network Potentials Using a Replica Ensemble Jeong, Wonseok; Yoo, Dongsun; Lee, Kyuhyun The Journal of Physical Chemistry Letters, Vol. 11, Issue 15 https://doi.org/10.1021/acs.jpclett.0c01614	journal	June 2020
Performance Assessment of Universal Machine Learning Interatomic Potentials: Challenges and Directions for Materials’ Surfaces Focassio, Bruno; M. Freitas, Luis Paulo; Schleder, Gabriel R. ACS Applied Materials & Interfaces, Vol. 17, Issue 9 https://doi.org/10.1021/acsami.4c03815	journal	July 2024
Open Catalyst 2020 (OC20) Dataset and Community Challenges Chanussot, Lowik; Das, Abhishek; Goyal, Siddharth ACS Catalysis, Vol. 11, Issue 10 https://doi.org/10.1021/acscatal.0c04525	journal	May 2021
Open Challenges in Developing Generalizable Large-Scale Machine-Learning Models for Catalyst Discovery Kolluru, Adeesh; Shuaibi, Muhammed; Palizhati, Aini ACS Catalysis, Vol. 12, Issue 14 https://doi.org/10.1021/acscatal.2c02291	journal	July 2022
The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts Tran, Richard; Lan, Janice; Shuaibi, Muhammed ACS Catalysis, Vol. 13, Issue 5 https://doi.org/10.1021/acscatal.2c05426	journal	February 2023
Evidential Deep Learning for Guided Molecular Property Prediction and Discovery Soleimany, Ava P.; Amini, Alexander; Goldman, Samuel ACS Central Science, Vol. 7, Issue 8 https://doi.org/10.1021/acscentsci.1c00546	journal	July 2021
Uncertainty quantification in molecular simulations with dropout neural network potentials Wen, Mingjian; Tadmor, Ellad B. npj Computational Materials, Vol. 6, Issue 1 https://doi.org/10.1038/s41524-020-00390-8	journal	August 2020
Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles Tan, Aik Rui; Urata, Shingo; Goldman, Samuel npj Computational Materials, Vol. 9, Issue 1 https://doi.org/10.1038/s41524-023-01180-8	journal	December 2023
Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning Allen, Alice E. A.; Lubbers, Nicholas; Matin, Sakib npj Computational Materials, Vol. 10, Issue 1 https://doi.org/10.1038/s41524-024-01339-x	journal	July 2024
Exploring the frontiers of condensed-phase chemistry with a general reactive machine learning potential Zhang, Shuhao; Makoś, Małgorzata Z.; Jadrich, Ryan B. Nature Chemistry https://doi.org/10.1038/s41557-023-01427-3	journal	March 2024
A universal graph deep learning interatomic potential for the periodic table Chen, Chi; Ong, Shyue Ping Nature Computational Science, Vol. 2, Issue 11 https://doi.org/10.1038/s43588-022-00349-3	journal	November 2022
Uncertainty-driven dynamics for active learning of interatomic potentials Kulichenko, Maksim; Barros, Kipton; Lubbers, Nicholas Nature Computational Science, Vol. 3, Issue 3 https://doi.org/10.1038/s43588-023-00406-5	journal	March 2023
ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost Smith, J. S.; Isayev, O.; Roitberg, A. E. Chemical Science, Vol. 8, Issue 4 https://doi.org/10.1039/C6SC05720A	journal	January 2017
Neural network potentials for chemistry: concepts, applications and prospects Käser, Silvan; Vazquez-Salazar, Luis Itza; Meuwly, Markus Digital Discovery, Vol. 2, Issue 1 https://doi.org/10.1039/D2DD00102K	journal	January 2023
Graph neural network interatomic potential ensembles with calibrated aleatoric and epistemic uncertainty on energy and forces Busk, Jonas; Schmidt, Mikkel N.; Winther, Ole Physical Chemistry Chemical Physics, Vol. 25, Issue 37 https://doi.org/10.1039/D3CP02143B	journal	January 2023
Less is more: Sampling chemical space with active learning Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas The Journal of Chemical Physics, Vol. 148, Issue 24 https://doi.org/10.1063/1.5023802	journal	June 2018
Committee neural network potentials control generalization errors and enable active learning Schran, Christoph; Brezina, Krystof; Marsalek, Ondrej The Journal of Chemical Physics, Vol. 153, Issue 10 https://doi.org/10.1063/5.0016004	journal	September 2020
Fast uncertainty estimates in deep learning interatomic potentials Zhu, Albert; Batzner, Simon; Musaelian, Albert The Journal of Chemical Physics, Vol. 158, Issue 16 https://doi.org/10.1063/5.0136574	journal	April 2023
Deep ensembles vs committees for uncertainty estimation in neural-network force fields: Comparison and application to active learning Carrete, Jesús; Montes-Campos, Hadrián; Wanzenböck, Ralf The Journal of Chemical Physics, Vol. 158, Issue 20 https://doi.org/10.1063/5.0146905	journal	May 2023
Uncertainty quantification in classical molecular dynamics Wan, Shunzhou; Sinclair, Robert C.; Coveney, Peter V. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 379, Issue 2197 https://doi.org/10.1098/rsta.2020.0082	journal	March 2021
Quality of uncertainty estimates from neural network potential ensembles Kahle, Leonid; Zipoli, Federico Physical Review E, Vol. 105, Issue 1 https://doi.org/10.1103/PhysRevE.105.015311	journal	January 2022
Modeling high-entropy transition metal alloys with alchemical compression Lopanitsyna, Nataliya; Fraux, Guillaume; Springer, Maximilian A. Physical Review Materials, Vol. 7, Issue 4 https://doi.org/10.1103/PhysRevMaterials.7.045802	journal	April 2023
Machine learning potentials for extended systems: a perspective Behler, Jörg; Csányi, Gábor The European Physical Journal B, Vol. 94, Issue 7 https://doi.org/10.1140/epjb/s10051-021-00156-1	journal	July 2021
Neural Network Potentials: A Concise Overview of Methods Kocer, Emir; Ko, Tsz Wai; Behler, Jörg Annual Review of Physical Chemistry, Vol. 73, Issue 1 https://doi.org/10.1146/annurev-physchem-082720-034254	journal	April 2022
Quantile Regression Koenker, Roger; Hallock, Kevin F. Journal of Economic Perspectives, Vol. 15, Issue 4 https://doi.org/10.1257/jep.15.4.143	journal	November 2001
Uncertainty quantification and propagation in atomistic machine learning Dai, Jin; Adhikari, Santosh; Wen, Mingjian Reviews in Chemical Engineering https://doi.org/10.1515/revce-2024-0028	journal	December 2024

Similar Records

pnnl/SNAP

Software · Mon Mar 24 20:00:00 EDT 2025 · OSTI ID:code-153158

Uncertainty quantification of a deep learning fuel property prediction model

Journal Article · Sat Sep 23 20:00:00 EDT 2023 · Applications in Energy and Combustion Science · OSTI ID:2536669

Related Subjects

36 MATERIALS SCIENCE
97 MATHEMATICS AND COMPUTING
atomistic models
computational methods

Uncertainty quantification for neural network potential foundation models

Citation Formats

References (38)

Similar Records

Related Subjects