Abstract
In this work, we detail two uncertainty quantification (UQ) methods that provide complementary information. Readout
ensembling, by finetuning only the readout layers of an ensemble of foundation models, provides information about model
uncertainty. Amending the final readout layer to predict upper and lower quantiles replaces point predictions with
distributional predictions, which provide information about uncertainty within the underlying training data. We
demonstrate our approach with the MACE-MP-0 model, applying UQ to both the foundation model and a series of finetuned
models. The uncertainties produced by the ensemble and quantile methods are demonstrated to be distinct measures by
which the quality of the NNP output can be judged.
- Developers:
-
Pope, Jenna (Bilbrey) [1] ; Choudhury, Sutanay [1] ; Firoz, Jesun [1]
- Pacific Northwest National Laboratory
- Release Date:
- 2025-03-25
- Project Type:
- Open Source, Publicly Available Repository
- Software Type:
- Scientific
- Licenses:
-
BSD 2-clause "Simplified" License
- Sponsoring Org.:
-
US DOEPrimary Award/Contract Number:AC05-76RL01830
- Code ID:
- 153158
- Site Accession Number:
- Battelle IPID 33288-E
- Research Org.:
- Pacific Northwest National Laboratory
- Country of Origin:
- United States
Citation Formats
Pope, Jenna (Bilbrey), Choudhury, Sutanay, and Firoz, Jesun.
pnnl/SNAP.
Computer Software.
https://github.com/pnnl/snap.
US DOE.
25 Mar. 2025.
Web.
doi:10.11578/dc.20250325.2.
Pope, Jenna (Bilbrey), Choudhury, Sutanay, & Firoz, Jesun.
(2025, March 25).
pnnl/SNAP.
[Computer software].
https://github.com/pnnl/snap.
https://doi.org/10.11578/dc.20250325.2.
Pope, Jenna (Bilbrey), Choudhury, Sutanay, and Firoz, Jesun.
"pnnl/SNAP." Computer software.
March 25, 2025.
https://github.com/pnnl/snap.
https://doi.org/10.11578/dc.20250325.2.
@misc{
doecode_153158,
title = {pnnl/SNAP},
author = {Pope, Jenna (Bilbrey) and Choudhury, Sutanay and Firoz, Jesun},
abstractNote = {In this work, we detail two uncertainty quantification (UQ) methods that provide complementary information. Readout
ensembling, by finetuning only the readout layers of an ensemble of foundation models, provides information about model
uncertainty. Amending the final readout layer to predict upper and lower quantiles replaces point predictions with
distributional predictions, which provide information about uncertainty within the underlying training data. We
demonstrate our approach with the MACE-MP-0 model, applying UQ to both the foundation model and a series of finetuned
models. The uncertainties produced by the ensemble and quantile methods are demonstrated to be distinct measures by
which the quality of the NNP output can be judged.},
doi = {10.11578/dc.20250325.2},
url = {https://doi.org/10.11578/dc.20250325.2},
howpublished = {[Computer Software] \url{https://doi.org/10.11578/dc.20250325.2}},
year = {2025},
month = {mar}
}