Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Uncertainty quantification of graph convolution neural network models of evolving processes

Journal Article · · Computer Methods in Applied Mechanics and Engineering
 [1];  [2];  [3];  [4];  [2]
  1. University of Michigan, Ann Arbor, MI (United States); University of Michigan
  2. Sandia National Laboratories (SNL-CA), Livermore, CA (United States)
  3. University of Michigan, Ann Arbor, MI (United States)
  4. Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
+ Show Author Affiliations
The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.
Research Organization:
University of Michigan, Ann Arbor, MI (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
Grant/Contract Number:
NA0003525; SC0021397
OSTI ID:
2440697
Journal Information:
Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Vol. 429; ISSN 0045-7825
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English

References (35)

Deep learning and multi-level featurization of graph representations of microstructural data journal March 2023
Comparison of Bayesian predictive methods for model selection journal April 2016
Hybrid Monte Carlo journal September 1987
Computational crystal plasticity journal January 2000
Diffusion maps journal July 2006
Overview of constitutive laws, kinematics, homogenization and multiscale methods in crystal plasticity finite-element modeling: Theory, experiments, applications journal February 2010
Sobolev training of thermodynamic-informed neural networks for interpretable elasto-plasticity models with level set hardening journal April 2021
Geometric learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity journal February 2023
Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning journal November 2023
Predicting the mechanical response of oligocrystals with deep learning journal November 2019
Data-driven probability concentration and sampling on manifold journal September 2016
Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification journal August 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations journal February 2019
Phase-field simulations of intergranular fission gas bubble behavior in U3Si2 nuclear fuel journal December 2020
An Introduction to MCMC for Machine Learning journal January 2003
Explicit reversible integrators for extended systems dynamics journal April 1996
Bayes Factors journal June 1995
Variational Inference: A Review for Statisticians journal July 2016
Numerical solution of the Navier-Stokes equations journal January 1968
Bayesian inference in physics journal September 2011
Bi-Directional ConvLSTM U-Net with Densley Connected Convolutions conference October 2019
Graph Neural Networks in TensorFlow and Keras with Spektral [Application Notes] journal February 2021
Advances in Variational Inference journal August 2019
Laws for Work-Hardening and Low-Temperature Creep journal January 1976
On the Construction and Comparison of Difference Schemes journal September 1968
Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions journal January 2011
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation journal June 2003
A Practical Bayesian Framework for Backpropagation Networks journal May 1992
Long Short-Term Memory journal November 1997
Modeling mesoscale fission gas behavior in UO2 by directly coupling the phase field method to spatially resolved cluster dynamics journal January 2022
Measuring and testing dependence by correlation of distances journal December 2007
Variational Hamiltonian Monte Carlo via Score Matching journal June 2018
Self-Attention ConvLSTM for Spatiotemporal Prediction journal April 2020
Mesh-Based Graph Convolutional Neural Networks for Modeling Materials with Microstructure journal January 2022
Hybrid deterministic-stochastic gradient Langevin dynamics for Bayesian learning journal January 2012

Figures / Tables (24)

Figure 1(p. 7)figure Figure 1
Table 1(p. 8)table Table 1
Figure 2(p. 9)figure Figure 2
Figure 3(p. 9)figure Figure 3
Figure 4(p. 10)figure Figure 4
Table 2(p. 10)table Table 2
Figure 5(p. 11)figure Figure 5
Figure 6(p. 11)figure Figure 6
Figure 7(p. 13)figure Figure 7
Figure 8(p. 14)figure Figure 8
Figure 9(p. 15)figure Figure 9
Table 3(p. 15)table Table 3
Table 4(p. 15)table Table 4
Figure 10(p. 16)figure Figure 10
Figure 11(p. 16)figure Figure 11
Figure 12(p. 17)figure Figure 12
Figure 13(p. 18)figure Figure 13
Figure 14(p. 19)figure Figure 14
Figure 15(p. 20)figure Figure 15
Figure 16(p. 21)figure Figure 16
Figure 17(p. 22)figure Figure 17
Figure 18(p. 22)figure Figure 18
Figure 19(p. 22)figure Figure 19
Figure 20(p. 23)figure Figure 20

Similar Records

General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification
Journal Article · Wed Jan 05 19:00:00 EST 2022 · Frontiers in Artificial Intelligence · OSTI ID:2953825
Randomized physics-informed neural networks for Bayesian data assimilation
Journal Article · Tue Jan 07 19:00:00 EST 2025 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:2497874

Related Subjects

36 MATERIALS SCIENCE
97 MATHEMATICS AND COMPUTING
Neural networks
Neural ordinary differential equations
Recurrent networks
Stein variational gradient descent
Uncertainty quantification