Surrogate construction via weight parameterization of residual neural networks
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Surrogate model development is a critical step for uncertainty quantification or other sample-intensive tasks for complex computational models. Here, in this work, we develop a multi-output surrogate form using a class of neural networks (NNs) that employ shortcut connections, namely Residual NNs (ResNets). ResNets are known to regularize the surrogate learning problem and improve the efficiency and accuracy of the resulting surrogate. Inspired by the continuous, Neural ODE analogy, we augment ResNets with weight parameterization strategy with respect to ResNet depth. Weight-parameterized ResNets regularize the NN surrogate learning problem and allow better generalization with a drastically reduced number of learnable parameters. We demonstrate that weight-parameterized ResNets are more accurate and efficient than conventional feed-forward multi-layer perceptron networks. We also compare various options for parameterization of the weights as functions of ResNet depth. We demonstrate the results on both synthetic examples and a large scale earth system model of interest.
- Research Organization:
- Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program
- Grant/Contract Number:
- NA0003525
- OSTI ID:
- 2476624
- Report Number(s):
- SAND--2024-15148J
- Journal Information:
- Computer Methods in Applied Mechanics and Engineering, Journal Name: Computer Methods in Applied Mechanics and Engineering Journal Issue: Part A Vol. 433; ISSN 0045-7825
- Publisher:
- ElsevierCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Layer-Parallel Training of Deep Residual Neural Networks
On the influence of over-parameterization in manifold based surrogates and deep neural operators