Multifidelity deep operator networks for data-driven and physics-informed problems

Howard, Amanda A.; Perego, Mauro; Karniadakis, George Em; Stinis, Panos

doi:10.1016/j.jcp.2023.112462

Title: Multifidelity deep operator networks for data-driven and physics-informed problems

Journal Article · 01 September 2023 · Journal of Computational Physics

DOI: https://doi.org/10.1016/j.jcp.2023.112462 · OSTI ID:2008415

^[1]; Perego, Mauro ^[2]; Karniadakis, George Em ^[3];

^[1]

Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Brown Univ., Providence, RI (United States)

Operator learning for complex nonlinear systems is increasingly common in modeling multi-physics and multi-scale systems. However, training such high-dimensional operators requires a large amount of expensive, high-fidelity data, either from experiments or simulations. In this work, we present a composite Deep Operator Network (DeepONet) for learning using two datasets with different levels of fidelity to accurately learn complex operators when sufficient high-fidelity data is not available. Additionally, we demonstrate that the presence of low-fidelity data can improve the predictions of physics-informed learning with DeepONets. We demonstrate the new multi-fidelity training in diverse examples, including modeling of the ice-sheet dynamics of the Humboldt glacier, Greenland, using two different fidelity models and also using the same physical model at two different resolutions.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States); Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)

Grant/Contract Number:: AC05-76RL01830; NA0003525

OSTI ID:: 2008415

Report Number(s):: PNNL-SA--172145

Journal Information:: Journal of Computational Physics, Journal Name: Journal of Computational Physics Vol. 493; ISSN 0021-9991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (43)

Optimal initial conditions for coupling ice sheet models to Earth system models: PEREGO ET AL. Perego, Mauro; Price, Stephen; Stadler, Georg Journal of Geophysical Research: Earth Surface, Vol. 119, Issue 9 https://doi.org/10.1002/2014JF003181	journal	September 2014
A hybrid neural network-first principles approach to process modeling Psichogios, Dimitris C.; Ungar, Lyle H. AIChE Journal, Vol. 38, Issue 10 https://doi.org/10.1002/aic.690381003	journal	October 1992
Exponential Time Differencing for Stiff Systems Cox, S. M.; Matthews, P. C. Journal of Computational Physics, Vol. 176, Issue 2 https://doi.org/10.1006/jcph.2002.6995	journal	March 2002
Bi-fidelity modeling of uncertain and partially unknown systems using DeepONets De, Subhayan; Reynolds, Matthew; Hassanaly, Malik Computational Mechanics https://doi.org/10.1007/s00466-023-02272-4	journal	March 2023
Improved Architectures and Training Algorithms for Deep Operator Networks Wang, Sifan; Wang, Hanwen; Perdikaris, Paris Journal of Scientific Computing, Vol. 92, Issue 2 https://doi.org/10.1007/s10915-022-01881-0	journal	June 2022
Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data Sun, Luning; Gao, Han; Pan, Shaowu Computer Methods in Applied Mechanics and Engineering, Vol. 361 https://doi.org/10.1016/j.cma.2019.112732	journal	April 2020
Multi-fidelity deep neural network surrogate model for aerodynamic shape optimization Zhang, Xinshuai; Xie, Fangfang; Ji, Tingwei Computer Methods in Applied Mechanics and Engineering, Vol. 373 https://doi.org/10.1016/j.cma.2020.113485	journal	January 2021
A fast multi-fidelity method with uncertainty quantification for complex data correlations: Application to vortex-induced vibrations of marine risers Meng, Xuhui; Wang, Zhicheng; Fan, Dixia Computer Methods in Applied Mechanics and Engineering, Vol. 386 https://doi.org/10.1016/j.cma.2021.114212	journal	December 2021
A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials Goswami, Somdatta; Yin, Minglang; Yu, Yue Computer Methods in Applied Mechanics and Engineering, Vol. 391 https://doi.org/10.1016/j.cma.2022.114587	journal	March 2022
A multifidelity deep operator network approach to closure for multiscale systems Ahmed, Shady E.; Stinis, Panos Computer Methods in Applied Mechanics and Engineering, Vol. 414 https://doi.org/10.1016/j.cma.2023.116161	journal	September 2023
Data-driven and physics-informed deep learning operators for solution of heat conduction equation with parametric heat source Koric, Seid; Abueidda, Diab W. International Journal of Heat and Mass Transfer, Vol. 203 https://doi.org/10.1016/j.ijheatmasstransfer.2022.123809	journal	April 2023
DGM: A deep learning algorithm for solving partial differential equations Sirignano, Justin; Spiliopoulos, Konstantinos Journal of Computational Physics, Vol. 375 https://doi.org/10.1016/j.jcp.2018.08.029	journal	December 2018
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Raissi, M.; Perdikaris, P.; Karniadakis, G. E. Journal of Computational Physics, Vol. 378 https://doi.org/10.1016/j.jcp.2018.10.045	journal	February 2019
A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems Meng, Xuhui; Karniadakis, George Em Journal of Computational Physics, Vol. 401 https://doi.org/10.1016/j.jcp.2019.109020	journal	January 2020
Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks Karumuri, Sharmila; Tripathy, Rohit; Bilionis, Ilias Journal of Computational Physics, Vol. 404 https://doi.org/10.1016/j.jcp.2019.109120	journal	March 2020
Transfer learning based multi-fidelity physics informed deep neural network Chakraborty, Souvik Journal of Computational Physics, Vol. 426 https://doi.org/10.1016/j.jcp.2020.109942	journal	February 2021
Multifidelity modeling for Physics-Informed Neural Networks (PINNs) Penwarden, Michael; Zhe, Shandian; Narayan, Akil Journal of Computational Physics, Vol. 451 https://doi.org/10.1016/j.jcp.2021.110844	journal	February 2022
Neural network training using ℓ1-regularization and bi-fidelity data De, Subhayan; Doostan, Alireza Journal of Computational Physics, Vol. 458 https://doi.org/10.1016/j.jcp.2022.111010	journal	June 2022
Nonlocal kernel network (NKN): A stable and resolution-independent deep neural network You, Huaiqian; Yu, Yue; D'Elia, Marta Journal of Computational Physics, Vol. 469 https://doi.org/10.1016/j.jcp.2022.111536	journal	November 2022
Self-adaptive physics-informed neural networks McClenny, Levi D.; Braga-Neto, Ulisses M. Journal of Computational Physics, Vol. 474 https://doi.org/10.1016/j.jcp.2022.111722	journal	February 2023
Long-time integration of parametric evolution equations with physics-informed DeepONets Wang, Sifan; Perdikaris, Paris Journal of Computational Physics, Vol. 475 https://doi.org/10.1016/j.jcp.2022.111855	journal	February 2023
A hybrid deep neural operator/finite element method for ice-sheet modeling He, QiZhi; Perego, Mauro; Howard, Amanda A. Journal of Computational Physics, Vol. 492 https://doi.org/10.1016/j.jcp.2023.112428	journal	November 2023
Deep learning of inverse water waves problems using multi-fidelity data: Application to Serre–Green–Naghdi equations Jagtap, Ameya D.; Mitsotakis, Dimitrios; Karniadakis, George Em Ocean Engineering, Vol. 248 https://doi.org/10.1016/j.oceaneng.2022.110775	journal	March 2022
A seamless multiscale operator neural network for inferring bubble dynamics Lin, Chensen; Maxey, Martin; Li, Zhen Journal of Fluid Mechanics, Vol. 929 https://doi.org/10.1017/jfm.2021.866	journal	October 2021
A Multifidelity Framework and Uncertainty Quantification for Sea Surface Temperature in the Massachusetts and Cape Cod Bays Babaee, H.; Bastidas, C.; DeFilippo, M. Earth and Space Science, Vol. 7, Issue 2 https://doi.org/10.1029/2019EA000954	journal	February 2020
On the dynamics of the ice sheets 2 Halfar, P. Journal of Geophysical Research, Vol. 88, Issue C10 https://doi.org/10.1029/JC088iC10p06043	journal	January 1983
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu, Lu; Jin, Pengzhan; Pang, Guofei Nature Machine Intelligence, Vol. 3, Issue 3 https://doi.org/10.1038/s42256-021-00302-5	journal	March 2021
Extraction of mechanical properties of materials through deep learning from instrumented indentation Lu, Lu; Dao, Ming; Kumar, Punit Proceedings of the National Academy of Sciences, Vol. 117, Issue 13 https://doi.org/10.1073/pnas.1922210117	journal	March 2020
Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport Lu, Lu; Pestourie, Raphaël; Johnson, Steven G. Physical Review Research, Vol. 4, Issue 2 https://doi.org/10.1103/PhysRevResearch.4.023210	journal	June 2022
Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks IEEE Transactions on Neural Networks, Vol. 6, Issue 4 https://doi.org/10.1109/72.392252	journal	July 1995
Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems IEEE Transactions on Neural Networks, Vol. 6, Issue 4 https://doi.org/10.1109/72.392253	journal	July 1995
Artificial neural networks for solving ordinary and partial differential equations Lagaris, I. E.; Likas, A.; Fotiadis, D. I. IEEE Transactions on Neural Networks, Vol. 9, Issue 5 https://doi.org/10.1109/72.712178	journal	January 1998
Multi-Fidelity Physics-Constrained Neural Network and Its Application in Materials Modeling Liu, Dehao; Wang, Yan Journal of Mechanical Design, Vol. 141, Issue 12 https://doi.org/10.1115/1.4044400	journal	September 2019
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets Wang, Sifan; Wang, Hanwen; Perdikaris, Paris Science Advances, Vol. 7, Issue 40 https://doi.org/10.1126/sciadv.abi8605	journal	October 2021
A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems Petra, Noemi; Martin, James; Stadler, Georg SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/130934805	journal	January 2014
Universal Approximation of Multiple Nonlinear Operators by Neural Networks Back, Andrew D.; Chen, Tianping Neural Computation, Vol. 14, Issue 11 https://doi.org/10.1162/089976602760407964	journal	November 2002
On Transfer Learning of Neural Networks Using Bi-Fidelity data for Uncertainty Propagation De, Subhayan; Britton, Jolene; Reynolds, Matthew International Journal for Uncertainty Quantification, Vol. 10, Issue 6 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2020033267	journal	January 2020
Consistent approximations and boundary conditions for ice-sheet dynamics from a principle of least action Dukowicz, John K.; Price, Stephen F.; Lipscomb, William H. Journal of Glaciology, Vol. 56, Issue 197 https://doi.org/10.3189/002214310792447851	journal	January 2010
Parallel finite-element implementation for higher-order ice-sheet models Perego, Mauro; Gunzburger, Max; Burkardt, John Journal of Glaciology, Vol. 58, Issue 207 https://doi.org/10.3189/2012JoG11J063	journal	January 2012
Steady Motion of Ice Sheets Morland, L. W.; Johnson, I. R. Journal of Glaciology, Vol. 25, Issue 92 https://doi.org/10.3189/S0022143000010467	journal	January 1980
A new vertically integrated MOno-Layer Higher-Order (MOLHO) ice flow model Dias dos Santos, Thiago; Morlighem, Mathieu; Brinkerhoff, Douglas The Cryosphere, Vol. 16, Issue 1 https://doi.org/10.5194/tc-16-179-2022	journal	January 2022
The contribution of Humboldt Glacier, northern Greenland, to sea-level rise through 2100 constrained by recent observations of speedup and retreat Hillebrand, Trevor R.; Hoffman, Matthew J.; Perego, Mauro The Cryosphere, Vol. 16, Issue 11 https://doi.org/10.5194/tc-16-4679-2022	journal	November 2022
A comparison of the stability and performance of depth-integrated ice-dynamics solvers Robinson, Alexander; Goldberg, Daniel; Lipscomb, William H. The Cryosphere, Vol. 16, Issue 2 https://doi.org/10.5194/tc-16-689-2022	journal	February 2022

Similar Records

On the Training and Generalization of Deep Operator Networks

Journal Article · 2024 · SIAM Journal on Scientific Computing · OSTI ID:2571744

Lee, Sanghyun; Shin, Yeonjong

A hybrid deep neural operator/finite element method for ice-sheet modeling

Journal Article · 2023 · Journal of Computational Physics · OSTI ID:1998151

He, QiZhi; Perego, Mauro; Howard, Amanda A.; +2 more

Fed-DeepONet: Stochastic Gradient-Based Federated Training of Deep Operator Networks

Journal Article · 2022 · Algorithms · OSTI ID:1886960

Moya, Christian; Lin, Guang

Related Subjects

97 MATHEMATICS AND COMPUTING
ice-sheet dynamics
multifidelity
neural operator
operator learning
physics-informed machine learning

Title: Multifidelity deep operator networks for data-driven and physics-informed problems

Citation Formats

References (43)

Similar Records

Related Subjects