A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems
Abstract
Presently the training of neural networks relies on data of comparable accuracy but in real applications only a very small set of highfidelity data is available while inexpensive lower fidelity data may be plentiful. We propose a new composite neural network (NN) that can be trained based on multifidelity data. It is comprised of three NNs, with the first NN trained using the lowfidelity data and coupled to two highfidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the lowfidelity and the highfidelity data. We first demonstrate the accuracy of the new multifidelity NN for approximating some standard benchmark functions but also a 20dimensional function that is not easy to approximate with other methods, e.g. Gaussian process regression. Subsequently, we extend the recently developed physicsinformed neural networks (PINNs) to be trained with multifidelity data sets (MPINNs). MPINNs contain four fullyconnected neural networks, where the first one approximates the lowfidelity data, while the second and third construct the correlation between the low and highfidelity data and produce the multifidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, by decomposingmore »
 Authors:

 Brown Univ., Providence, RI (United States)
 Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Brown Univ., Providence, RI (United States)
 Publication Date:
 Research Org.:
 Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
 Sponsoring Org.:
 USDOE
 OSTI Identifier:
 1598680
 Grant/Contract Number:
 SC0019453; SC0019434; FA95501710013; HR00111990025
 Resource Type:
 Journal Article: Accepted Manuscript
 Journal Name:
 Journal of Computational Physics
 Additional Journal Information:
 Journal Volume: 401; Journal Issue: C; Journal ID: ISSN 00219991
 Publisher:
 Elsevier
 Country of Publication:
 United States
 Language:
 English
 Subject:
 97 MATHEMATICS AND COMPUTING; 71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; Multifidelity; Physicsinformed neural networks; Adversarial data; Porous media; Reactive transport
Citation Formats
Meng, Xuhui, and Karniadakis, George Em. A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems. United States: N. p., 2019.
Web. doi:10.1016/j.jcp.2019.109020.
Meng, Xuhui, & Karniadakis, George Em. A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems. United States. doi:10.1016/j.jcp.2019.109020.
Meng, Xuhui, and Karniadakis, George Em. Fri .
"A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems". United States. doi:10.1016/j.jcp.2019.109020. https://www.osti.gov/servlets/purl/1598680.
@article{osti_1598680,
title = {A composite neural network that learns from multifidelity data: Application to function approximation and inverse PDE problems},
author = {Meng, Xuhui and Karniadakis, George Em},
abstractNote = {Presently the training of neural networks relies on data of comparable accuracy but in real applications only a very small set of highfidelity data is available while inexpensive lower fidelity data may be plentiful. We propose a new composite neural network (NN) that can be trained based on multifidelity data. It is comprised of three NNs, with the first NN trained using the lowfidelity data and coupled to two highfidelity NNs, one with activation functions and another one without, in order to discover and exploit nonlinear and linear correlations, respectively, between the lowfidelity and the highfidelity data. We first demonstrate the accuracy of the new multifidelity NN for approximating some standard benchmark functions but also a 20dimensional function that is not easy to approximate with other methods, e.g. Gaussian process regression. Subsequently, we extend the recently developed physicsinformed neural networks (PINNs) to be trained with multifidelity data sets (MPINNs). MPINNs contain four fullyconnected neural networks, where the first one approximates the lowfidelity data, while the second and third construct the correlation between the low and highfidelity data and produce the multifidelity approximation, which is then used in the last NN that encodes the partial differential equations (PDEs). Specifically, by decomposing the correlation into a linear and nonlinear part, the present model is capable of learning both the linear and complex nonlinear correlations between the low and highfidelity data adaptively. By training the MPINNs, we can: (1) obtain the correlation between the low and highfidelity data, (2) infer the quantities of interest based on a few scattered data, and (3) identify the unknown parameters in the PDEs. In particular, we employ the MPINNs to learn the hydraulic conductivity field for unsaturated flows as well as the reactive models for reactive transport. The results demonstrate that MPINNs can achieve relatively high accuracy based on a very small set of highfidelity data. Despite the relatively low dimension and limited number of fidelities (twofidelity levels) for the benchmark problems in the present study, the proposed model can be readily extended to very highdimensional regression and classification problems involving multifidelity data.},
doi = {10.1016/j.jcp.2019.109020},
journal = {Journal of Computational Physics},
issn = {00219991},
number = C,
volume = 401,
place = {United States},
year = {2019},
month = {10}
}
Web of Science