Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems

Tartakovsky, A. M.; Ortiz Marrero, C.; Perdikaris, Paris; Tartakovsky, G. D.; Barajas‐Solano, D.

doi:10.1029/2019wr026731

Title: Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems

Abstract

In this paper, we present a physics informed deep neural network (DNN) method for estimating parameters and unknown physics (constitutive relationships) in partial differential equation (PDE) models. We use PDEs in addition to measurements to train DNNs to approximate unknown parameters and constitutive relationships as well as states. The proposed approach increases the accuracy of DNN approximations of partially known functions when a limited number of measurements is available and allows for training DNNs when no direct measurements of the functions of interest are available. We employ physics informed DNNs to estimate the unknown space-dependent diffusion coefficient in a linear diffusion equation and an unknown constitutive relationship in a non-linear diffusion equation. For the parameter estimation problem, we assume that partial measurements of the coefficient and states are available and demonstrate that under these conditions, the proposed method is more accurate than state-of-the-art methods. For the non-linear diffusion PDE model with a fully unknown constitutive relationship (i.e., no measurements of constitutive relationship are available), the physics informed DNN method can accurately estimate the non-linear constitutive relationship based on state measurements only. Finally, we demonstrate that the proposed method remains accurate in the presence of measurement noise.

Authors:

^[1];

^[2]; Tartakovsky, G. D. ^[1];

^[1]

Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Univ. of Pennsylvania, Philadelphia, PA (United States)

Publication Date:: Sat Apr 04 00:00:00 EDT 2020

Research Org.:: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1639036

Report Number(s):: PNNL-SA-137164
Journal ID: ISSN 0043-1397

Grant/Contract Number:: AC05-76RL01830

Resource Type:: Accepted Manuscript

Journal Name:: Water Resources Research

Additional Journal Information:: Journal Volume: 56; Journal Issue: 5; Journal ID: ISSN 0043-1397

Publisher:: American Geophysical Union (AGU)

Country of Publication:: United States

Language:: English

Subject:: 42 ENGINEERING; machine learning, physics informed ML, Parameter Estimation, model discovery, Model Learning

Citation Formats


                    Tartakovsky, A. M., Ortiz Marrero, C., Perdikaris, Paris, Tartakovsky, G. D., and Barajas‐Solano, D. Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems.  United States: N. p., 2020. 
Web.  doi:10.1029/2019wr026731.

Copy to clipboard


                    Tartakovsky, A. M., Ortiz Marrero, C., Perdikaris, Paris, Tartakovsky, G. D., & Barajas‐Solano, D. Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems.  United States.  https://doi.org/10.1029/2019wr026731

Copy to clipboard


                    Tartakovsky, A. M., Ortiz Marrero, C., Perdikaris, Paris, Tartakovsky, G. D., and Barajas‐Solano, D. Sat .  
"Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems".  United States.  https://doi.org/10.1029/2019wr026731.  https://www.osti.gov/servlets/purl/1639036.

Copy to clipboard


                    
@article{osti_1639036,

  title        = {Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems},

  author       = {Tartakovsky, A. M. and Ortiz Marrero, C. and Perdikaris, Paris and Tartakovsky, G. D. and Barajas‐Solano, D.},

  abstractNote = {In this paper, we present a physics informed deep neural network (DNN) method for estimating parameters and unknown physics (constitutive relationships) in partial differential equation (PDE) models. We use PDEs in addition to measurements to train DNNs to approximate unknown parameters and constitutive relationships as well as states. The proposed approach increases the accuracy of DNN approximations of partially known functions when a limited number of measurements is available and allows for training DNNs when no direct measurements of the functions of interest are available. We employ physics informed DNNs to estimate the unknown space-dependent diffusion coefficient in a linear diffusion equation and an unknown constitutive relationship in a non-linear diffusion equation. For the parameter estimation problem, we assume that partial measurements of the coefficient and states are available and demonstrate that under these conditions, the proposed method is more accurate than state-of-the-art methods. For the non-linear diffusion PDE model with a fully unknown constitutive relationship (i.e., no measurements of constitutive relationship are available), the physics informed DNN method can accurately estimate the non-linear constitutive relationship based on state measurements only. Finally, we demonstrate that the proposed method remains accurate in the presence of measurement noise.},

  doi          = {10.1029/2019wr026731},

  journal      = {Water Resources Research},

  number       = 5,

  volume       = 56,

  place        = {United States},

  year         = {Sat Apr 04 00:00:00 EDT 2020},

  month        = {Sat Apr 04 00:00:00 EDT 2020}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1029/2019wr026731

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 89 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Inverse problem in hydrogeology
journal, February 2005

Carrera, Jes�s; Alcolea, Andr�s; Medina, Agust�n
Hydrogeology Journal, Vol. 13, Issue 1
DOI: 10.1007/s10040-004-0404-7

A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
journal, June 2015

Williams, Matthew O.; Kevrekidis, Ioannis G.; Rowley, Clarence W.
Journal of Nonlinear Science, Vol. 25, Issue 6
DOI: 10.1007/s00332-015-9258-5

Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
journal, October 2019

Zhu, Yinhao; Zabaras, Nicholas; Koutsourelakis, Phaedon-Stelios
Journal of Computational Physics, Vol. 394
DOI: 10.1016/j.jcp.2019.05.024

Modeling fluid flow and transport in variably saturated porous media with the STOMP simulator. 1. Nonvolatile three-phase model description
journal, January 1995

White, M. D.; Oostrom, M.; Lenhard, R. J.
Advances in Water Resources, Vol. 18, Issue 6
DOI: 10.1016/0309-1708(95)00018-E

Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification
journal, August 2018

Zhu, Yinhao; Zabaras, Nicholas
Journal of Computational Physics, Vol. 366
DOI: 10.1016/j.jcp.2018.04.018

A Limited Memory Algorithm for Bound Constrained Optimization
journal, September 1995

Byrd, Richard H.; Lu, Peihuang; Nocedal, Jorge
SIAM Journal on Scientific Computing, Vol. 16, Issue 5
DOI: 10.1137/0916069

VAMPnets for deep learning of molecular kinetics
journal, January 2018

Mardt, Andreas; Pasquali, Luca; Wu, Hao
Nature Communications, Vol. 9, Issue 1
DOI: 10.1038/s41467-017-02388-1

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019

Raissi, M.; Perdikaris, P.; Karniadakis, G. E.
Journal of Computational Physics, Vol. 378
DOI: 10.1016/j.jcp.2018.10.045

Representations of non-linear systems: the NARMAX model
journal, March 1989

Chen, S.; Billings, S. A.
International Journal of Control, Vol. 49, Issue 3
DOI: 10.1080/00207178908559683

Chaos as an intermittently forced linear system
journal, May 2017

Brunton, Steven L.; Brunton, Bingni W.; Proctor, Joshua L.
Nature Communications, Vol. 8, Issue 1
DOI: 10.1038/s41467-017-00030-8

Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
journal, June 2018

Wehmeyer, Christoph; Noé, Frank
The Journal of Chemical Physics, Vol. 148, Issue 24
DOI: 10.1063/1.5011399

Adversarial uncertainty quantification in physics-informed neural networks
journal, October 2019

Yang, Yibo; Perdikaris, Paris
Journal of Computational Physics, Vol. 394
DOI: 10.1016/j.jcp.2019.05.027

Identification of distributed parameter systems: A neural net based approach
journal, March 1998

González-García, R.; Rico-Martínez, R.; Kevrekidis, I. G.
Computers & Chemical Engineering, Vol. 22
DOI: 10.1016/S0098-1354(98)00191-4

Highly-scalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs
conference, November 2019

Yang, Liu; Prabhat, Mr; Karniadakis, George
2019 IEEE/ACM Third Workshop on Deep Learning on Supercomputers (DLS)
DOI: 10.1109/DLS49591.2019.00006

A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
journal, May 1979

McKay, M. D.; Beckman, R. J.; Conover, W. J.
Technometrics, Vol. 21, Issue 2
DOI: 10.2307/1268522

Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability
journal, January 2012

Giannakis, D.; Majda, A. J.
Proceedings of the National Academy of Sciences, Vol. 109, Issue 7
DOI: 10.1073/pnas.1118984109

Inverse problems: A Bayesian perspective
journal, May 2010

Stuart, A. M.
Acta Numerica, Vol. 19
DOI: 10.1017/S0962492910000061

Deep learning for universal linear embeddings of nonlinear dynamics
journal, November 2018

Lusch, Bethany; Kutz, J. Nathan; Brunton, Steven L.
Nature Communications, Vol. 9, Issue 1
DOI: 10.1038/s41467-018-07210-0

Linear functional minimization for inverse modeling: LINEAR FUNCTIONAL MINIMIZATION FOR INVERSE MODELING
journal, June 2015

Barajas-Solano, D. A.; Wohlberg, B. E.; Vesselinov, V. V.
Water Resources Research, Vol. 51, Issue 6
DOI: 10.1002/2014WR016179

Water and sediment temperature dynamics in shallow tidal environments: The role of the heat flux at the sediment-water interface
journal, March 2018

Pivato, M.; Carniello, L.; Gardner, J.
Advances in Water Resources, Vol. 113
DOI: 10.1016/j.advwatres.2018.01.009

Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
journal, January 2010

Lieberman, Chad; Willcox, Karen; Ghattas, Omar
SIAM Journal on Scientific Computing, Vol. 32, Issue 5
DOI: 10.1137/090775622

Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis
journal, January 2003

Gear, C. William; Hyman, James M.; Kevrekidid, Panagiotis G.
Communications in Mathematical Sciences, Vol. 1, Issue 4
DOI: 10.4310/CMS.2003.v1.n4.a5

Variable Projection Methods for an Optimized Dynamic Mode Decomposition
journal, January 2018

Askham, Travis; Kutz, J. Nathan
SIAM Journal on Applied Dynamical Systems, Vol. 17, Issue 1
DOI: 10.1137/M1124176

A unified deep artificial neural network approach to partial differential equations in complex geometries
journal, November 2018

Berg, Jens; Nyström, Kaj
Neurocomputing, Vol. 317
DOI: 10.1016/j.neucom.2018.06.056

Learning to fail: Predicting fracture evolution in brittle material models using recurrent graph convolutional neural networks
journal, May 2019

Schwarzer, Max; Rogan, Bryce; Ruan, Yadong
Computational Materials Science, Vol. 162
DOI: 10.1016/j.commatsci.2019.02.046

A Closed-form Equation for Predicting the Hydraulic Conductivity of Unsaturated Soils1
journal, January 1980

van Genuchten, M. Th.
Soil Science Society of America Journal, Vol. 44, Issue 5
DOI: 10.2136/sssaj1980.03615995004400050002x

Groundwater level forecasting using artificial neural networks
journal, July 2005

Daliakopoulos, Ioannis N.; Coulibaly, Paulin; Tsanis, Ioannis K.
Journal of Hydrology, Vol. 309, Issue 1-4
DOI: 10.1016/j.jhydrol.2004.12.001

Deep Convolutional Encoder‐Decoder Networks for Uncertainty Quantification of Dynamic Multiphase Flow in Heterogeneous Media
journal, January 2019

Mo, Shaoxing; Zhu, Yinhao; Zabaras, Nicholas
Water Resources Research, Vol. 55, Issue 1
DOI: 10.1029/2018WR023528

Similar Records in DOE PAGES and OSTI.GOV collections:

Physics-informed Karhunen-Loeve and Neural Network Approximations for Solving Inverse Differential Equation Problems

Journal Article Li, Jing ; Tartakovsky, Alexandre M. - Journal of Computational Physics

Here we present the PI-CKL-NN method for parameter estimation in differential equation (DE) models given sparse measurements of the parameters and states. In the proposed approach, the space- or time-dependent parameters are approximated by Karhunen-Loeve (KL) expansions that are conditioned on the parameters’ measurements, and the states are approximated by deep neural networks (DNNs). The unknown weights in the KL expansions and DNNs are found my minimizing the cost function that enforces the measurements of the states the DE constraint. Regularization is achieved by adding the l2 norm of the conditional KL coefficients into the loss function. Our approach assumesmore »« less
https://doi.org/10.1016/j.jcp.2022.111230

Full Text Available
Multi-fidelity Bayesian neural networks: Algorithms and applications

Journal Article Meng, Xuhui ; Babaee, Hessam ; Karniadakis, George Em - Journal of Computational Physics

Here we propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one ismore »« less
https://doi.org/10.1016/j.jcp.2021.110361

Full Text Available
Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Journal Article Chen, Xiaoli ; Duan, Jinqiao ; Karniadakis, George E.M. - European Journal of Applied Mathematics

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation methodmore »« less
https://doi.org/10.1017/s0956792520000169

Full Text Available
Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport

Journal Article He, Qizhi ; Barajas-Solano, David A. ; Tartakovsky, Guzel D. ; ... - Advances in Water Resources

Data assimilation for parameter and state estimation in subsurface transport problems remains a significant challenge due to the sparsity of measurements, the porous media heterogeneity, and high computational cost of forward numerical models. We present a physics-informed deep neural networks (DNNs) machine learning method for estimating space-dependent hydraulic conductivity, hydraulic head, and concentration fields from sparse measurements. In this approach, we employ individual DNNs to approximate the unknown parameters (e.g., hydraulic conductivity) and states (e.g., hydraulic head and concentration) of a physical system, and jointly train these DNNs by minimizing the loss function consisting of the residuals of the governingmore »« less
https://doi.org/10.1016/j.advwatres.2020.103610
Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients

Journal Article Barajas-Solano, David A. ; Tartakovsky, Alexandre M. - Journal of Computational Physics

We present two approximate Bayesian inference methods for parameter estimation in partial differential equation (PDE) models with space-dependent and state-dependent parameters. We demonstrate that these methods provide accurate and cost-effective alternatives to Markov Chain Monte Carlo simulation. We assume a parameterized Gaussian prior on the unknown functions, and approximate the posterior density by a parameterized multivariate Gaussian density. The parameters of the prior and posterior are estimated from sparse observations of the PDE model's states and the unknown functions themselves by maximizing the evidence lower bound (ELBO), a lower bound on the log marginal likelihood of the observations. The firstmore »« less
https://doi.org/10.1016/j.jcp.2019.06.010

Similar Records

Title: Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems

Abstract

Citation Formats

Inverse problem in hydrogeology journal, February 2005

A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition journal, June 2015

Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data journal, October 2019

Modeling fluid flow and transport in variably saturated porous media with the STOMP simulator. 1. Nonvolatile three-phase model description journal, January 1995

Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification journal, August 2018

A Limited Memory Algorithm for Bound Constrained Optimization journal, September 1995

VAMPnets for deep learning of molecular kinetics journal, January 2018

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations journal, February 2019

Representations of non-linear systems: the NARMAX model journal, March 1989

Chaos as an intermittently forced linear system journal, May 2017

Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics journal, June 2018

Adversarial uncertainty quantification in physics-informed neural networks journal, October 2019

Identification of distributed parameter systems: A neural net based approach journal, March 1998

Highly-scalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs conference, November 2019

A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code journal, May 1979

Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability journal, January 2012

Inverse problems: A Bayesian perspective journal, May 2010

Deep learning for universal linear embeddings of nonlinear dynamics journal, November 2018

Linear functional minimization for inverse modeling: LINEAR FUNCTIONAL MINIMIZATION FOR INVERSE MODELING journal, June 2015

Water and sediment temperature dynamics in shallow tidal environments: The role of the heat flux at the sediment-water interface journal, March 2018

Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems journal, January 2010

Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis journal, January 2003

Variable Projection Methods for an Optimized Dynamic Mode Decomposition journal, January 2018

A unified deep artificial neural network approach to partial differential equations in complex geometries journal, November 2018

Learning to fail: Predicting fracture evolution in brittle material models using recurrent graph convolutional neural networks journal, May 2019

A Closed-form Equation for Predicting the Hydraulic Conductivity of Unsaturated Soils1 journal, January 1980

Groundwater level forecasting using artificial neural networks journal, July 2005

Deep Convolutional Encoder‐Decoder Networks for Uncertainty Quantification of Dynamic Multiphase Flow in Heterogeneous Media journal, January 2019

Inverse problem in hydrogeology
journal, February 2005

A Data–Driven Approximation of the Koopman Operator: Extending Dynamic Mode Decomposition
journal, June 2015

Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
journal, October 2019

Modeling fluid flow and transport in variably saturated porous media with the STOMP simulator. 1. Nonvolatile three-phase model description
journal, January 1995

Bayesian deep convolutional encoder–decoder networks for surrogate modeling and uncertainty quantification
journal, August 2018

A Limited Memory Algorithm for Bound Constrained Optimization
journal, September 1995

VAMPnets for deep learning of molecular kinetics
journal, January 2018

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
journal, February 2019

Representations of non-linear systems: the NARMAX model
journal, March 1989

Chaos as an intermittently forced linear system
journal, May 2017

Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics
journal, June 2018

Adversarial uncertainty quantification in physics-informed neural networks
journal, October 2019

Identification of distributed parameter systems: A neural net based approach
journal, March 1998

Highly-scalable, Physics-Informed GANs for Learning Solutions of Stochastic PDEs
conference, November 2019

A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
journal, May 1979

Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability
journal, January 2012

Inverse problems: A Bayesian perspective
journal, May 2010

Deep learning for universal linear embeddings of nonlinear dynamics
journal, November 2018

Linear functional minimization for inverse modeling: LINEAR FUNCTIONAL MINIMIZATION FOR INVERSE MODELING
journal, June 2015

Water and sediment temperature dynamics in shallow tidal environments: The role of the heat flux at the sediment-water interface
journal, March 2018

Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems
journal, January 2010

Equation-Free, Coarse-Grained Multiscale Computation: Enabling Mocroscopic Simulators to Perform System-Level Analysis
journal, January 2003

Variable Projection Methods for an Optimized Dynamic Mode Decomposition
journal, January 2018

A unified deep artificial neural network approach to partial differential equations in complex geometries
journal, November 2018

Learning to fail: Predicting fracture evolution in brittle material models using recurrent graph convolutional neural networks
journal, May 2019

A Closed-form Equation for Predicting the Hydraulic Conductivity of Unsaturated Soils1
journal, January 1980

Groundwater level forecasting using artificial neural networks
journal, July 2005

Deep Convolutional Encoder‐Decoder Networks for Uncertainty Quantification of Dynamic Multiphase Flow in Heterogeneous Media
journal, January 2019