skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Machine-learning error models for approximate solutions to parameterized systems of nonlinear equations

Journal Article · · Computer Methods in Applied Mechanics and Engineering
ORCiD logo [1];  [2]
  1. Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
  2. Sandia National Lab. (SNL-CA), Livermore, CA (United States)
+ Show Author Affiliations

This work proposes a machine-learning framework for constructing statistical models of errors incurred by approximate solutions to parameterized systems of nonlinear equations. These approximate solutions may arise from early termination of an iterative method, a lower-fidelity model, or a projection-based reduced-order model, for example. The proposed statistical model comprises the sum of a deterministic regression-function model and a stochastic noise model. The method constructs the regression-function model by applying regression techniques from machine learning (e.g., support vector regression, artificial neural networks) to map features (i.e., error indicators such as sampled elements of the residual) to a prediction of the approximate-solution error. The method constructs the noise model as a mean-zero Gaussian random variable whose variance is computed as the sample variance of the approximate-solution error on a test set; this variance can be interpreted as the epistemic uncertainty introduced by the approximate solution. Here, this work considers a wide range of feature-engineering methods, data-set-construction techniques, and regression techniques that aim to ensure that (1) the features are cheaply computable, (2) the noise model exhibits low variance (i.e., low epistemic uncertainty introduced), and (3) the regression model generalizes to independent test data. Numerical experiments performed on several computational-mechanics problems and types of approximate solutions demonstrate the ability of the method to generate statistical models of the error that satisfy these criteria and significantly outperform more commonly adopted approaches for error modeling.

Research Organization:
Sandia National Lab. (SNL-NM), Albuquerque, NM (United States); Sandia National Lab. (SNL-CA), Livermore, CA (United States)
Sponsoring Organization:
USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC04-94AL85000; NA0003525
OSTI ID:
1498489
Alternate ID(s):
OSTI ID: 1498763
Report Number(s):
SAND-2018-13294J; SAND-2018-8670J; 670277
Journal Information:
Computer Methods in Applied Mechanics and Engineering, Vol. 348, Issue C; ISSN 0045-7825
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 13 works
Citation information provided by
Web of Science

References (53)

Parametric Reduced-Order Models for Probabilistic Analysis of Unsteady Aerodynamic Applications journal October 2008
Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space journal January 2008
Residual based sampling in POD model order reduction of drift-diffusion equations in parametrized electrical networks journal September 2011
Design optimization using hyper-reduced-order models journal November 2014
Adaptive training of local reduced bases for unsteady incompressible Navier-Stokes flows: Adaptive training of local reduced bases for unsteady incompressible Navier-Stokes flows journal March 2015
Progressive construction of a parametric reduced-order model for PDE-constrained optimization: PDE-CONSTRAINED OPTIMIZATION USING A PROGRESSIVELY CONSTRUCTED ROM journal December 2014
The post-processing approach in the finite element method—part 1: Calculation of displacements, stresses and other higher derivatives of the displacements journal June 1984
Adjoint Error Estimation and Grid Adaptation for Functional Outputs: Application to Quasi-One-Dimensional Flow journal October 2000
Grid Adaptation for Functional Outputs: Application to Two-Dimensional Inviscid Flows journal February 2002
Adjoint-Based, Three-Dimensional Error Prediction and Grid Adaptation journal September 2004
Efficient model reduction in non-linear dynamics using the Karhunen-Lo�ve expansion and dual-weighted-residual methods journal May 2003
Adaptive h -refinement for reduced-order models : ADAPTIVE journal November 2014
Randomized Residual-Based Error Estimators for Parametrized Equations journal January 2019
A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations journal January 2005
Reduced Basis Approximation and a Posteriori Error Estimation for Affinely Parametrized Elliptic Coercive Partial Differential Equations: Application to Transport and Continuum Mechanics journal May 2008
A natural-norm Successive Constraint Method for inf-sup lower bounds journal June 2010
A Posteriori Error Estimation for DEIM Reduced Nonlinear Dynamical Systems journal January 2014
Hybrid Variable Fidelity Optimization by Using a Kriging-Based Scaling Function journal November 2005
Sequential kriging optimization using multiple-fidelity evaluations journal May 2006
Provably Convergent Multifidelity Optimization Algorithm Not Requiring High-Fidelity Derivatives journal May 2012
Approximation and Model Management in Aerodynamic Optimization with Variable-Fidelity Models journal November 2001
Bayesian calibration of computer models journal August 2001
Combining Field Data and Computer Simulations for Calibration and Prediction journal January 2004
Efficient State/Parameter Estimation in Nonlinear Unsteady PDEs by a Reduced Basis Ensemble Kalman Filter journal January 2017
Multivariate predictions of local reduced-order-model errors and dimensions
  • Moosavi, Azam; Ştefănescu, Răzvan; Sandu, Adrian
  • International Journal for Numerical Methods in Engineering, Vol. 113, Issue 3 https://doi.org/10.1002/nme.5624
journal October 2017
Parametric domain decomposition for accurate reduced order models: Applications of MP-LROM methodology journal October 2018
The ROMES Method for Statistical Modeling of Reduced-Order-Model Error journal January 2015
Accurate Solution of Bayesian Inverse Uncertainty Quantification Problems Combining Reduced Basis Methods and Reduction Error Models journal January 2016
Error modeling for surrogates of dynamical systems using machine learning: Machine-learning-based error model for surrogates of dynamical systems
  • Trehan, Sumeet; Carlberg, Kevin T.; Durlofsky, Louis J.
  • International Journal for Numerical Methods in Engineering, Vol. 112, Issue 12 https://doi.org/10.1002/nme.5583
journal July 2017
Analysis of Inexact Trust-Region SQP Algorithms journal January 2002
Turbulence and the dynamics of coherent structures. I. Coherent structures journal January 1987
A proper orthogonal decomposition method for nonlinear flows with deforming meshes journal December 2014
Reduced Basis Technique for Nonlinear Analysis of Structures journal April 1980
A Reduced-Order Method for Simulation and Control of Fluid Flows journal July 1998
A low-cost, goal-oriented ‘compact proper orthogonal decomposition’ basis for model reduction of static systems journal December 2010
Efficient non-linear model reduction via a least-squares Petrov-Galerkin projection and compressive tensor approximations
  • Carlberg, Kevin; Bou-Mosleh, Charbel; Farhat, Charbel
  • International Journal for Numerical Methods in Engineering, Vol. 86, Issue 2 https://doi.org/10.1002/nme.3050
journal October 2010
The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows journal June 2013
Galerkin v. least-squares Petrov–Galerkin projection in nonlinear model reduction journal February 2017
Karhunen–Loève procedure for gappy data journal January 1995
An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations journal November 2004
Nonlinear Model Reduction via Discrete Empirical Interpolation journal January 2010
Jacobian-free Newton–Krylov methods: a survey of approaches and applications journal January 2004
Gappy data and reconstruction procedures for flow past a cylinder journal January 1999
Unsteady flow sensing and estimation via the gappy proper orthogonal decomposition journal February 2006
Aerodynamic Data Reconstruction and Inverse Design Using Proper Orthogonal Decomposition journal August 2004
Dynamic data-driven model reduction: adapting reduced models from incomplete data journal March 2016
Decreasing the temporal complexity for nonlinear, implicit reduced-order models by forecasting journal June 2015
A New Selection Operator for the Discrete Empirical Interpolation Method---Improved A Priori Error Bound and Extensions journal January 2016
A tutorial on support vector regression journal August 2004
Random Forests journal January 2001
Albany: Using Component-Based Design to Develop a Flexible, Generic Multiphysics Analysis code journal January 2016
High-order compact exponential finite difference methods for convection–diffusion type problems journal January 2007
Missing Point Estimation in Models Described by Proper Orthogonal Decomposition journal November 2008

Cited By (1)

On-the-fly adaptivity for nonlinear twoscale simulations using artificial neural networks and reduced order modeling text January 2019

Similar Records

Time-series machine-learning error models for approximate solutions to parameterized dynamical systems
Journal Article · Sat Mar 28 00:00:00 EDT 2020 · Computer Methods in Applied Mechanics and Engineering · OSTI ID:1498489
Error modeling for surrogates of dynamical systems using machine learning: Machine-learning-based error model for surrogates of dynamical systems
Journal Article · Fri Jul 14 00:00:00 EDT 2017 · International Journal for Numerical Methods in Engineering · OSTI ID:1498489
Epistemic Uncertainty-Aware Barlow Twins Reduced Order Modeling for Nonlinear Contact Problems
Journal Article · Sun Jan 01 00:00:00 EST 2023 · IEEE Access · OSTI ID:1498489
+2 more

Related Subjects

97 MATHEMATICS AND COMPUTING
Error modeling
Supervised machine learning
High-dimensional regression
Parameterized nonlinear equations
Model reduction
ROMES method