skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: A frequentist approach to computer model calibration

Abstract

The paper considers the computer model calibration problem and provides a general frequentist solution. Under the framework proposed, the data model is semiparametric with a non-parametric discrepancy function which accounts for any discrepancy between physical reality and the computer model. In an attempt to solve a fundamentally important (but often ignored) identifiability issue between the computer model parameters and the discrepancy function, the paper proposes a new and identifiable parameterization of the calibration problem. It also develops a two-step procedure for estimating all the relevant quantities under the new parameterization. This estimation procedure is shown to enjoy excellent rates of convergence and can be straightforwardly implemented with existing software. For uncertainty quantification, bootstrapping is adopted to construct confidence regions for the quantities of interest. As a result, the practical performance of the methodology is illustrated through simulation examples and an application to a computational fluid dynamics model.

Authors:
 [1];  [2];  [3]
  1. Iowa State Univ., Ames, IA (United States)
  2. Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
  3. Univ. of California, Davis, CA (United States)
Publication Date:
Research Org.:
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1375159
Report Number(s):
LA-UR-14-29354
Journal ID: ISSN 1369-7412
Grant/Contract Number:  
AC52-06NA25396
Resource Type:
Accepted Manuscript
Journal Name:
Journal of the Royal Statistical Society: Series B (Statistical Methodology)
Additional Journal Information:
Journal Name: Journal of the Royal Statistical Society: Series B (Statistical Methodology); Journal Volume: 79; Journal Issue: 2; Journal ID: ISSN 1369-7412
Publisher:
Royal Statistical Society - Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Mathematics; Bootstrap; Inverse problem; Model misspecification; Semiparametric modelling; Surrogate model; Uncertainty analysis

Citation Formats

Wong, Raymond K. W., Storlie, Curtis Byron, and Lee, Thomas C. M. A frequentist approach to computer model calibration. United States: N. p., 2016. Web. https://doi.org/10.1111/rssb.12182.
Wong, Raymond K. W., Storlie, Curtis Byron, & Lee, Thomas C. M. A frequentist approach to computer model calibration. United States. https://doi.org/10.1111/rssb.12182
Wong, Raymond K. W., Storlie, Curtis Byron, and Lee, Thomas C. M. Thu . "A frequentist approach to computer model calibration". United States. https://doi.org/10.1111/rssb.12182. https://www.osti.gov/servlets/purl/1375159.
@article{osti_1375159,
title = {A frequentist approach to computer model calibration},
author = {Wong, Raymond K. W. and Storlie, Curtis Byron and Lee, Thomas C. M.},
abstractNote = {The paper considers the computer model calibration problem and provides a general frequentist solution. Under the framework proposed, the data model is semiparametric with a non-parametric discrepancy function which accounts for any discrepancy between physical reality and the computer model. In an attempt to solve a fundamentally important (but often ignored) identifiability issue between the computer model parameters and the discrepancy function, the paper proposes a new and identifiable parameterization of the calibration problem. It also develops a two-step procedure for estimating all the relevant quantities under the new parameterization. This estimation procedure is shown to enjoy excellent rates of convergence and can be straightforwardly implemented with existing software. For uncertainty quantification, bootstrapping is adopted to construct confidence regions for the quantities of interest. As a result, the practical performance of the methodology is illustrated through simulation examples and an application to a computational fluid dynamics model.},
doi = {10.1111/rssb.12182},
journal = {Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
number = 2,
volume = 79,
place = {United States},
year = {2016},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

Empirical Bayesian Analysis for Computer Experiments Involving Finite-Difference Codes
journal, December 2006

  • Drignei, Dorin; Morris, Max D.
  • Journal of the American Statistical Association, Vol. 101, Issue 476
  • DOI: 10.1198/016214506000000898

Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA
journal, January 2015

  • Storlie, Curtis B.; Lane, William A.; Ryan, Emily M.
  • Journal of the American Statistical Association, Vol. 110, Issue 509
  • DOI: 10.1080/01621459.2014.979993

Computer Model Calibration Using High-Dimensional Output
journal, June 2008

  • Higdon, Dave; Gattiker, James; Williams, Brian
  • Journal of the American Statistical Association, Vol. 103, Issue 482
  • DOI: 10.1198/016214507000000888

Learning about physical parameters: the importance of model discrepancy
journal, October 2014


On Bootstrap Confidence Intervals in Nonparametric Regression
journal, June 1992


Bayesian calibration of computer models
journal, August 2001

  • Kennedy, Marc C.; O'Hagan, Anthony
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 63, Issue 3
  • DOI: 10.1111/1467-9868.00294

Genetic Optimization Using Derivatives
journal, January 1998


Numerical modeling and uncertainty quantification of a bubbling fluidized bed with immersed horizontal tubes
journal, February 2014


Simultaneous confidence and prediction intervals for nonlinear regression models with application to a groundwater flow model
journal, July 1987


Probabilistic sensitivity analysis of complex models: a Bayesian approach
journal, August 2004

  • Oakley, Jeremy E.; O'Hagan, Anthony
  • Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 66, Issue 3
  • DOI: 10.1111/j.1467-9868.2004.05304.x

A Framework for Validation of Computer Models
journal, May 2007


Combining Field Data and Computer Simulations for Calibration and Prediction
journal, January 2004

  • Higdon, Dave; Kennedy, Marc; Cavendish, James C.
  • SIAM Journal on Scientific Computing, Vol. 26, Issue 2
  • DOI: 10.1137/S1064827503426693

Analysis of computationally demanding models with continuous and categorical inputs
journal, May 2013

  • Storlie, Curtis B.; Reich, Brian J.; Helton, Jon C.
  • Reliability Engineering & System Safety, Vol. 113
  • DOI: 10.1016/j.ress.2012.11.018

Hierarchical calibration and validation of computational fluid dynamics models for solid sorbent-based carbon capture
journal, January 2016


Variable Selection in Bayesian Smoothing Spline ANOVA Models: Application to Deterministic Computer Codes
journal, May 2009

  • Reich, Brian J.; Storlie, Curtis B.; Bondell, Howard D.
  • Technometrics, Vol. 51, Issue 2
  • DOI: 10.1198/TECH.2009.0013

Gaussian process emulation of dynamic computer codes
journal, June 2009


Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models
journal, March 2006


Bayesian inference with misspecified models
journal, October 2013


Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models
journal, November 2009

  • Storlie, Curtis B.; Swiler, Laura P.; Helton, Jon C.
  • Reliability Engineering & System Safety, Vol. 94, Issue 11
  • DOI: 10.1016/j.ress.2009.05.007

Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments
journal, December 1991


Carbon Capture Simulation Initiative: A Case Study in Multiscale Modeling and New Challenges
journal, June 2014


Statistical Adjustments to Engineering Models
journal, October 2009


    Works referencing / citing this record:

    Nonparametric estimation of probabilistic sensitivity measures
    journal, August 2019

    • Antoniano-Villalobos, Isadora; Borgonovo, Emanuele; Lu, Xuefei
    • Statistics and Computing, Vol. 30, Issue 2
    • DOI: 10.1007/s11222-019-09887-9