Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Jakeman, John D.; Kouri, Drew P.; Huerta, J. Gabriel

doi:10.1016/j.ress.2021.108280

Title: Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Journal Article · Sat Jan 01 00:00:00 EST 2022 · Reliability Engineering and System Safety

DOI:https://doi.org/10.1016/j.ress.2021.108280· OSTI ID:1845389

^[1];

^[1]; Huerta, J. Gabriel ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

We present a surrogate modeling framework for conservatively estimating measures of risk from limited realizations of an expensive physical experiment or computational simulation. Risk measures combine objective probabilities with the subjective values of a decision maker to quantify anticipated outcomes. Given a set of samples, we construct a surrogate model that produces estimates of risk measures that are always greater than their empirical approximations obtained from the training data. These surrogate models limit over-confidence in reliability and safety assessments and produce estimates of risk measures that converge much faster to the true value than purely sample-based estimates. We first detail the construction of conservative surrogate models that can be tailored to a stakeholder’s risk preferences and then present an approach, based on stochastic orders, for constructing surrogate models that are conservative with respect to families of risk measures. Our surrogate models include biases that permit them to conservatively estimate the target risk measures. We provide theoretical results that show that these biases decay at the same rate as the L₂ error in the surrogate model. Numerical demonstrations confirm that risk-adapted surrogate models do indeed overestimate the target risk measures while converging at the expected rate.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: NA0003525

OSTI ID:: 1845389

Report Number(s):: SAND2022-1272J; 703227

Journal Information:: Reliability Engineering and System Safety, Vol. 221; ISSN 0951-8320

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (53)

Subset simulation for problems with strongly non-Gaussian, highly anisotropic, and degenerate distributions Shields, Michael D.; Giovanis, Dimitris G.; Sundar, V. S. Computers & Structures, Vol. 245 https://doi.org/10.1016/j.compstruc.2020.106431	journal	March 2021
Accelerated subset simulation with neural networks for reliability analysis Papadopoulos, Vissarion; Giovanis, Dimitris G.; Lagaros, Nikos D. Computer Methods in Applied Mechanics and Engineering, Vol. 223-224 https://doi.org/10.1016/j.cma.2012.02.013	journal	June 2012
An Old-New Concept of Convex risk Measures: the Optimized Certainty Equivalent Ben-Tal, Aharon; Teboulle, Marc Mathematical Finance, Vol. 17, Issue 3 https://doi.org/10.1111/j.1467-9965.2007.00311.x	journal	July 2007
Fast surrogate modeling using dimensionality reduction in model inputs and field output: Application to additive manufacturing Vohra, Manav; Nath, Paromita; Mahadevan, Sankaran Reliability Engineering & System Safety, Vol. 201 https://doi.org/10.1016/j.ress.2020.106986	journal	September 2020
Cross-entropy based importance sampling for stochastic simulation models Cao, Quoc Dung; Choe, Youngjun Reliability Engineering & System Safety, Vol. 191 https://doi.org/10.1016/j.ress.2019.106526	journal	November 2019
Hamiltonian Monte Carlo methods for Subset Simulation in reliability analysis Wang, Ziqi; Broccardo, Marco; Song, Junho Structural Safety, Vol. 76 https://doi.org/10.1016/j.strusafe.2018.05.005	journal	January 2019
Sparse polynomial chaos expansions and adaptive stochastic finite elements using a regression approach Blatman, Géraud; Sudret, Bruno Comptes Rendus Mécanique, Vol. 336, Issue 6 https://doi.org/10.1016/j.crme.2008.02.013	journal	June 2008
Risk-Averse PDE-Constrained Optimization Using the Conditional Value-At-Risk Kouri, D. P.; Surowiec, T. M. SIAM Journal on Optimization, Vol. 26, Issue 1 https://doi.org/10.1137/140954556	journal	January 2016
Coherent Measures of Risk Artzner, Philippe; Delbaen, Freddy; Eber, Jean-Marc Mathematical Finance, Vol. 9, Issue 3 https://doi.org/10.1111/1467-9965.00068	journal	July 1999
Annealed importance sampling Neal, Radford M. Statistics and Computing, Vol. 11, Issue 2, p. 125-139 https://doi.org/10.1023/A:1008923215028	journal	April 2001
Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces Constantine, Paul G.; Dow, Eric; Wang, Qiqi SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/130916138	journal	January 2014
Conservative strategy-based ensemble surrogate model for optimal groundwater remediation design at DNAPLs-contaminated sites Ouyang, Qi; Lu, Wenxi; Lin, Jin Journal of Contaminant Hydrology, Vol. 203 https://doi.org/10.1016/j.jconhyd.2017.05.007	journal	August 2017
Reliability analysis with stratified importance sampling based on adaptive Kriging Xiao, Sinan; Oladyshkin, Sergey; Nowak, Wolfgang Reliability Engineering & System Safety, Vol. 197 https://doi.org/10.1016/j.ress.2020.106852	journal	May 2020
Regularity and Generalized Polynomial Chaos Approximation of Parametric and Random Second-Order Hyperbolic Partial Differential Equations Hoang, Viet Ha; Schwab, Christoph Analysis and Applications, Vol. 10, Issue 03 https://doi.org/10.1142/S0219530512500145	journal	July 2012
Information Reuse for Importance Sampling in Reliability-Based Design Optimization Chaudhuri, Anirban; Kramer, Boris; Willcox, Karen E. Reliability Engineering & System Safety, Vol. 201 https://doi.org/10.1016/j.ress.2020.106853	journal	September 2020
Risk Tuning with Generalized Linear Regression Rockafellar, R. Tyrrell; Uryasev, Stan; Zabarankin, Michael Mathematics of Operations Research, Vol. 33, Issue 3 https://doi.org/10.1287/moor.1080.0313	journal	August 2008
Adaptive Reduced-Order Model Construction for Conditional Value-at-Risk Estimation Heinkenschloss, Matthias; Kramer, Boris; Takhtaganov, Timur SIAM/ASA Journal on Uncertainty Quantification, Vol. 8, Issue 2 https://doi.org/10.1137/19M1257433	journal	January 2020
Particle swarm optimization trained neural network for structural failure prediction of multistoried RC buildings Chatterjee, Sankhadeep; Sarkar, Sarbartha; Hore, Sirshendu Neural Computing and Applications, Vol. 28, Issue 8 https://doi.org/10.1007/s00521-016-2190-2	journal	January 2016
An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk Zou, Zilong; Kouri, Drew P.; Aquino, Wilkins 19th AIAA Non-Deterministic Approaches Conference https://doi.org/10.2514/6.2017-1325	conference	January 2017
Multidisciplinary Risk-Adaptive Set-Based Design of Supercavitating Hydrofoils Bonfiglio, Luca; Royset, Johannes O. AIAA Journal, Vol. 57, Issue 8 https://doi.org/10.2514/1.J057731	journal	August 2019
Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions Bichon, B. J.; Eldred, M. S.; Swiler, L. P. AIAA Journal, Vol. 46, Issue 10 https://doi.org/10.2514/1.34321	journal	October 2008
SciPy 1.0: fundamental algorithms for scientific computing in Python Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E. Nature Methods https://doi.org/10.1038/s41592-019-0686-2	journal	February 2020
Probability of failure sensitivity analysis using polynomial expansion Torii, André Jacomel; Lopez, Rafael Holdorf; Miguel, Leandro Fleck Fadel Probabilistic Engineering Mechanics, Vol. 48 https://doi.org/10.1016/j.probengmech.2017.06.001	journal	April 2017
Polynomial chaos expansions for dependent random variables Jakeman, John D.; Franzelin, Fabian; Narayan, Akil Computer Methods in Applied Mechanics and Engineering, Vol. 351 https://doi.org/10.1016/j.cma.2019.03.049	journal	July 2019
Stable architectures for deep neural networks Haber, Eldad; Ruthotto, Lars Inverse Problems, Vol. 34, Issue 1 https://doi.org/10.1088/1361-6420/aa9a90	journal	December 2017
Adaptive subset searching-based deep neural network method for structural reliability analysis Bao, Yuequan; Xiang, Zhengliang; Li, Hui Reliability Engineering & System Safety, Vol. 213 https://doi.org/10.1016/j.ress.2021.107778	journal	September 2021
Distribution-free polynomial chaos expansion surrogate models for efficient structural reliability analysis Lim, HyeongUk; Manuel, Lance Reliability Engineering & System Safety, Vol. 205 https://doi.org/10.1016/j.ress.2020.107256	journal	January 2021
Conditional-Value-at-Risk Estimation via Reduced-Order Models Heinkenschloss, Matthias; Kramer, Boris; Takhtaganov, Timur SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, Issue 4 https://doi.org/10.1137/17M1160069	journal	January 2018
Adaptive surrogate model with active refinement combining Kriging and a trust region method Gaspar, B.; Teixeira, A. P.; Guedes Soares, C. Reliability Engineering & System Safety, Vol. 165 https://doi.org/10.1016/j.ress.2017.03.035	journal	September 2017
Optimization with Stochastic Dominance Constraints Dentcheva, Darinka; Ruszczynski, Andrzej SIAM Journal on Optimization, Vol. 14, Issue 2 https://doi.org/10.1137/S1052623402420528	journal	January 2003
Surrogate model uncertainty quantification for reliability-based design optimization Li, Mingyang; Wang, Zequn Reliability Engineering & System Safety, Vol. 192 https://doi.org/10.1016/j.ress.2019.03.039	journal	December 2019
Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients Todor, Radu Alexandru; Schwab, Christoph IMA Journal of Numerical Analysis, Vol. 27, Issue 2 https://doi.org/10.1093/imanum/drl025	journal	April 2007
Convex measures of risk and trading constraints Föllmer, Hans; Schied, Alexander Finance and Stochastics, Vol. 6, Issue 4 https://doi.org/10.1007/s007800200072	journal	October 2002
Monte Carlo Methods for Value-at-Risk and Conditional Value-at-Risk: A Review Hong, L. Jeff; Hu, Zhaolin; Liu, Guangwu ACM Transactions on Modeling and Computer Simulation, Vol. 24, Issue 4 https://doi.org/10.1145/2661631	journal	August 2014
Non-intrusive low-rank separated approximation of high-dimensional stochastic models Doostan, Alireza; Validi, AbdoulAhad; Iaccarino, Gianluca Computer Methods in Applied Mechanics and Engineering, Vol. 263 https://doi.org/10.1016/j.cma.2013.04.003	journal	August 2013
Gradient-based optimization for regression in the functional tensor-train format Gorodetsky, Alex A.; Jakeman, John D. Journal of Computational Physics, Vol. 374 https://doi.org/10.1016/j.jcp.2018.08.010	journal	December 2018
An Interior Point Algorithm for Large-Scale Nonlinear Programming Byrd, Richard H.; Hribar, Mary E.; Nocedal, Jorge SIAM Journal on Optimization, Vol. 9, Issue 4 https://doi.org/10.1137/S1052623497325107	journal	January 1999
Measures of Residual Risk with Connections to Regression, Risk Tracking, Surrogate Models, and Ambiguity Rockafellar, R. Tyrrell; Royset, Johannes O. SIAM Journal on Optimization, Vol. 25, Issue 2 https://doi.org/10.1137/151003271	journal	January 2015
An active-learning algorithm that combines sparse polynomial chaos expansions and bootstrap for structural reliability analysis Marelli, Stefano; Sudret, Bruno Structural Safety, Vol. 75 https://doi.org/10.1016/j.strusafe.2018.06.003	journal	November 2018
Using Cross Validation to Design Conservative Surrogates Viana, Felipe A. C.; Picheny, Victor; Haftka, Raphael T. AIAA Journal, Vol. 48, Issue 10 https://doi.org/10.2514/1.J050327	journal	October 2010
LIF: A new Kriging based learning function and its application to structural reliability analysis Sun, Zhili; Wang, Jian; Li, Rui Reliability Engineering & System Safety, Vol. 157 https://doi.org/10.1016/j.ress.2016.09.003	journal	January 2017
A scenario optimization approach to reliability-based and risk-based design: Soft-constrained modulation of failure probability bounds Rocchetta, Roberto; Crespo, Luis G. Reliability Engineering & System Safety, Vol. 216 https://doi.org/10.1016/j.ress.2021.107900	journal	December 2021
Deep Learning of Parameterized Equations with Applications to Uncertainty Quantification Qin, Tong; Chen, Zhen; Jakeman, John D. International Journal for Uncertainty Quantification, Vol. 11, Issue 2 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2020034123	journal	January 2021
Estimation of small failure probabilities in high dimensions by subset simulation Au, Siu-Kui; Beck, James L. Probabilistic Engineering Mechanics, Vol. 16, Issue 4 https://doi.org/10.1016/S0266-8920(01)00019-4	journal	October 2001
Structural reliability assessment through surrogate based importance sampling with dimension reduction Zuniga, M. Munoz; Murangira, A.; Perdrizet, T. Reliability Engineering & System Safety, Vol. 207 https://doi.org/10.1016/j.ress.2020.107289	journal	March 2021
Existence and Optimality Conditions for Risk-Averse PDE-Constrained Optimization Kouri, D. P.; Surowiec, T. M. SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, Issue 2 https://doi.org/10.1137/16M1086613	journal	January 2018
Stochastic Dominance Constraints in Elastic Shape Optimization Conti, Sergio; Rumpf, Martin; Schultz, Rüdiger SIAM Journal on Control and Optimization, Vol. 56, Issue 4 https://doi.org/10.1137/16M108313X	journal	January 2018
On buffered failure probability in design and optimization of structures Rockafellar, R. T.; Royset, J. O. Reliability Engineering & System Safety, Vol. 95, Issue 5 https://doi.org/10.1016/j.ress.2010.01.001	journal	May 2010
On confidence intervals for failure probability estimates in Kriging-based reliability analysis Wang, Zeyu; Shafieezadeh, Abdollah Reliability Engineering & System Safety, Vol. 196 https://doi.org/10.1016/j.ress.2019.106758	journal	April 2020
The future of risk assessment Zio, E. Reliability Engineering & System Safety, Vol. 177 https://doi.org/10.1016/j.ress.2018.04.020	journal	September 2018
Risk-Adaptive Set-Based Design and Applications to Shaping a Hydrofoil Royset, J. O.; Bonfiglio, L.; Vernengo, G. Journal of Mechanical Design, Vol. 139, Issue 10 https://doi.org/10.1115/1.4037623	journal	August 2017
On the convergence of generalized polynomial chaos expansions Ernst, Oliver G.; Mugler, Antje; Starkloff, Hans-Jörg ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 46, Issue 2 https://doi.org/10.1051/m2an/2011045	journal	October 2011
An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk Zou, Zilong; Kouri, Drew; Aquino, Wilkins Computer Methods in Applied Mechanics and Engineering, Vol. 345 https://doi.org/10.1016/j.cma.2018.10.028	journal	March 2019

Similar Records

Surrogate Modeling For Efficiently Accurately and Conservatively Estimating Measures of Risk

Technical Report · Sun Jul 11 00:00:00 EDT 2021 · OSTI ID:1845389

Jakeman, John Davis; Kouri, Drew Philip; Huerta, Jose Gabriel

Risk-Adaptive Experimental Design for High-Consequence Systems: LDRD Final Report

Technical Report · Wed Sep 01 00:00:00 EDT 2021 · OSTI ID:1845389

Kouri, Drew P.; Jakeman, John D.; Huerta, Jose Gabriel; +4 more

Stakeholder-Engaged Structured Decision Making for the Los Alamos Legacy Cleanup Mission - 20501

Conference · Wed Jul 01 00:00:00 EDT 2020 · OSTI ID:1845389

Black, Paul; Stockton, Tom; Black, Kelly

Related Subjects

42 ENGINEERING
surrogate models
risk measures
stochastic dominance
least squares regression
statistical models
polynomial chaos
risk management

Title: Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Citation Formats

References (53)

Similar Records

Related Subjects