Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Jakeman, John D.; Kouri, Drew P.; Huerta, J. Gabriel

doi:10.1016/j.ress.2021.108280

Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Journal Article · 01 January 2022 · Reliability Engineering and System Safety

DOI:https://doi.org/10.1016/j.ress.2021.108280· OSTI ID:1845389

^[1]; ^[1]; Huerta, J. Gabriel ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

We present a surrogate modeling framework for conservatively estimating measures of risk from limited realizations of an expensive physical experiment or computational simulation. Risk measures combine objective probabilities with the subjective values of a decision maker to quantify anticipated outcomes. Given a set of samples, we construct a surrogate model that produces estimates of risk measures that are always greater than their empirical approximations obtained from the training data. These surrogate models limit over-confidence in reliability and safety assessments and produce estimates of risk measures that converge much faster to the true value than purely sample-based estimates. We first detail the construction of conservative surrogate models that can be tailored to a stakeholder’s risk preferences and then present an approach, based on stochastic orders, for constructing surrogate models that are conservative with respect to families of risk measures. Our surrogate models include biases that permit them to conservatively estimate the target risk measures. We provide theoretical results that show that these biases decay at the same rate as the L₂ error in the surrogate model. Numerical demonstrations confirm that risk-adapted surrogate models do indeed overestimate the target risk measures while converging at the expected rate.

View Accepted Manuscript (DOE)

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: NA0003525

OSTI ID:: 1845389

Report Number(s):: SAND2022-1272J; 703227

Journal Information:: Reliability Engineering and System Safety, Journal Name: Reliability Engineering and System Safety Vol. 221; ISSN 0951-8320

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (53)

Particle swarm optimization trained neural network for structural failure prediction of multistoried RC buildings Chatterjee, Sankhadeep; Sarkar, Sarbartha; Hore, Sirshendu Neural Computing and Applications, Vol. 28, Issue 8 https://doi.org/10.1007/s00521-016-2190-2	journal	January 2016
Convex measures of risk and trading constraints Föllmer, Hans; Schied, Alexander Finance and Stochastics, Vol. 6, Issue 4 https://doi.org/10.1007/s007800200072	journal	October 2002
Estimation of small failure probabilities in high dimensions by subset simulation Au, Siu-Kui; Beck, James L. Probabilistic Engineering Mechanics, Vol. 16, Issue 4 https://doi.org/10.1016/S0266-8920(01)00019-4	journal	October 2001
Accelerated subset simulation with neural networks for reliability analysis Papadopoulos, Vissarion; Giovanis, Dimitris G.; Lagaros, Nikos D. Computer Methods in Applied Mechanics and Engineering, Vol. 223-224 https://doi.org/10.1016/j.cma.2012.02.013	journal	June 2012
Non-intrusive low-rank separated approximation of high-dimensional stochastic models Doostan, Alireza; Validi, AbdoulAhad; Iaccarino, Gianluca Computer Methods in Applied Mechanics and Engineering, Vol. 263 https://doi.org/10.1016/j.cma.2013.04.003	journal	August 2013
An adaptive local reduced basis method for solving PDEs with uncertain inputs and evaluating risk Zou, Zilong; Kouri, Drew; Aquino, Wilkins Computer Methods in Applied Mechanics and Engineering, Vol. 345 https://doi.org/10.1016/j.cma.2018.10.028	journal	March 2019
Polynomial chaos expansions for dependent random variables Jakeman, John D.; Franzelin, Fabian; Narayan, Akil Computer Methods in Applied Mechanics and Engineering, Vol. 351 https://doi.org/10.1016/j.cma.2019.03.049	journal	July 2019
Subset simulation for problems with strongly non-Gaussian, highly anisotropic, and degenerate distributions Shields, Michael D.; Giovanis, Dimitris G.; Sundar, V. S. Computers & Structures, Vol. 245 https://doi.org/10.1016/j.compstruc.2020.106431	journal	March 2021
Sparse polynomial chaos expansions and adaptive stochastic finite elements using a regression approach Blatman, Géraud; Sudret, Bruno Comptes Rendus Mécanique, Vol. 336, Issue 6 https://doi.org/10.1016/j.crme.2008.02.013	journal	June 2008
Conservative strategy-based ensemble surrogate model for optimal groundwater remediation design at DNAPLs-contaminated sites Ouyang, Qi; Lu, Wenxi; Lin, Jin Journal of Contaminant Hydrology, Vol. 203 https://doi.org/10.1016/j.jconhyd.2017.05.007	journal	August 2017
Gradient-based optimization for regression in the functional tensor-train format Gorodetsky, Alex A.; Jakeman, John D. Journal of Computational Physics, Vol. 374 https://doi.org/10.1016/j.jcp.2018.08.010	journal	December 2018
Probability of failure sensitivity analysis using polynomial expansion Torii, André Jacomel; Lopez, Rafael Holdorf; Miguel, Leandro Fleck Fadel Probabilistic Engineering Mechanics, Vol. 48 https://doi.org/10.1016/j.probengmech.2017.06.001	journal	April 2017
On buffered failure probability in design and optimization of structures Rockafellar, R. T.; Royset, J. O. Reliability Engineering & System Safety, Vol. 95, Issue 5 https://doi.org/10.1016/j.ress.2010.01.001	journal	May 2010
LIF: A new Kriging based learning function and its application to structural reliability analysis Sun, Zhili; Wang, Jian; Li, Rui Reliability Engineering & System Safety, Vol. 157 https://doi.org/10.1016/j.ress.2016.09.003	journal	January 2017
Adaptive surrogate model with active refinement combining Kriging and a trust region method Gaspar, B.; Teixeira, A. P.; Guedes Soares, C. Reliability Engineering & System Safety, Vol. 165 https://doi.org/10.1016/j.ress.2017.03.035	journal	September 2017
The future of risk assessment Zio, E. Reliability Engineering & System Safety, Vol. 177 https://doi.org/10.1016/j.ress.2018.04.020	journal	September 2018
Surrogate model uncertainty quantification for reliability-based design optimization Li, Mingyang; Wang, Zequn Reliability Engineering & System Safety, Vol. 192 https://doi.org/10.1016/j.ress.2019.03.039	journal	December 2019
Cross-entropy based importance sampling for stochastic simulation models Cao, Quoc Dung; Choe, Youngjun Reliability Engineering & System Safety, Vol. 191 https://doi.org/10.1016/j.ress.2019.106526	journal	November 2019
On confidence intervals for failure probability estimates in Kriging-based reliability analysis Wang, Zeyu; Shafieezadeh, Abdollah Reliability Engineering & System Safety, Vol. 196 https://doi.org/10.1016/j.ress.2019.106758	journal	April 2020
Reliability analysis with stratified importance sampling based on adaptive Kriging Xiao, Sinan; Oladyshkin, Sergey; Nowak, Wolfgang Reliability Engineering & System Safety, Vol. 197 https://doi.org/10.1016/j.ress.2020.106852	journal	May 2020
Information Reuse for Importance Sampling in Reliability-Based Design Optimization Chaudhuri, Anirban; Kramer, Boris; Willcox, Karen E. Reliability Engineering & System Safety, Vol. 201 https://doi.org/10.1016/j.ress.2020.106853	journal	September 2020
Fast surrogate modeling using dimensionality reduction in model inputs and field output: Application to additive manufacturing Vohra, Manav; Nath, Paromita; Mahadevan, Sankaran Reliability Engineering & System Safety, Vol. 201 https://doi.org/10.1016/j.ress.2020.106986	journal	September 2020
Distribution-free polynomial chaos expansion surrogate models for efficient structural reliability analysis Lim, HyeongUk; Manuel, Lance Reliability Engineering & System Safety, Vol. 205 https://doi.org/10.1016/j.ress.2020.107256	journal	January 2021
Structural reliability assessment through surrogate based importance sampling with dimension reduction Zuniga, M. Munoz; Murangira, A.; Perdrizet, T. Reliability Engineering & System Safety, Vol. 207 https://doi.org/10.1016/j.ress.2020.107289	journal	March 2021
Adaptive subset searching-based deep neural network method for structural reliability analysis Bao, Yuequan; Xiang, Zhengliang; Li, Hui Reliability Engineering & System Safety, Vol. 213 https://doi.org/10.1016/j.ress.2021.107778	journal	September 2021
A scenario optimization approach to reliability-based and risk-based design: Soft-constrained modulation of failure probability bounds Rocchetta, Roberto; Crespo, Luis G. Reliability Engineering & System Safety, Vol. 216 https://doi.org/10.1016/j.ress.2021.107900	journal	December 2021
Hamiltonian Monte Carlo methods for Subset Simulation in reliability analysis Wang, Ziqi; Broccardo, Marco; Song, Junho Structural Safety, Vol. 76 https://doi.org/10.1016/j.strusafe.2018.05.005	journal	January 2019
An active-learning algorithm that combines sparse polynomial chaos expansions and bootstrap for structural reliability analysis Marelli, Stefano; Sudret, Bruno Structural Safety, Vol. 75 https://doi.org/10.1016/j.strusafe.2018.06.003	journal	November 2018
Annealed importance sampling Neal, Radford M. Statistics and Computing, Vol. 11, Issue 2, p. 125-139 https://doi.org/10.1023/A:1008923215028	journal	April 2001
SciPy 1.0: fundamental algorithms for scientific computing in Python Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E. Nature Methods https://doi.org/10.1038/s41592-019-0686-2	journal	February 2020
On the convergence of generalized polynomial chaos expansions Ernst, Oliver G.; Mugler, Antje; Starkloff, Hans-Jörg ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 46, Issue 2 https://doi.org/10.1051/m2an/2011045	journal	October 2011
Stable architectures for deep neural networks Haber, Eldad; Ruthotto, Lars Inverse Problems, Vol. 34, Issue 1 https://doi.org/10.1088/1361-6420/aa9a90	journal	December 2017
Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients Todor, Radu Alexandru; Schwab, Christoph IMA Journal of Numerical Analysis, Vol. 27, Issue 2 https://doi.org/10.1093/imanum/drl025	journal	April 2007
Coherent Measures of Risk Artzner, Philippe; Delbaen, Freddy; Eber, Jean-Marc Mathematical Finance, Vol. 9, Issue 3 https://doi.org/10.1111/1467-9965.00068	journal	July 1999
An Old-New Concept of Convex risk Measures: the Optimized Certainty Equivalent Ben-Tal, Aharon; Teboulle, Marc Mathematical Finance, Vol. 17, Issue 3 https://doi.org/10.1111/j.1467-9965.2007.00311.x	journal	July 2007
Risk-Adaptive Set-Based Design and Applications to Shaping a Hydrofoil Royset, J. O.; Bonfiglio, L.; Vernengo, G. Journal of Mechanical Design, Vol. 139, Issue 10 https://doi.org/10.1115/1.4037623	journal	August 2017
Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces Constantine, Paul G.; Dow, Eric; Wang, Qiqi SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/130916138	journal	January 2014
Risk-Averse PDE-Constrained Optimization Using the Conditional Value-At-Risk Kouri, D. P.; Surowiec, T. M. SIAM Journal on Optimization, Vol. 26, Issue 1 https://doi.org/10.1137/140954556	journal	January 2016
Measures of Residual Risk with Connections to Regression, Risk Tracking, Surrogate Models, and Ambiguity Rockafellar, R. Tyrrell; Royset, Johannes O. SIAM Journal on Optimization, Vol. 25, Issue 2 https://doi.org/10.1137/151003271	journal	January 2015
Stochastic Dominance Constraints in Elastic Shape Optimization Conti, Sergio; Rumpf, Martin; Schultz, Rüdiger SIAM Journal on Control and Optimization, Vol. 56, Issue 4 https://doi.org/10.1137/16M108313X	journal	January 2018
Existence and Optimality Conditions for Risk-Averse PDE-Constrained Optimization Kouri, D. P.; Surowiec, T. M. SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, Issue 2 https://doi.org/10.1137/16M1086613	journal	January 2018
Conditional-Value-at-Risk Estimation via Reduced-Order Models Heinkenschloss, Matthias; Kramer, Boris; Takhtaganov, Timur SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, Issue 4 https://doi.org/10.1137/17M1160069	journal	January 2018
Adaptive Reduced-Order Model Construction for Conditional Value-at-Risk Estimation Heinkenschloss, Matthias; Kramer, Boris; Takhtaganov, Timur SIAM/ASA Journal on Uncertainty Quantification, Vol. 8, Issue 2 https://doi.org/10.1137/19M1257433	journal	January 2020
Optimization with Stochastic Dominance Constraints Dentcheva, Darinka; Ruszczynski, Andrzej SIAM Journal on Optimization, Vol. 14, Issue 2 https://doi.org/10.1137/S1052623402420528	journal	January 2003
An Interior Point Algorithm for Large-Scale Nonlinear Programming Byrd, Richard H.; Hribar, Mary E.; Nocedal, Jorge SIAM Journal on Optimization, Vol. 9, Issue 4 https://doi.org/10.1137/S1052623497325107	journal	January 1999
Regularity and Generalized Polynomial Chaos Approximation of Parametric and Random Second-Order Hyperbolic Partial Differential Equations Hoang, Viet Ha; Schwab, Christoph Analysis and Applications, Vol. 10, Issue 03 https://doi.org/10.1142/S0219530512500145	journal	July 2012
Monte Carlo Methods for Value-at-Risk and Conditional Value-at-Risk: A Review Hong, L. Jeff; Hu, Zhaolin; Liu, Guangwu ACM Transactions on Modeling and Computer Simulation, Vol. 24, Issue 4 https://doi.org/10.1145/2661631	journal	August 2014
Risk Tuning with Generalized Linear Regression Rockafellar, R. Tyrrell; Uryasev, Stan; Zabarankin, Michael Mathematics of Operations Research, Vol. 33, Issue 3 https://doi.org/10.1287/moor.1080.0313	journal	August 2008
Deep Learning of Parameterized Equations with Applications to Uncertainty Quantification Qin, Tong; Chen, Zhen; Jakeman, John D. International Journal for Uncertainty Quantification, Vol. 11, Issue 2 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2020034123	journal	January 2021
Efficient Global Reliability Analysis for Nonlinear Implicit Performance Functions Bichon, B. J.; Eldred, M. S.; Swiler, L. P. AIAA Journal, Vol. 46, Issue 10 https://doi.org/10.2514/1.34321	journal	October 2008
Using Cross Validation to Design Conservative Surrogates Viana, Felipe A. C.; Picheny, Victor; Haftka, Raphael T. AIAA Journal, Vol. 48, Issue 10 https://doi.org/10.2514/1.J050327	journal	October 2010
Multidisciplinary Risk-Adaptive Set-Based Design of Supercavitating Hydrofoils Bonfiglio, Luca; Royset, Johannes O. AIAA Journal, Vol. 57, Issue 8 https://doi.org/10.2514/1.J057731	journal	August 2019
An Adaptive Sampling Approach for Solving PDEs with Uncertain Inputs and Evaluating Risk Zou, Zilong; Kouri, Drew P.; Aquino, Wilkins 19th AIAA Non-Deterministic Approaches Conference https://doi.org/10.2514/6.2017-1325	conference	January 2017

Similar Records

Surrogate Modeling For Efficiently Accurately and Conservatively Estimating Measures of Risk

Technical Report · 2021 · OSTI ID:1807455

Efficient learning of accurate surrogates for simulations of complex systems

Journal Article · 2024 · Nature Machine Intelligence · OSTI ID:2426829

Related Subjects

42 ENGINEERING
least squares regression
polynomial chaos
risk management
risk measures
statistical models
stochastic dominance
surrogate models

Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk

Citation Formats

References (53)

Similar Records

Related Subjects