Discovering Active Subspaces for High-Dimensional Computer Models

Rumsey, Kellin; Francom, Devin; Wiel, Scott Vander

doi:10.1080/10618600.2023.2277297

Discovering Active Subspaces for High-Dimensional Computer Models

Journal Article · Tue Dec 19 00:00:00 EST 2023 · Journal of Computational and Graphical Statistics

DOI:https://doi.org/10.1080/10618600.2023.2277297· OSTI ID:2217500

^[1]; Francom, Devin ^[1]; Wiel, Scott Vander ^[1]

Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Dimension reduction techniques have long been an important topic in statistics, and active subspaces (AS) have received much attention this past decade in the computer experiments literature. The most common approach towards estimating the AS is to use Monte Carlo with numerical gradient evaluation. While sensible in some settings, this approach has obvious drawbacks. Recent research has demonstrated that active subspace calculations can be obtained in closed form, conditional on a Gaussian process (GP) surrogate, which can be limiting in high-dimensional settings for computational reasons. In this paper, we produce the relevant calculations for a more general case when the model of interest is a linear combination of tensor products. These general equations can be applied to the GP, recovering previous results as a special case, or applied to the models constructed by other regression techniques including multivariate adaptive regression splines (MARS). Furthermore, using a MARS surrogate has many advantages including improved scaling, better estimation of active subspaces in high dimensions and the ability to handle a large number of prior distributions in closed form. In one real-world example, we obtain the active subspace of a radiation-transport code with 240 inputs and 9,372 model runs in under half an hour.

View Accepted Manuscript (DOE)

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: 89233218CNA000001

OSTI ID:: 2217500

Report Number(s):: LA-UR--22-31726

Journal Information:: Journal of Computational and Graphical Statistics, Journal Name: Journal of Computational and Graphical Statistics Journal Issue: 3 Vol. 33; ISSN 1061-8600

Publisher:: Taylor & FrancisCopyright Statement

Country of Publication:: United States

Language:: English

References (50)

Discovering an active subspace in a single-diode solar cell model: Active Subspaces in Solar Cells Constantine, Paul G.; Zaharatos, Brian; Campanelli, Mark Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 8, Issue 5-6 https://doi.org/10.1002/sam.11281	journal	July 2015
Time‐dependent global sensitivity analysis with active subspaces for a lithium ion battery model Constantine, Paul G.; Doostan, Alireza Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 10, Issue 5 https://doi.org/10.1002/sam.11347	journal	August 2017
Multiple additive regression trees with application in epidemiology Friedman, Jerome H.; Meulman, Jacqueline J. Statistics in Medicine, Vol. 22, Issue 9 https://doi.org/10.1002/sim.1501	journal	January 2003
Space and Space-Time Modeling using Process Convolutions Higdon, Dave Quantitative Methods for Current Environmental Issues https://doi.org/10.1007/978-1-4471-0657-9_2	book	January 2002
Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods Tezzele, Marco; Ballarin, Francesco; Rozza, Gianluigi SEMA SIMAI Springer Series https://doi.org/10.1007/978-3-319-96649-6_8	book	January 2018
Neutronics of Advanced Nuclear Systems Wu, Yican https://doi.org/10.1007/978-981-13-6520-1	book	January 2019
Bayesian projection pursuit regression Collins, Gavin; Francom, Devin; Rumsey, Kellin Statistics and Computing, Vol. 34, Issue 1 https://doi.org/10.1007/s11222-023-10334-z	journal	November 2023
Why piecewise linear functions are dense in C[0, 1] Shekhtman, Boris Journal of Approximation Theory, Vol. 36, Issue 3 https://doi.org/10.1016/0021-9045(82)90045-4	journal	November 1982
Robust Bayesian analysis: sensitivity to the prior Berger, James O. Journal of Statistical Planning and Inference, Vol. 25, Issue 3 https://doi.org/10.1016/0378-3758(90)90079-A	journal	July 1990
Optimal Latin-hypercube designs for computer experiments Park, Jeong-Soo Journal of Statistical Planning and Inference, Vol. 39, Issue 1 https://doi.org/10.1016/0378-3758(94)90115-5	journal	April 1994
Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates Sobol′, I. M. Mathematics and Computers in Simulation, Vol. 55, Issue 1-3 https://doi.org/10.1016/S0378-4754(00)00270-6	journal	February 2001
Adaptive hinging hyperplanes and its applications in dynamic system identification Xu, Jun; Huang, Xiaolin; Wang, Shuning Automatica, Vol. 45, Issue 10 https://doi.org/10.1016/j.automatica.2009.06.013	journal	October 2009
Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model Jefferson, Jennifer L.; Gilbert, James M.; Constantine, Paul G. Computers & Geosciences, Vol. 83 https://doi.org/10.1016/j.cageo.2015.07.001	journal	October 2015
Surrogate assisted active subspace and active subspace assisted surrogate—A new paradigm for high dimensional structural reliability analysis N., Navaneeth; Chakraborty, Souvik Computer Methods in Applied Mechanics and Engineering, Vol. 389 https://doi.org/10.1016/j.cma.2021.114374	journal	February 2022
Active subspace-based dimension reduction for chemical kinetics applications with epistemic uncertainty Vohra, Manav; Alexanderian, Alen; Guy, Hayley Combustion and Flame, Vol. 204 https://doi.org/10.1016/j.combustflame.2019.03.006	journal	June 2019
A framework for expert-driven subpopulation discovery and evaluation using subspace clustering for epidemiological data Hielscher, Tommy; Niemann, Uli; Preim, Bernhard Expert Systems with Applications, Vol. 113 https://doi.org/10.1016/j.eswa.2018.07.003	journal	December 2018
Validation of the Preston–Tonks–Wallace strength model at strain rates approaching ∼1011 s−1 for Al-1100, tantalum and copper using hypervelocity impact crater morphologies Price, M. C.; Kearsley, A. T.; Burchell, M. J. International Journal of Impact Engineering, Vol. 52 https://doi.org/10.1016/j.ijimpeng.2012.09.001	journal	February 2013
Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification Tripathy, Rohit K.; Bilionis, Ilias Journal of Computational Physics, Vol. 375 https://doi.org/10.1016/j.jcp.2018.08.036	journal	December 2018
Inference for multivariate normal mixtures Chen, Jiahua; Tan, Xianming Journal of Multivariate Analysis, Vol. 100, Issue 7 https://doi.org/10.1016/j.jmva.2008.12.005	journal	August 2009
Quantifying kinetic uncertainty in turbulent combustion simulations using active subspaces Ji, Weiqi; Ren, Zhuyin; Marzouk, Youssef Proceedings of the Combustion Institute, Vol. 37, Issue 2 https://doi.org/10.1016/j.proci.2018.06.206	journal	January 2019
Global sensitivity metrics from active subspaces Constantine, Paul G.; Diaz, Paul Reliability Engineering & System Safety, Vol. 162 https://doi.org/10.1016/j.ress.2017.01.013	journal	June 2017
Kernel estimators of regression functions Bierens, Herman J.; Bewley, Truman F. Advances in Econometrics https://doi.org/10.1017/CCOL0521344301.003	book	January 2013
Bayesian MARS Denison, D. G. T.; Mallick, B. K.; Smith, A. F. M. Statistics and Computing, Vol. 8, Issue 4, p. 337-346 https://doi.org/10.1023/A:1008824606259	journal	December 1998
Model of plastic deformation for extreme loading conditions Preston, Dean L.; Tonks, Davis L.; Wallace, Duane C. Journal of Applied Physics, Vol. 93, Issue 1 https://doi.org/10.1063/1.1524706	journal	January 2003
Projection Pursuit Regression Friedman, Jerome H.; Stuetzle, Werner Journal of the American Statistical Association, Vol. 76, Issue 376 https://doi.org/10.1080/01621459.1981.10477729	journal	December 1981
Investigating Smooth Multiple Regression by the Method of Average Derivatives Härdle, Wolfgang; Stoker, Thomas M. Journal of the American Statistical Association, Vol. 84, Issue 408 https://doi.org/10.1080/01621459.1989.10478863	journal	December 1989
Sliced Inverse Regression for Dimension Reduction Li, Ker-Chau Journal of the American Statistical Association, Vol. 86, Issue 414 https://doi.org/10.1080/01621459.1991.10475035	journal	June 1991
Sequential Learning of Active Subspaces Wycoff, Nathan; Binois, Mickaël; Wild, Stefan M. Journal of Computational and Graphical Statistics, Vol. 30, Issue 4 https://doi.org/10.1080/10618600.2021.1874962	journal	March 2021
Sparse inverse covariance estimation with the graphical lasso Friedman, J.; Hastie, T.; Tibshirani, R. Biostatistics, Vol. 9, Issue 3 https://doi.org/10.1093/biostatistics/kxm045	journal	December 2007
Artificial neural networks for feature extraction and multivariate data projection No authors listed IEEE Transactions on Neural Networks, Vol. 6, Issue 2 https://doi.org/10.1109/72.363467	journal	March 1995
Canonical piecewise-linear approximations Lin, J. -N.; Unbehauen, R. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, Vol. 39, Issue 8 https://doi.org/10.1109/81.168933	journal	January 1992
Uncovering Active Structural Subspaces Associated with Changes in Indicators for Alzheimer’s Disease Batta, Ishaan; Abrol, Anees; Calhoun, Vince 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) https://doi.org/10.1109/EMBC46164.2021.9629930	conference	November 2021
Bayesian calibration of computer models Kennedy, Marc C.; O'Hagan, Anthony Journal of the Royal Statistical Society: Series B (Statistical Methodology), Vol. 63, Issue 3 https://doi.org/10.1111/1467-9868.00294	journal	August 2001
A Review on Dimension Reduction: A Review on Dimension Reduction Ma, Yanyuan; Zhu, Liping International Statistical Review, Vol. 81, Issue 1 https://doi.org/10.1111/j.1751-5823.2012.00182.x	journal	December 2012
Analytical Variance-Based Global Sensitivity Analysis in Simulation-Based Design Under Uncertainty Chen, Wei; Jin, Ruichen; Sudjianto, Agus Journal of Mechanical Design, Vol. 127, Issue 5 https://doi.org/10.1115/1.1904642	journal	January 2005
Turbomachinery Active Subspace Performance Maps Seshadri, Pranay; Shahpar, Shahrokh; Constantine, Paul Journal of Turbomachinery, Vol. 140, Issue 4 https://doi.org/10.1115/1.4038839	journal	January 2018
Active Subspaces Constantine, Paul G. Society for Industrial and Applied Mathematics https://doi.org/10.1137/1.9781611973860	book	January 2015
Mixture Densities, Maximum Likelihood and the EM Algorithm Redner, Richard A.; Walker, Homer F. SIAM Review, Vol. 26, Issue 2 https://doi.org/10.1137/1026034	journal	April 1984
Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces Constantine, Paul G.; Dow, Eric; Wang, Qiqi SIAM Journal on Scientific Computing, Vol. 36, Issue 4 https://doi.org/10.1137/130916138	journal	January 2014
Accelerating Markov Chain Monte Carlo with Active Subspaces Constantine, Paul G.; Kent, Carson; Bui-Thanh, Tan SIAM Journal on Scientific Computing, Vol. 38, Issue 5 https://doi.org/10.1137/15M1042127	journal	January 2016
Approximation and Radial-Basis-Function Networks Park, Jooyoung; Sandberg, Irwin W. Neural Computation, Vol. 5, Issue 2 https://doi.org/10.1162/neco.1993.5.2.305	journal	March 1993
A Multiple-Index Model and Dimension Reduction Xia, Yingcun Journal of the American Statistical Association, Vol. 103, Issue 484 https://doi.org/10.1198/016214508000000805	journal	December 2008
Surrogates Gramacy, Robert B. https://doi.org/10.1201/9780367815493	book	January 2020
Multivariate Adaptive Regression Splines Friedman, Jerome H. The Annals of Statistics, Vol. 19, Issue 1 https://doi.org/10.1214/aos/1176347963	journal	March 1991
BASS : An R Package for Fitting and Performing Sensitivity Analysis of Bayesian Adaptive Spline Surfaces Francom, Devin; Sansó, Bruno Journal of Statistical Software, Vol. 94, Issue 8 https://doi.org/10.18637/jss.v094.i08	journal	January 2020
hetGP: Heteroskedastic Gaussian Process Modeling and Sequential Design in R Binois, Mickaël; Gramacy, Robert B. Journal of Statistical Software, Vol. 98, Issue 13 https://doi.org/10.18637/jss.v098.i13	journal	January 2021
ENDF/B-VIIIrc1 239Pu Uncertainties: Constraints, Sampling, and Calibration to Jezebel Sigeti, David Edward; Parsons, Donald Kent; White, Morgan Curtis https://doi.org/10.2172/1425781	report	March 2018
Nuclear Data Dimension Reduction Francom, Devin Craig; Vander Wiel, Scott Alan; Weaver, Brian Phillip https://doi.org/10.2172/1561059	report	September 2019
Active Subspaces for Shape Optimization Lukaczyk, Trent W.; Constantine, Paul; Palacios, Francisco 10th AIAA Multidisciplinary Design Optimization Conference https://doi.org/10.2514/6.2014-1171	conference	January 2014
On Active Subspaces in Car Aerodynamics Othmer, Carsten; Lukaczyk, Trent W.; Constantine, Paul 17th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference https://doi.org/10.2514/6.2016-4294	conference	June 2016

Similar Records

Concordance

Software · Fri May 12 20:00:00 EDT 2023 · OSTI ID:code-107900

Sequential Learning of Active Subspaces

Journal Article · Sun Mar 07 19:00:00 EST 2021 · Journal of Computational and Graphical Statistics · OSTI ID:1845719

Related Subjects

97 MATHEMATICS AND COMPUTING
active subspaces
bayesian MARS
dimension reduction

Discovering Active Subspaces for High-Dimensional Computer Models

Citation Formats

References (50)

Similar Records

Related Subjects