Inverse regression for ridge recovery: a data-driven approach for parameter reduction in computer experiments

Glaws, Andrew; Constantine, Paul G.; Cook, R. Dennis

doi:10.1007/s11222-019-09876-y

Title: Inverse regression for ridge recovery: a data-driven approach for parameter reduction in computer experiments

Journal Article · Fri May 31 00:00:00 EDT 2019 · Statistics and Computing

DOI:https://doi.org/10.1007/s11222-019-09876-y· OSTI ID:1802197

Glaws, Andrew ^[1]; Constantine, Paul G. ^[1]; Cook, R. Dennis ^[2]

Univ. of Colorado, Boulder, CO (United States)
Univ. of Minnesota, Minneapolis, MN (United States)

Parameter reduction can enable otherwise infeasible design and uncertainty studies with modern computational science models that contain several input parameters. In statistical regression, techniques for sufficient dimension reduction (SDR) use data to reduce the predictor dimension of a regression problem. A computational scientist hoping to use SDR for parameter reduction encounters a problem: a computer prediction is best represented by a deterministic function of the inputs, so data comprised of computer simulation queries fail to satisfy the SDR assumptions. To address this problem, here we interpret SDR methods sliced inverse regression (SIR) and sliced average variance estimation (SAVE) as estimating the directions of a ridge function, which is a composition of a low-dimensional linear transformation with a nonlinear function. Within this interpretation, SIR and SAVE estimate matrices of integrals whose column spaces are contained in the ridge directions’ span; we analyze and numerically verify convergence of these column spaces as the number of computer model queries increases. Moreover, we show example functions that are not ridge functions but whose inverse conditional moment matrices are low-rank. Consequently, the computational scientist should beware when using SIR and SAVE for parameter reduction, since SIR and SAVE may mistakenly suggest that truly important directions are unimportant.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Colorado School of Mines, Golden, CO (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0011077

OSTI ID:: 1802197

Journal Information:: Statistics and Computing, Vol. 30, Issue 2; ISSN 0960-3174

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (32)

Computing active subspaces efficiently with gradient sketching Constantine, Paul G.; Eftekhari, Armin; Wakin, Michael B. 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) https://doi.org/10.1109/CAMSAP.2015.7383809	conference	December 2015
Learning Functions of Few Arbitrary Linear Parameters in High Dimensions Fornasier, Massimo; Schnass, Karin; Vybiral, Jan Foundations of Computational Mathematics, Vol. 12, Issue 2 https://doi.org/10.1007/s10208-012-9115-y	journal	February 2012
Python Active-subspaces Utility Library Constantine, Paul; Howard, Ryan; Glaws, Andrew The Journal of Open Source Software, Vol. 1, Issue 5 https://doi.org/10.21105/joss.00079	journal	September 2016
Sliced inverse regression-based sparse polynomial chaos expansions for reliability analysis in high dimensions Pan, Qiujing; Dias, Daniel Reliability Engineering & System Safety, Vol. 167 https://doi.org/10.1016/j.ress.2017.06.026	journal	November 2017
Graphics for Regressions with a Binary Response Cook, R. Dennis Journal of the American Statistical Association, Vol. 91, Issue 435 https://doi.org/10.1080/01621459.1996.10476968	journal	September 1996
Capturing Ridge Functions in High Dimensions from Point Queries Cohen, Albert; Daubechies, Ingrid; DeVore, Ronald Constructive Approximation, Vol. 35, Issue 2 https://doi.org/10.1007/s00365-011-9147-6	journal	December 2011
The Elements of Statistical Learning Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome Springer Series in Statistics https://doi.org/10.1007/978-0-387-84858-7	book	January 2009
Review of Metamodeling Techniques in Support of Engineering Design Optimization Wang, G. Gary; Shan, S. Journal of Mechanical Design, Vol. 129, Issue 4 https://doi.org/10.1115/1.2429697	journal	May 2006
Active Subspaces Constantine, Paul G. Society for Industrial and Applied Mathematics https://doi.org/10.1137/1.9781611973860	book	January 2015
A near-stationary subspace for ridge approximation Constantine, Paul G.; Eftekhari, Armin; Hokanson, Jeffrey Computer Methods in Applied Mechanics and Engineering, Vol. 326 https://doi.org/10.1016/j.cma.2017.07.038	journal	November 2017
Sufficient dimension reduction and prediction in regression Adragni, Kofi P.; Cook, R. Dennis Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 367, Issue 1906 https://doi.org/10.1098/rsta.2009.0110	journal	November 2009
Sliced Inverse Regression for Dimension Reduction Li, Ker-Chau Journal of the American Statistical Association, Vol. 86, Issue 414 https://doi.org/10.1080/01621459.1991.10475035	journal	June 1991
Inverse regression-based uncertainty quantification algorithms for high-dimensional models: Theory and practice Li, Weixuan; Lin, Guang; Li, Bing Journal of Computational Physics, Vol. 321 https://doi.org/10.1016/j.jcp.2016.05.040	journal	September 2016
On the Interpretation of Regression Plots Cook, R. Dennis Journal of the American Statistical Association, Vol. 89, Issue 425 https://doi.org/10.1080/01621459.1994.10476459	journal	March 1994
The Design and Analysis of Computer Experiments Santner, Thomas J.; Williams, Brian J.; Notz, William I. Springer Series in Statistics https://doi.org/10.1007/978-1-4757-3799-8	book	January 2003
Learning non-parametric basis independent models from point queries via low-rank methods Tyagi, Hemant; Cevher, Volkan Applied and Computational Harmonic Analysis, Vol. 37, Issue 3 https://doi.org/10.1016/j.acha.2014.01.002	journal	November 2014
Successive direction extraction for estimating the central subspace in a multiple-index regression Yin, Xiangrong; Li, Bing; Cook, R. Dennis Journal of Multivariate Analysis, Vol. 99, Issue 8 https://doi.org/10.1016/j.jmva.2008.01.006	journal	September 2008
Efficient evaluation of small failure probability in high-dimensional groundwater contaminant transport modeling via a two-stage Monte Carlo method: FAILURE PROBABILITY Zhang, Jiangjiang; Li, Weixuan; Lin, Guang Water Resources Research, Vol. 53, Issue 3 https://doi.org/10.1002/2016WR019518	journal	March 2017
Surrogate Modeling for Uncertainty Assessment with Application to Aviation Environmental System Models Allaire, D.; Willcox, K. AIAA Journal, Vol. 48, Issue 8 https://doi.org/10.2514/1.J050247	journal	August 2010
A characterization of spherical distributions Eaton, Morris L. Journal of Multivariate Analysis, Vol. 20, Issue 2 https://doi.org/10.1016/0047-259X(86)90083-7	journal	December 1986
Projection Pursuit Regression Friedman, Jerome H.; Stuetzle, Werner Journal of the American Statistical Association, Vol. 76, Issue 376 https://doi.org/10.1080/01621459.1981.10477729	journal	December 1981
Regression Analysis Under Link Violation Li, Ker-Chau; Duan, Naihua The Annals of Statistics, Vol. 17, Issue 3 https://doi.org/10.1214/aos/1176347254	journal	September 1989
Design and Analysis of Computer Experiments Sacks, Jerome; Welch, William J.; Mitchell, Toby J. Statistical Science, Vol. 4, Issue 4 https://doi.org/10.1214/ss/1177012413	journal	November 1989
Review of surrogate modeling in water resources: REVIEW Razavi, Saman; Tolson, Bryan A.; Burn, Donald H. Water Resources Research, Vol. 48, Issue 7 https://doi.org/10.1029/2011WR011527	journal	July 2012
Gauss–Christoffel quadrature for inverse regression: applications to computer experiments Glaws, Andrew; Constantine, Paul G. Statistics and Computing, Vol. 29, Issue 3 https://doi.org/10.1007/s11222-018-9816-4	journal	June 2018
Conditioning as disintegration Chang, J. T.; Pollard, D. Statistica Neerlandica, Vol. 51, Issue 3 https://doi.org/10.1111/1467-9574.00056	journal	November 1997
Likelihood-Based Sufficient Dimension Reduction Cook, R. Dennis; Forzani, Liliana Journal of the American Statistical Association, Vol. 104, Issue 485 https://doi.org/10.1198/jasa.2009.0106	journal	March 2009
Save: a method for dimension reduction and graphics in regression Dennis Cook, R. Communications in Statistics - Theory and Methods, Vol. 29, Issue 9-10 https://doi.org/10.1080/03610920008832598	journal	January 2000
Magnetohydrodynamics Cowling, T. G.; Lindsay, R. B. Physics Today, Vol. 10, Issue 9 https://doi.org/10.1063/1.3060498	journal	September 1957
A Taxonomy of Global Optimization Methods Based on Response Surfaces Jones, Donald R. Journal of Global Optimization, Vol. 21, Issue 4, p. 345-383 https://doi.org/10.1023/A:1012771025575	journal	December 2001
Dimension reduction in magnetohydrodynamics power generation models: Dimensional analysis and active subspaces: GLAWS Glaws, Andrew; Constantine, Paul G.; Shadid, John N. Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 10, Issue 5 https://doi.org/10.1002/sam.11355	journal	August 2017
On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma Li, Ker-Chau Journal of the American Statistical Association, Vol. 87, Issue 420 https://doi.org/10.1080/01621459.1992.10476258	journal	December 1992

Similar Records

Inverse regression-based uncertainty quantification algorithms for high-dimensional models: Theory and practice

Journal Article · Thu Sep 01 00:00:00 EDT 2016 · Journal of Computational Physics · OSTI ID:1802197

Li, Weixuan; Lin, Guang; Li, Bing

Sliced-Inverse-Regression--Aided Rotated Compressive Sensing Method for Uncertainty Quantification

Journal Article · Mon Jan 01 00:00:00 EST 2018 · SIAM/ASA Journal on Uncertainty Quantification · OSTI ID:1802197

Yang, Xiu; Li, Weixuan; Tartakovsky, Alexandre

Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation

Journal Article · Thu Sep 15 00:00:00 EDT 2016 · Journal of Computational Physics · OSTI ID:1802197

Tripathy, Rohit; Bilionis, Ilias; Gonzalez, Marcial

Related Subjects

97 MATHEMATICS AND COMPUTING
computer science
mathematics
sufficient dimension reduction
ridge functions
ridge recovery

Title: Inverse regression for ridge recovery: a data-driven approach for parameter reduction in computer experiments

Citation Formats

References (32)

Similar Records

Related Subjects