Bayesian projection pursuit regression

Journal Article · 03 November 2023 · Statistics and Computing

DOI: https://doi.org/10.1007/s11222-023-10334-z · OSTI ID:2280566

Collins, Gavin Quinn ^[1]; Francom, Devin Craig  ^[2]; Rumsey, Kellin Norman  ^[2]

---

1. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States); The Ohio State Univ., Columbus, OH (United States)
2. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of $$M$$ “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and $$M$$ is typically chosen via cross-validation. Here, we introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case ($$M$$=1), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions $$M$$. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.

• Research Organization: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
• Sponsoring Organization: USDOE National Nuclear Security Administration (NNSA); USDOE Laboratory Directed Research and Development (LDRD) Program; National Science Foundation (NSF)
• Grant/Contract Number: 89233218CNA000001
• OSTI ID: 2280566
• Report Number(s): LA-UR--22-21174
• Journal Information: Statistics and Computing, Journal Name: Statistics and Computing Journal Issue: 1 Vol. 34; ISSN 0960-3174
• Publisher: SpringerCopyright Statement
• Country of Publication: United States
• Language: English