A mixed $\ell_1$ regularization approach for sparse simultaneous approximation of parameterized PDEs

Webster, Clayton; Tran, Hoang; Dexter, Nick

doi:10.1051/m2an/2019048

Title: A mixed $$\ell_1$$ regularization approach for sparse simultaneous approximation of parameterized PDEs

Journal Article · Thu Jun 27 00:00:00 EDT 2019 · Mathematical Modelling and Numerical Analysis

DOI:https://doi.org/10.1051/m2an/2019048· OSTI ID:1564178

^[1];

^[2]; Dexter, Nick ^[3]

Univ. of Tennessee, Knoxville, TN (United States); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Tennessee, Knoxville, TN (United States); Simon Fraser Univ., Burnaby, BC (Canada)

We introduce and assess a novel sparse polynomial technique for the simultaneous approximation of parameterized partial differential equations (PDEs) with deterministic and stochastic inputs. Our approach treats the numerical solution as a jointly sparse reconstruction problem through the reformulation of the standard basis pursuit denoising, where the set of jointly sparse vectors is infinite. To achieve global reconstruction of sparse solutions to parameterized elliptic PDEs over both physical and parametric domains, we combine the standard measurement scheme developed for compressed sensing in the context of bounded orthonormal systems with a novel mixed-norm based $$\ell_1$$ regularization method that exploits both energy and sparsity. Moreover, we are able to prove that, with minimal sample complexity, error estimates comparable to the best $$s$$-term and quasi-optimal approximations are achievable, while requiring only {\em a priori} bounds on polynomial truncation error with respect to the energy norm.Finally, we perform extensive numerical experiments on several high-dimensional parameterized elliptic PDE models to demonstrate the superior recovery properties of the proposed approach.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: AC05-00OR22725

OSTI ID:: 1564178

Journal Information:: Mathematical Modelling and Numerical Analysis, Vol. 53, Issue 6; ISSN 0764-583X

Publisher:: EDP SciencesCopyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 4 works

Citation information provided by
Web of Science

References (81)

Signal Recovery by Proximal Forward-Backward Splitting Combettes, Patrick L.; Wajs, Valérie R. Multiscale Modeling & Simulation, Vol. 4, Issue 4 https://doi.org/10.1137/050626090	journal	January 2005
Sparse Legendre expansions via <mml:math altimg="si12.gif" display="inline" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msub><mml:mrow><mml:mi>ℓ</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math>-minimization Rauhut, Holger; Ward, Rachel Journal of Approximation Theory, Vol. 164, Issue 5 https://doi.org/10.1016/j.jat.2012.01.008	journal	May 2012
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, I.; Defrise, M.; De Mol, C. Communications on Pure and Applied Mathematics, Vol. 57, Issue 11 https://doi.org/10.1002/cpa.20042	journal	January 2004
Sparse adaptive Taylor approximation algorithms for parametric and stochastic elliptic PDEs Chkifa, Abdellah; Cohen, Albert; DeVore, Ronald ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 47, Issue 1 https://doi.org/10.1051/m2an/2012027	journal	November 2012
Error Forgetting of Bregman Iteration Yin, Wotao; Osher, Stanley Journal of Scientific Computing, Vol. 54, Issue 2-3 https://doi.org/10.1007/s10915-012-9616-5	journal	June 2012
Polynomial approximation via compressed sensing of high-dimensional functions on lower sets Chkifa, Abdellah; Dexter, Nick; Tran, Hoang Mathematics of Computation, Vol. 87, Issue 311 https://doi.org/10.1090/mcom/3272	journal	September 2017
On the Stability and Accuracy of Least Squares Approximations Cohen, Albert; Davenport, Mark A.; Leviatan, Dany Foundations of Computational Mathematics, Vol. 13, Issue 5 https://doi.org/10.1007/s10208-013-9142-3	journal	March 2013
A Generalized Sampling and Preconditioning Scheme for Sparse Approximation of Polynomial Chaos Expansions Jakeman, John D.; Narayan, Akil; Zhou, Tao SIAM Journal on Scientific Computing, Vol. 39, Issue 3 https://doi.org/10.1137/16M1063885	journal	January 2017
Bregman Iterative Algorithms for $\ell_1$-Minimization with Applications to Compressed Sensing Yin, Wotao; Osher, Stanley; Goldfarb, Donald SIAM Journal on Imaging Sciences, Vol. 1, Issue 1 https://doi.org/10.1137/070703983	journal	January 2008
Robustness to Unknown Error in Sparse Regularization Brugiapaglia, Simone; Adcock, Ben IEEE Transactions on Information Theory, Vol. 64, Issue 10 https://doi.org/10.1109/TIT.2017.2788445	journal	October 2018
A non-adapted sparse approximation of PDEs with stochastic inputs Doostan, Alireza; Owhadi, Houman Journal of Computational Physics, Vol. 230, Issue 8 https://doi.org/10.1016/j.jcp.2011.01.002	journal	April 2011
Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients Tran, Hoang; Webster, Clayton G.; Zhang, Guannan Numerische Mathematik, Vol. 137, Issue 2 https://doi.org/10.1007/s00211-017-0878-6	journal	March 2017
Compressed sensing Donoho, D. L. IEEE Transactions on Information Theory, Vol. 52, Issue 4 https://doi.org/10.1109/TIT.2006.871582	journal	April 2006
Sparse Tensor Galerkin Discretization of Parametric and Random Parabolic PDEs---Analytic Regularity and Generalized Polynomial Chaos Approximation Hoang, Viet Ha; Schwab, Christoph SIAM Journal on Mathematical Analysis, Vol. 45, Issue 5 https://doi.org/10.1137/100793682	journal	January 2013
Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation Eldar, Yonina C.; Rauhut, Holger IEEE Transactions on Information Theory, Vol. 56, Issue 1 https://doi.org/10.1109/TIT.2009.2034789	journal	January 2010
Sparse Adaptive Approximation of High Dimensional Parametric Initial Value Problems Hansen, Markus; Schwab, Christoph Vietnam Journal of Mathematics, Vol. 41, Issue 2 https://doi.org/10.1007/s10013-013-0011-9	journal	March 2013
Regularity and Generalized Polynomial Chaos Approximation of Parametric and Random Second-Order Hyperbolic Partial Differential Equations Hoang, Viet Ha; Schwab, Christoph Analysis and Applications, Vol. 10, Issue 03 https://doi.org/10.1142/S0219530512500145	journal	July 2012
The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming Bregman, L. M. USSR Computational Mathematics and Mathematical Physics, Vol. 7, Issue 3 https://doi.org/10.1016/0041-5553(67)90040-7	journal	January 1967
Modeling uncertainty in steady state diffusion problems via generalized polynomial chaos Xiu, Dongbin; Em Karniadakis, George Computer Methods in Applied Mechanics and Engineering, Vol. 191, Issue 43 https://doi.org/10.1016/S0045-7825(02)00421-8	journal	September 2002
Stochastic finite element methods for partial differential equations with random input data Gunzburger, Max D.; Webster, Clayton G.; Zhang, Guannan Acta Numerica, Vol. 23 https://doi.org/10.1017/S0962492914000075	journal	May 2014
Analytic regularity and nonlinear approximation of a class of parametric semilinear elliptic PDEs Hansen, Markus; Schwab, Christoph Mathematische Nachrichten, Vol. 286, Issue 8-9 https://doi.org/10.1002/mana.201100131	journal	December 2012
Sparse Tensor Discretization of Elliptic sPDEs Bieri, Marcel; Andreev, Roman; Schwab, Christoph SIAM Journal on Scientific Computing, Vol. 31, Issue 6 https://doi.org/10.1137/090749256	journal	January 2010
N-TERM WIENER CHAOS APPROXIMATION RATES FOR ELLIPTIC PDEs WITH LOGNORMAL GAUSSIAN RANDOM INPUTS Hoang, Viet Ha; Schwab, Christoph Mathematical Models and Methods in Applied Sciences, Vol. 24, Issue 04 https://doi.org/10.1142/S0218202513500681	journal	January 2014
Assessment of Collocation and Galerkin Approaches to Linear Diffusion Equations with Random data Tuminaro, Raymond S.; Phipps, Eric T.; Miller, Christopher W. International Journal for Uncertainty Quantification, Vol. 1, Issue 1 https://doi.org/10.1615/Int.J.UncertaintyQuantification.v1.i1.20	journal	January 2011
Compressive sensing Petrov-Galerkin approximation of high-dimensional parametric operator equations Rauhut, Holger; Schwab, Christoph Mathematics of Computation, Vol. 86, Issue 304 https://doi.org/10.1090/mcom/3113	journal	May 2016
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence Hale, Elaine T.; Yin, Wotao; Zhang, Yin SIAM Journal on Optimization, Vol. 19, Issue 3 https://doi.org/10.1137/070698920	journal	January 2008
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, M.; Eldar, Y. C. IEEE Transactions on Signal Processing, Vol. 56, Issue 10 https://doi.org/10.1109/TSP.2008.927802	journal	October 2008
Discrete least squares polynomial approximation with random evaluations − application to parametric and stochastic elliptic PDEs Chkifa, Abdellah; Cohen, Albert; Migliorati, Giovanni ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 49, Issue 3 https://doi.org/10.1051/m2an/2014050	journal	April 2015
On polynomial chaos expansion via gradient-enhanced ℓ1-minimization Peng, Ji; Hampton, Jerrad; Doostan, Alireza Journal of Computational Physics, Vol. 310 https://doi.org/10.1016/j.jcp.2015.12.049	journal	April 2016
Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms Gribonval, Rémi; Rauhut, Holger; Schnass, Karin Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6 https://doi.org/10.1007/s00041-008-9044-y	journal	October 2008
A Compressed Sensing Approach for Partial Differential Equations with Random Input Data Mathelin, L.; Gallivan, K. A. Communications in Computational Physics, Vol. 12, Issue 4 https://doi.org/10.4208/cicp.151110.090911a	journal	October 2012
Compressed Sensing With Cross Validation Ward, Rachel IEEE Transactions on Information Theory, Vol. 55, Issue 12 https://doi.org/10.1109/TIT.2009.2032712	journal	December 2009
Convergence Rates of Best N-term Galerkin Approximations for a Class of Elliptic sPDEs Cohen, Albert; DeVore, Ronald; Schwab, Christoph Foundations of Computational Mathematics, Vol. 10, Issue 6 https://doi.org/10.1007/s10208-010-9072-2	journal	July 2010
Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies Hampton, Jerrad; Doostan, Alireza Journal of Computational Physics, Vol. 280 https://doi.org/10.1016/j.jcp.2014.09.019	journal	January 2015
A dynamically adaptive sparse grids method for quasi-optimal interpolation of multidimensional functions Stoyanov, Miroslav K.; Webster, Clayton G. Computers & Mathematics with Applications, Vol. 71, Issue 11 https://doi.org/10.1016/j.camwa.2015.12.045	journal	June 2016
An Iterative Regularization Method for Total Variation-Based Image Restoration Osher, Stanley; Burger, Martin; Goldfarb, Donald Multiscale Modeling & Simulation, Vol. 4, Issue 2 https://doi.org/10.1137/040605412	journal	January 2005
Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients Todor, Radu Alexandru; Schwab, Christoph IMA Journal of Numerical Analysis, Vol. 27, Issue 2 https://doi.org/10.1093/imanum/drl025	journal	April 2007
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Fixed-Point Continuation Applied to Compressed Sensing: Implementation and Numerical Experiments Zhang, Elaine T. Hale Wotao Yin and Yin Journal of Computational Mathematics, Vol. 28, Issue 2 https://doi.org/10.4208/jcm.2009.10-m1007	journal	June 2010
Infinite-Dimensional Compressed Sensing and Function Interpolation Adcock, Ben Foundations of Computational Mathematics, Vol. 18, Issue 3 https://doi.org/10.1007/s10208-017-9350-3	journal	April 2017
Explicit cost bounds of stochastic Galerkin approximations for parameterized PDEs with random coefficients Dexter, Nick C.; Webster, Clayton G.; Zhang, Guannan Computers & Mathematics with Applications, Vol. 71, Issue 11 https://doi.org/10.1016/j.camwa.2015.12.005	journal	June 2016
A weighted -minimization approach for sparse polynomial chaos expansions Peng, Ji; Hampton, Jerrad; Doostan, Alireza Journal of Computational Physics, Vol. 267 https://doi.org/10.1016/j.jcp.2014.02.024	journal	June 2014
A Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data Nobile, F.; Tempone, R.; Webster, C. G. SIAM Journal on Numerical Analysis, Vol. 46, Issue 5 https://doi.org/10.1137/060663660	journal	January 2008
The Homogeneous Chaos Wiener, Norbert American Journal of Mathematics, Vol. 60, Issue 4 https://doi.org/10.2307/2371268	journal	October 1938
The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations Xiu, Dongbin; Karniadakis, George Em SIAM Journal on Scientific Computing, Vol. 24, Issue 2 https://doi.org/10.1137/S1064827501387826	journal	January 2002
Analytic Regularity and Polynomial Approximation of Parametric and Stochastic Elliptic Pde'S Cohen, Albert; Devore, Ronald; Schwab, Christoph Analysis and Applications, Vol. 09, Issue 01 https://doi.org/10.1142/S0219530511001728	journal	January 2011
An Anisotropic Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data Nobile, F.; Tempone, R.; Webster, C. G. SIAM Journal on Numerical Analysis, Vol. 46, Issue 5 https://doi.org/10.1137/070680540	journal	January 2008
Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics Le Maître, O. P.; Knio, Omar M. Scientific Computation https://doi.org/10.1007/978-90-481-3520-2	book	January 2010
Reweighted minimization method for stochastic elliptic differential equations Yang, Xiu; Karniadakis, George Em Journal of Computational Physics, Vol. 248 https://doi.org/10.1016/j.jcp.2013.04.004	journal	September 2013
Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs Chkifa, Abdellah; Cohen, Albert; Schwab, Christoph Journal de Mathématiques Pures et Appliquées, Vol. 103, Issue 2 https://doi.org/10.1016/j.matpur.2014.04.009	journal	February 2015
Convergence of quasi-optimal Stochastic Galerkin methods for a class of PDES with random coefficients Beck, Joakim; Nobile, Fabio; Tamellini, Lorenzo Computers & Mathematics with Applications, Vol. 67, Issue 4 https://doi.org/10.1016/j.camwa.2013.03.004	journal	March 2014
Infinite-Dimensional $$\ell ^1$$ ℓ 1 Minimization and Function Approximation from Pointwise Data Adcock, Ben Constructive Approximation, Vol. 45, Issue 3 https://doi.org/10.1007/s00365-017-9369-3	journal	February 2017
A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data Babuška, Ivo; Nobile, Fabio; Tempone, Raúl SIAM Journal on Numerical Analysis, Vol. 45, Issue 3 https://doi.org/10.1137/050645142	journal	January 2007
STOCHASTIC COLLOCATION ALGORITHMS USING l1-MINIMIZATION Yan, Liang; Guo, Ling; Xiu, Dongbin International Journal for Uncertainty Quantification, Vol. 2, Issue 3 https://doi.org/10.1615/Int.J.UncertaintyQuantification.2012003925	journal	January 2012
A gradient enhanced ℓ1-minimization for sparse approximation of polynomial chaos expansions Guo, Ling; Narayan, Akil; Zhou, Tao Journal of Computational Physics, Vol. 367 https://doi.org/10.1016/j.jcp.2018.04.026	journal	August 2018
On the Optimal Polynomial Approximation of Stochastic pdes by Galerkin and Collocation Methods Beck, Joakim; Tempone, Raul; Nobile, Fabio Mathematical Models and Methods in Applied Sciences, Vol. 22, Issue 09 https://doi.org/10.1142/S0218202512500236	journal	July 2012
The null space property for sparse recovery from multiple measurement vectors Lai, Ming-Jun; Liu, Yang Applied and Computational Harmonic Analysis, Vol. 30, Issue 3 https://doi.org/10.1016/j.acha.2010.11.002	journal	May 2011
Theoretical and Empirical Results for Recovery From Multiple Measurements van den Berg, Ewout; Friedlander, Michael P. IEEE Transactions on Information Theory, Vol. 56, Issue 5 https://doi.org/10.1109/TIT.2010.2043876	journal	May 2010
Sparse polynomial approximation of parametric elliptic PDEs. Part II: lognormal coefficients Bachmayr, Markus; Cohen, Albert; DeVore, Ronald ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 51, Issue 1 https://doi.org/10.1051/m2an/2016051	journal	December 2016
Nonlinear approximation DeVore, Ronald A. Acta Numerica, Vol. 7 https://doi.org/10.1017/S0962492900002816	journal	January 1998
Interpolation via weighted ℓ1 minimization Rauhut, Holger; Ward, Rachel Applied and Computational Harmonic Analysis, Vol. 40, Issue 2 https://doi.org/10.1016/j.acha.2015.02.003	journal	March 2016
A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data Babuška, Ivo; Nobile, Fabio; Tempone, Raúl SIAM Review, Vol. 52, Issue 2 https://doi.org/10.1137/100786356	journal	January 2010
Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs Albert, Cohen,; Abdellah, Chkifa,; Christoph, Schwab, ETH Zurich https://doi.org/10.3929/ethz-a-010388376	text	January 2013
Sparse adaptive approximation of high dimensional parametric initial value problems Hansen, M.; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010397660	text	January 2012
Sparse adaptive Taylor approximation algorithms for parametric and stochastic elliptic PDEs Abdellah, Chkifa,; Albert, Cohen,; Ronald, DeVore, ETH Zurich https://doi.org/10.3929/ethz-a-010402173	text	January 2011
N-term Wiener chaos approximation rates for elliptic PDEs with lognormal Gaussian random inputs Hoang, Viet Ha; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010393914	text	January 2013
Convergence rates of best N-term Galerkin approximations for a class of elliptic sPDEs Cohen, Albert; DeVore, Ronald; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010406289	text	January 2009
Convergence rates for sparse chaos approximations of elliptic problems with stochastic coefficients Tudor, Radu A.; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-b-000006046	text	January 2007
Compressive sensing Petrov-Galerkin approximation of high-dimensional parametric operator equations Rauhut, Holger; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010386168	text	January 2014
Analytic regularity and nonlinear approximation of a class of parametric semilinear elliptic PDEs Hansen, Markus; Schwab, Christoph ETH Zurich https://doi.org/10.3929/ethz-a-010403202	text	January 2011
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, Moshe; Eldar, Yonina C. arXiv https://doi.org/10.48550/arxiv.0802.1311	text	January 2008
Compressed Sensing with Cross Validation Ward, Rachel arXiv https://doi.org/10.48550/arxiv.0803.1845	preprint	January 2008
Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation Eldar, Yonina C.; Rauhut, Holger arXiv https://doi.org/10.48550/arxiv.0904.0494	preprint	January 2009
A non-adapted sparse approximation of PDEs with stochastic inputs Doostan, Alireza; Owhadi, Houman arXiv https://doi.org/10.48550/arxiv.1006.2151	text	January 2010
On the stability and accuracy of least squares approximations Cohen, Albert; Davenport, Mark A.; Leviatan, Dany arXiv https://doi.org/10.48550/arxiv.1111.4422	preprint	January 2011
Explicit cost bounds of stochastic Galerkin approximations for parameterized PDEs with random coefficients Dexter, Nick; Webster, Clayton; Zhang, Guannan arXiv https://doi.org/10.48550/arxiv.1507.05545	text	January 2015
Infinite-dimensional compressed sensing and function interpolation Adcock, Ben arXiv https://doi.org/10.48550/arxiv.1509.06073	preprint	January 2015
A gradient enhanced $\ell_1$-minimization for sparse approximation of polynomial chaos expansions Guo, Ling; Narayan, Akil; Zhou, Tao arXiv https://doi.org/10.48550/arxiv.1802.08837	text	January 2018
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, Ingrid; Defrise, Michel; De Mol, Christine arXiv https://doi.org/10.48550/arxiv.math/0307152	preprint	January 2003
Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information Candes, Emmanuel; Romberg, Justin; Tao, Terence arXiv https://doi.org/10.48550/arxiv.math/0409186	preprint	January 2004
Coherence motivated sampling and convergence analysis of least squares polynomial Chaos regression Hampton, Jerrad; Doostan, Alireza Computer Methods in Applied Mechanics and Engineering, Vol. 290 https://doi.org/10.1016/j.cma.2015.02.006	journal	June 2015