On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Dexter, Nick; Tran, Hoang; Webster, Clayton G.

doi:10.1007/s11228-021-00603-2

Title: On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Journal Article · 12 August 2021 · Set-valued and Variational Analysis

DOI: https://doi.org/10.1007/s11228-021-00603-2 · OSTI ID:1818754

^[1];

^[2]; Webster, Clayton G. ^[3]

Simon Fraser Univ., Burnaby, BC (Canada)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Texas, Austin, TX (United States)

We consider the problem of reconstructing an infinite set of sparse, finite-dimensional vectors, that share a common sparsity pattern, from incomplete measurements. This is in contrast to the work (Daubechies et al., Pure Appl. Math. 57(11), 1413–1457, 2004), where the single vector signal can be infinite-dimensional, and (Fornasier and Rauhut, SIAM J. Numer. Anal. 46(2), 577613, 2008), which extends the aforementioned work to the joint sparse recovery of finite number of infinite-dimensional vectors. In our case, to take account of the joint sparsity and promote the coupling of nonvanishing components, we employ a convex relaxation approach with mixed norm penalty ℓ_2,1. This paper discusses the computation of the solutions of linear inverse problems with such relaxation by a forward-backward splitting algorithm. However, since the solution matrix possesses infinitely many columns, the arguments of Daubechies et al. (Pure Appl. Math. 57(11), 1413–1457, 2004) no longer apply. As such, we establish new strong convergence results for the algorithm, in particular when the set of jointly sparse vectors is infinite.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC)

Grant/Contract Number:: AC02-05CH11231; AC05-00OR22725

OSTI ID:: 1818754

Journal Information:: Set-valued and Variational Analysis, Journal Name: Set-valued and Variational Analysis Journal Issue: 2 Vol. 30; ISSN 1877-0533

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (49)

Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms Gribonval, Rémi; Rauhut, Holger; Schnass, Karin Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6 https://doi.org/10.1007/s00041-008-9044-y	journal	October 2008
An iterative solution of a variational inequality for certain monotone operators in Hilbert space Bruck Jr., Ronald E. Bulletin of the American Mathematical Society, Vol. 81, Issue 5 https://doi.org/10.1090/s0002-9904-1975-13874-2	journal	January 1975
Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays Parvaresh, Farzad; Vikalo, Haris; Misra, Sidhant IEEE Journal of Selected Topics in Signal Processing, Vol. 2, Issue 3 https://doi.org/10.1109/jstsp.2008.924384	journal	June 2008
Empirical Bayes Estimation of a Sparse Vector of Gene Expression Changes Erickson, Stephen; Sabatti, Chiara Statistical Applications in Genetics and Molecular Biology, Vol. 4, Issue 1 https://doi.org/10.2202/1544-6115.1132	journal	January 2005
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, Moshe; Eldar, Yonina C. arXiv https://doi.org/10.48550/arxiv.0802.1311	text	January 2008
Robust Recovery of Signals From a Structured Union of Subspaces Eldar, Yonina C.; Mishali, Moshe arXiv https://doi.org/10.48550/arxiv.0807.4581	preprint	January 2008
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, I.; Defrise, M.; De Mol, C. Communications on Pure and Applied Mathematics, Vol. 57, Issue 11 https://doi.org/10.1002/cpa.20042	journal	January 2004
Convex Analysis and Monotone Operator Theory in Hilbert Spaces Bauschke, Heinz H.; Combettes, Patrick L. CMS Books in Mathematics https://doi.org/10.1007/978-1-4419-9467-7	book	December 2013
Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms Gribonval, Rémi; Rauhut, Holger; Schnass, Karin Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6 https://doi.org/10.1007/s00041-008-9044-y	journal	October 2008
Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients Tran, Hoang; Webster, Clayton G.; Zhang, Guannan Numerische Mathematik, Vol. 137, Issue 2 https://doi.org/10.1007/s00211-017-0878-6	journal	March 2017
Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods Attouch, Hedy; Bolte, Jérôme; Svaiter, Benar Fux Mathematical Programming, Vol. 137, Issue 1-2 https://doi.org/10.1007/s10107-011-0484-9	journal	August 2011
Infinite-Dimensional Compressed Sensing and Function Interpolation Adcock, Ben Foundations of Computational Mathematics, Vol. 18, Issue 3 https://doi.org/10.1007/s10208-017-9350-3	journal	April 2017
“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern? Nutini, Julie; Schmidt, Mark; Hare, Warren Optimization Letters, Vol. 13, Issue 4 https://doi.org/10.1007/s11590-018-1325-z	journal	September 2018
Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm Gorodnitsky, Irina F.; George, John S.; Rao, Bhaskar D. Electroencephalography and Clinical Neurophysiology, Vol. 95, Issue 4 https://doi.org/10.1016/0013-4694(95)00107-A	journal	October 1995
Reconstruction of jointly sparse vectors via manifold optimization Petrosyan, Armenak; Tran, Hoang; Webster, Clayton Applied Numerical Mathematics, Vol. 144 https://doi.org/10.1016/j.apnum.2019.05.022	journal	October 2019
Sparse Legendre expansions via ℓ1-minimization Rauhut, Holger; Ward, Rachel Journal of Approximation Theory, Vol. 164, Issue 5 https://doi.org/10.1016/j.jat.2012.01.008	journal	May 2012
Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs Chkifa, Abdellah; Cohen, Albert; Schwab, Christoph Journal de Mathématiques Pures et Appliquées, Vol. 103, Issue 2 https://doi.org/10.1016/j.matpur.2014.04.009	journal	February 2015
Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit Tropp, Joel A.; Gilbert, Anna C.; Strauss, Martin J. Signal Processing, Vol. 86, Issue 3 https://doi.org/10.1016/j.sigpro.2005.05.030	journal	March 2006
Algorithms for simultaneous sparse approximation. Part II: Convex relaxation Tropp, Joel A. Signal Processing, Vol. 86, Issue 3 https://doi.org/10.1016/j.sigpro.2005.05.031	journal	March 2006
A mixed ℓ ₁ regularization approach for sparse simultaneous approximation of parameterized PDEs Dexter, Nick; Tran, Hoang; Webster, Clayton ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 53, Issue 6 https://doi.org/10.1051/m2an/2019048	journal	November 2019
Solving monotone inclusions via compositions of nonexpansive averaged operators Combettes *, Patrick L. Optimization, Vol. 53, Issue 5-6 https://doi.org/10.1080/02331930412331327157	journal	October 2004
Convex programming in Hilbert space Goldstein, A. A. Bulletin of the American Mathematical Society, Vol. 70, Issue 5 https://doi.org/10.1090/S0002-9904-1964-11178-2	journal	January 1964
An iterative solution of a variational inequality for certain monotone operators in Hilbert space Bruck Jr., Ronald E. Bulletin of the American Mathematical Society, Vol. 81, Issue 5 https://doi.org/10.1090/S0002-9904-1975-13874-2	journal	January 1975
Polynomial approximation via compressed sensing of high-dimensional functions on lower sets Chkifa, Abdellah; Dexter, Nick; Tran, Hoang Mathematics of Computation, Vol. 87, Issue 311 https://doi.org/10.1090/mcom/3272	journal	September 2017
MEG-based imaging of focal neuronal current sources Phillips, J. W.; Leahy, R. M.; Mosher, J. C. IEEE Transactions on Medical Imaging, Vol. 16, Issue 3 https://doi.org/10.1109/42.585768	journal	June 1997
Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm Gorodnitsky, I. F.; Rao, B. D. IEEE Transactions on Signal Processing, Vol. 45, Issue 3 https://doi.org/10.1109/78.558475	journal	March 1997
Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays Parvaresh, Farzad; Vikalo, Haris; Misra, Sidhant IEEE Journal of Selected Topics in Signal Processing, Vol. 2, Issue 3 https://doi.org/10.1109/JSTSP.2008.924384	journal	June 2008
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Compressed sensing Donoho, D. L. IEEE Transactions on Information Theory, Vol. 52, Issue 4 https://doi.org/10.1109/TIT.2006.871582	journal	April 2006
Robust Recovery of Signals From a Structured Union of Subspaces Eldar, Yonina C.; Mishali, Moshe IEEE Transactions on Information Theory, Vol. 55, Issue 11 https://doi.org/10.1109/TIT.2009.2030471	journal	November 2009
Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation Eldar, Yonina C.; Rauhut, Holger IEEE Transactions on Information Theory, Vol. 56, Issue 1 https://doi.org/10.1109/TIT.2009.2034789	journal	January 2010
Model-Based Compressive Sensing Baraniuk, Richard G.; Cevher, Volkan; Duarte, Marco F. IEEE Transactions on Information Theory, Vol. 56, Issue 4 https://doi.org/10.1109/TIT.2010.2040894	journal	April 2010
Theoretical and Empirical Results for Recovery From Multiple Measurements van den Berg, Ewout; Friedlander, Michael P. IEEE Transactions on Information Theory, Vol. 56, Issue 5 https://doi.org/10.1109/TIT.2010.2043876	journal	May 2010
Rank Awareness in Joint Sparse Recovery Davies, Mike E.; Eldar, Yonina C. IEEE Transactions on Information Theory, Vol. 58, Issue 2 https://doi.org/10.1109/TIT.2011.2173722	journal	February 2012
Subspace Methods for Joint Sparse Recovery Lee, Kiryung; Bresler, Yoram; Junge, Marius IEEE Transactions on Information Theory, Vol. 58, Issue 6 https://doi.org/10.1109/TIT.2012.2189196	journal	June 2012
Theoretical Results on Sparse Representations of Multiple-Measurement Vectors Chen, Jie; Huo, Xiaoming IEEE Transactions on Signal Processing, Vol. 54, Issue 12 https://doi.org/10.1109/TSP.2006.881263	journal	January 2006
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, M.; Eldar, Y. C. IEEE Transactions on Signal Processing, Vol. 56, Issue 10 https://doi.org/10.1109/TSP.2008.927802	journal	October 2008
On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements Stojnic, M.; Parvaresh, F.; Hassibi, B. IEEE Transactions on Signal Processing, Vol. 57, Issue 8 https://doi.org/10.1109/TSP.2009.2020754	journal	August 2009
Block-Sparse Signals: Uncertainty Relations and Efficient Recovery Eldar, Yonina C.; Kuppinger, Patrick; Bolcskei, Helmut IEEE Transactions on Signal Processing, Vol. 58, Issue 6 https://doi.org/10.1109/TSP.2010.2044837	journal	June 2010
Group sparse optimization by alternating direction method Deng, Wei; Yin, Wotao; Zhang, Yin SPIE Optical Engineering + Applications, SPIE Proceedings https://doi.org/10.1117/12.2024410	conference	September 2013
Signal Recovery by Proximal Forward-Backward Splitting Combettes, Patrick L.; Wajs, Valérie R. Multiscale Modeling & Simulation, Vol. 4, Issue 4 https://doi.org/10.1137/050626090	journal	January 2005
Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints Fornasier, Massimo; Rauhut, Holger SIAM Journal on Numerical Analysis, Vol. 46, Issue 2 https://doi.org/10.1137/0606668909	journal	January 2008
Proximal Thresholding Algorithm for Minimization over Orthonormal Bases Combettes, Patrick L.; Pesquet, Jean-Christophe SIAM Journal on Optimization, Vol. 18, Issue 4 https://doi.org/10.1137/060669498	journal	January 2008
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence Hale, Elaine T.; Yin, Wotao; Zhang, Yin SIAM Journal on Optimization, Vol. 19, Issue 3 https://doi.org/10.1137/070698920	journal	January 2008
Activity Identification and Local Linear Convergence of Forward--Backward-type Methods Liang, Jingwei; Fadili, Jalal; Peyré, Gabriel SIAM Journal on Optimization, Vol. 27, Issue 1 https://doi.org/10.1137/16M106340X	journal	January 2017
Sensitivity Analysis for Mirror-Stratifiable Convex Functions Fadili, Jalal; Malick, Jérôme; Peyré, Gabriel SIAM Journal on Optimization, Vol. 28, Issue 4 https://doi.org/10.1137/17M113825X	journal	January 2018
Convergence Rates in Forward--Backward Splitting Chen, George H-G.; Rockafellar, R. T. SIAM Journal on Optimization, Vol. 7, Issue 2 https://doi.org/10.1137/S1052623495290179	journal	May 1997
Analytic Regularity and Polynomial Approximation of Parametric and Stochastic Elliptic Pde'S Cohen, Albert; Devore, Ronald; Schwab, Christoph Analysis and Applications, Vol. 09, Issue 01 https://doi.org/10.1142/S0219530511001728	journal	January 2011
Empirical Bayes Estimation of a Sparse Vector of Gene Expression Changes Erickson, Stephen; Sabatti, Chiara Statistical Applications in Genetics and Molecular Biology, Vol. 4, Issue 1 https://doi.org/10.2202/1544-6115.1132	journal	January 2005

Similar Records

Reconstruction of jointly sparse vectors via manifold optimization

Journal Article · 2019 · Applied Numerical Mathematics · OSTI ID:1559612

Petrosyan, Armenak; Tran, Hoang A.; Webster, Clayton G.

Crystal growth of pure substances: Phase-field simulations in comparison with analytical and experimental results

Journal Article · 2005 · Journal of Computational Physics · OSTI ID:20687245

Nestler, B; Danilov, D; Galenko, P

Weak and Strong Connectivity Regimes for a General Time Elapsed Neuron Network Model

Journal Article · 2018 · Journal of Statistical Physics · OSTI ID:22783692

Related Subjects

97 MATHEMATICS AND COMPUTING
compressed sensing
convex minimization
forward-backward splitting
incomplete data
infinite vectors
joint sparsity
linear inverse problems
mixed norm relaxation
parameterized PDEs
strong convergence

Title: On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Citation Formats

References (49)

Similar Records

Related Subjects