On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Dexter, Nick; Tran, Hoang; Webster, Clayton G.

doi:10.1007/s11228-021-00603-2

Title: On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Journal Article · Fri Aug 13 00:00:00 EDT 2021 · Set-valued and Variational Analysis

DOI:https://doi.org/10.1007/s11228-021-00603-2· OSTI ID:1818754

^[1];

^[2]; Webster, Clayton G. ^[3]

Simon Fraser Univ., Burnaby, BC (Canada)
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Univ. of Texas, Austin, TX (United States)

We consider the problem of reconstructing an infinite set of sparse, finite-dimensional vectors, that share a common sparsity pattern, from incomplete measurements. This is in contrast to the work (Daubechies et al., Pure Appl. Math. 57(11), 1413–1457, 2004), where the single vector signal can be infinite-dimensional, and (Fornasier and Rauhut, SIAM J. Numer. Anal. 46(2), 577613, 2008), which extends the aforementioned work to the joint sparse recovery of finite number of infinite-dimensional vectors. In our case, to take account of the joint sparsity and promote the coupling of nonvanishing components, we employ a convex relaxation approach with mixed norm penalty ℓ_2,1. This paper discusses the computation of the solutions of linear inverse problems with such relaxation by a forward-backward splitting algorithm. However, since the solution matrix possesses infinitely many columns, the arguments of Daubechies et al. (Pure Appl. Math. 57(11), 1413–1457, 2004) no longer apply. As such, we establish new strong convergence results for the algorithm, in particular when the set of jointly sparse vectors is infinite.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR). Scientific Discovery through Advanced Computing (SciDAC); USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC05-00OR22725; AC02-05CH11231; ERKJ314; ERKJ331; ERKJ345

OSTI ID:: 1818754

Journal Information:: Set-valued and Variational Analysis, Vol. 30, Issue 2; ISSN 1877-0533

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (44)

Signal Recovery by Proximal Forward-Backward Splitting Combettes, Patrick L.; Wajs, Valérie R. Multiscale Modeling & Simulation, Vol. 4, Issue 4 https://doi.org/10.1137/050626090	journal	January 2005
Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm Gorodnitsky, Irina F.; George, John S.; Rao, Bhaskar D. Electroencephalography and Clinical Neurophysiology, Vol. 95, Issue 4 https://doi.org/10.1016/0013-4694(95)00107-A	journal	October 1995
On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements Stojnic, M.; Parvaresh, F.; Hassibi, B. IEEE Transactions on Signal Processing, Vol. 57, Issue 8 https://doi.org/10.1109/TSP.2009.2020754	journal	August 2009
Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints Fornasier, Massimo; Rauhut, Holger SIAM Journal on Numerical Analysis, Vol. 46, Issue 2 https://doi.org/10.1137/0606668909	journal	January 2008
Sparse Legendre expansions via <mml:math altimg="si12.gif" display="inline" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd"><mml:msub><mml:mrow><mml:mi>ℓ</mml:mi></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math>-minimization Rauhut, Holger; Ward, Rachel Journal of Approximation Theory, Vol. 164, Issue 5 https://doi.org/10.1016/j.jat.2012.01.008	journal	May 2012
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, I.; Defrise, M.; De Mol, C. Communications on Pure and Applied Mathematics, Vol. 57, Issue 11 https://doi.org/10.1002/cpa.20042	journal	January 2004
Solving monotone inclusions via compositions of nonexpansive averaged operators Combettes *, Patrick L. Optimization, Vol. 53, Issue 5-6 https://doi.org/10.1080/02331930412331327157	journal	October 2004
Robust Recovery of Signals From a Structured Union of Subspaces Eldar, Yonina C.; Mishali, Moshe IEEE Transactions on Information Theory, Vol. 55, Issue 11 https://doi.org/10.1109/TIT.2009.2030471	journal	November 2009
Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods Attouch, Hedy; Bolte, Jérôme; Svaiter, Benar Fux Mathematical Programming, Vol. 137, Issue 1-2 https://doi.org/10.1007/s10107-011-0484-9	journal	August 2011
Polynomial approximation via compressed sensing of high-dimensional functions on lower sets Chkifa, Abdellah; Dexter, Nick; Tran, Hoang Mathematics of Computation, Vol. 87, Issue 311 https://doi.org/10.1090/mcom/3272	journal	September 2017
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit Tropp, Joel A.; Gilbert, Anna C.; Strauss, Martin J. Signal Processing, Vol. 86, Issue 3 https://doi.org/10.1016/j.sigpro.2005.05.030	journal	March 2006
Group sparse optimization by alternating direction method Deng, Wei; Yin, Wotao; Zhang, Yin SPIE Optical Engineering + Applications, SPIE Proceedings https://doi.org/10.1117/12.2024410	conference	September 2013
Algorithms for simultaneous sparse approximation. Part II: Convex relaxation Tropp, Joel A. Signal Processing, Vol. 86, Issue 3 https://doi.org/10.1016/j.sigpro.2005.05.031	journal	March 2006
Convergence Rates in Forward--Backward Splitting Chen, George H-G.; Rockafellar, R. T. SIAM Journal on Optimization, Vol. 7, Issue 2 https://doi.org/10.1137/S1052623495290179	journal	May 1997
Infinite-Dimensional Compressed Sensing and Function Interpolation Adcock, Ben Foundations of Computational Mathematics, Vol. 18, Issue 3 https://doi.org/10.1007/s10208-017-9350-3	journal	April 2017
MEG-based imaging of focal neuronal current sources Phillips, J. W.; Leahy, R. M.; Mosher, J. C. IEEE Transactions on Medical Imaging, Vol. 16, Issue 3 https://doi.org/10.1109/42.585768	journal	June 1997
“Active-set complexity” of proximal gradient: How long does it take to find the sparsity pattern? Nutini, Julie; Schmidt, Mark; Hare, Warren Optimization Letters, Vol. 13, Issue 4 https://doi.org/10.1007/s11590-018-1325-z	journal	September 2018
Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients Tran, Hoang; Webster, Clayton G.; Zhang, Guannan Numerische Mathematik, Vol. 137, Issue 2 https://doi.org/10.1007/s00211-017-0878-6	journal	March 2017
Compressed sensing Donoho, D. L. IEEE Transactions on Information Theory, Vol. 52, Issue 4 https://doi.org/10.1109/TIT.2006.871582	journal	April 2006
Sensitivity Analysis for Mirror-Stratifiable Convex Functions Fadili, Jalal; Malick, Jérôme; Peyré, Gabriel SIAM Journal on Optimization, Vol. 28, Issue 4 https://doi.org/10.1137/17M113825X	journal	January 2018
Model-Based Compressive Sensing Baraniuk, Richard G.; Cevher, Volkan; Duarte, Marco F. IEEE Transactions on Information Theory, Vol. 56, Issue 4 https://doi.org/10.1109/TIT.2010.2040894	journal	April 2010
Analytic Regularity and Polynomial Approximation of Parametric and Stochastic Elliptic Pde'S Cohen, Albert; Devore, Ronald; Schwab, Christoph Analysis and Applications, Vol. 09, Issue 01 https://doi.org/10.1142/S0219530511001728	journal	January 2011
Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation Eldar, Yonina C.; Rauhut, Holger IEEE Transactions on Information Theory, Vol. 56, Issue 1 https://doi.org/10.1109/TIT.2009.2034789	journal	January 2010
Rank Awareness in Joint Sparse Recovery Davies, Mike E.; Eldar, Yonina C. IEEE Transactions on Information Theory, Vol. 58, Issue 2 https://doi.org/10.1109/TIT.2011.2173722	journal	February 2012
Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs Chkifa, Abdellah; Cohen, Albert; Schwab, Christoph Journal de Mathématiques Pures et Appliquées, Vol. 103, Issue 2 https://doi.org/10.1016/j.matpur.2014.04.009	journal	February 2015
Block-Sparse Signals: Uncertainty Relations and Efficient Recovery Eldar, Yonina C.; Kuppinger, Patrick; Bolcskei, Helmut IEEE Transactions on Signal Processing, Vol. 58, Issue 6 https://doi.org/10.1109/TSP.2010.2044837	journal	June 2010
Subspace Methods for Joint Sparse Recovery Lee, Kiryung; Bresler, Yoram; Junge, Marius IEEE Transactions on Information Theory, Vol. 58, Issue 6 https://doi.org/10.1109/TIT.2012.2189196	journal	June 2012
Proximal Thresholding Algorithm for Minimization over Orthonormal Bases Combettes, Patrick L.; Pesquet, Jean-Christophe SIAM Journal on Optimization, Vol. 18, Issue 4 https://doi.org/10.1137/060669498	journal	January 2008
Reconstruction of jointly sparse vectors via manifold optimization Petrosyan, Armenak; Tran, Hoang; Webster, Clayton Applied Numerical Mathematics, Vol. 144 https://doi.org/10.1016/j.apnum.2019.05.022	journal	October 2019
A mixed ℓ ₁ regularization approach for sparse simultaneous approximation of parameterized PDEs Dexter, Nick; Tran, Hoang; Webster, Clayton ESAIM: Mathematical Modelling and Numerical Analysis, Vol. 53, Issue 6 https://doi.org/10.1051/m2an/2019048	journal	November 2019
Convex programming in Hilbert space Goldstein, A. A. Bulletin of the American Mathematical Society, Vol. 70, Issue 5 https://doi.org/10.1090/S0002-9904-1964-11178-2	journal	January 1964
Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm Gorodnitsky, I. F.; Rao, B. D. IEEE Transactions on Signal Processing, Vol. 45, Issue 3 https://doi.org/10.1109/78.558475	journal	March 1997
Activity Identification and Local Linear Convergence of Forward--Backward-type Methods Liang, Jingwei; Fadili, Jalal; Peyré, Gabriel SIAM Journal on Optimization, Vol. 27, Issue 1 https://doi.org/10.1137/16M106340X	journal	January 2017
Fixed-Point Continuation for $\ell_1$-Minimization: Methodology and Convergence Hale, Elaine T.; Yin, Wotao; Zhang, Yin SIAM Journal on Optimization, Vol. 19, Issue 3 https://doi.org/10.1137/070698920	journal	January 2008
Empirical Bayes Estimation of a Sparse Vector of Gene Expression Changes Erickson, Stephen; Sabatti, Chiara Statistical Applications in Genetics and Molecular Biology, Vol. 4, Issue 1 https://doi.org/10.2202/1544-6115.1132	journal	January 2005
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, M.; Eldar, Y. C. IEEE Transactions on Signal Processing, Vol. 56, Issue 10 https://doi.org/10.1109/TSP.2008.927802	journal	October 2008
Theoretical and Empirical Results for Recovery From Multiple Measurements van den Berg, Ewout; Friedlander, Michael P. IEEE Transactions on Information Theory, Vol. 56, Issue 5 https://doi.org/10.1109/TIT.2010.2043876	journal	May 2010
Theoretical Results on Sparse Representations of Multiple-Measurement Vectors Chen, Jie; Huo, Xiaoming IEEE Transactions on Signal Processing, Vol. 54, Issue 12 https://doi.org/10.1109/TSP.2006.881263	journal	January 2006
Atoms of All Channels, Unite! Average Case Analysis of Multi-Channel Sparse Recovery Using Greedy Algorithms Gribonval, Rémi; Rauhut, Holger; Schnass, Karin Journal of Fourier Analysis and Applications, Vol. 14, Issue 5-6 https://doi.org/10.1007/s00041-008-9044-y	journal	October 2008
An iterative solution of a variational inequality for certain monotone operators in Hilbert space Bruck Jr., Ronald E. Bulletin of the American Mathematical Society, Vol. 81, Issue 5 https://doi.org/10.1090/s0002-9904-1975-13874-2	journal	January 1975
Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays Parvaresh, Farzad; Vikalo, Haris; Misra, Sidhant IEEE Journal of Selected Topics in Signal Processing, Vol. 2, Issue 3 https://doi.org/10.1109/jstsp.2008.924384	journal	June 2008
Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors Mishali, Moshe; Eldar, Yonina C. arXiv https://doi.org/10.48550/arxiv.0802.1311	text	January 2008
Robust Recovery of Signals From a Structured Union of Subspaces Eldar, Yonina C.; Mishali, Moshe arXiv https://doi.org/10.48550/arxiv.0807.4581	preprint	January 2008

Similar Records

A finite integration method for conformal, structured-grid, electromagnetic simulation

Journal Article · Sat Jun 10 00:00:00 EDT 2006 · Journal of Computational Physics · OSTI ID:1818754

Cooke, S J; Shtokhamer, R; Mondelli, A A; +1 more

On bipartite pure-state entanglement structure in terms of disentanglement

Journal Article · Fri Dec 15 00:00:00 EST 2006 · Journal of Mathematical Physics · OSTI ID:1818754

Herbut, Fedor

Analysis of sparse recovery for Legendre expansions using envelope bound

Journal Article · Thu Mar 10 00:00:00 EST 2022 · Numerical Methods for Partial Differential Equations · OSTI ID:1818754

Tran, Hoang; Webster, Clayton

Related Subjects

97 MATHEMATICS AND COMPUTING
compressed sensing
joint sparsity
strong convergence
convex minimization
incomplete data
infinite vectors
mixed norm relaxation
forward-backward splitting
linear inverse problems
parameterized PDEs

Title: On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

Citation Formats

References (44)

Similar Records

Related Subjects