Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Bian, Xiao; Krim, Hamid; Bronstein, Alex; Dai, Liyi

doi:10.1137/15M1030376

Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Journal Article · Tue Aug 09 04:00:00 EDT 2016 · SIAM Journal on Imaging Sciences

DOI:https://doi.org/10.1137/15M1030376· OSTI ID:1437433

Bian, Xiao ^[1]; Krim, Hamid ^[2]; Bronstein, Alex ^[3]; Dai, Liyi ^[4]

North Carolina State Univ., Raleigh, NC (United States). Department of Electrical Engineering; CNEC/NC State
North Carolina State Univ., Raleigh, NC (United States). Department of Electrical Engineering
Tel Aviv Univ., Ramat Aviv (Israel). School of Electrical Engineering
U.S. Army Research Offce, Research Triangle Park, NC (United States). Computing Sciences Division

Sparse models in dictionary learning have been successfully applied in a wide variety of machine learning and computer vision problems, and as a result have recently attracted increased research interest. Another interesting related problem based on linear equality constraints, namely the sparse null space (SNS) problem, first appeared in 1986 and has since inspired results on sparse basis pursuit. In this work, we investigate the relation between the SNS problem and the analysis dictionary learning (ADL) problem, and show that the SNS problem plays a central role, and may be utilized to solve dictionary learning problems. Moreover, we propose an efficient algorithm of sparse null space basis pursuit (SNS-BP) and extend it to a solution of ADL. Lastly, experimental results on numerical synthetic data and real-world data are further presented to validate the performance of our method.

Research Organization:: North Carolina State University, Raleigh, NC (United States). Consortium for Nonproliferation Enabling Capabilities (CNEC)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA), Office of Nonproliferation and Verification Research and Development (NA-22)

DOE Contract Number:: NA0002576

OSTI ID:: 1437433

Journal Information:: SIAM Journal on Imaging Sciences, Journal Name: SIAM Journal on Imaging Sciences Journal Issue: 3 Vol. 9; ISSN 1936-4954

Publisher:: Society for Industrial and Applied Mathematics (SIAM)

Country of Publication:: United States

Language:: English

References (21)

The Null Space Problem I. Complexity Coleman, Thomas F.; Pothen, Alex SIAM Journal on Algebraic Discrete Methods, Vol. 7, Issue 4 https://doi.org/10.1137/0607059	journal	October 1986
Filtering for texture classification: a comparative study Randen, T.; Husoy, J. H. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, Issue 4 https://doi.org/10.1109/34.761261	journal	April 1999
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation Aharon, M.; Elad, M.; Bruckstein, A. IEEE Transactions on Signal Processing, Vol. 54, Issue 11 https://doi.org/10.1109/TSP.2006.881199	journal	November 2006
On the Role of Sparse and Redundant Representations in Image Processing Elad, Michael; Figueiredo, Mário A. T.; Ma, Yi Proceedings of the IEEE, Vol. 98, Issue 6 https://doi.org/10.1109/JPROC.2009.2037655	journal	June 2010
Insights Into Analysis Operator Learning: From Patch-Based Sparse Models to Higher Order MRFs No authors listed IEEE Transactions on Image Processing, Vol. 23, Issue 3 https://doi.org/10.1109/TIP.2014.2299065	journal	March 2014
Stable signal recovery from incomplete and inaccurate measurements Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223 https://doi.org/10.1002/cpa.20124	journal	January 2006
Dictionary Learning Algorithms for Sparse Representation Kreutz-Delgado, Kenneth; Murray, Joseph F.; Rao, Bhaskar D. Neural Computation, Vol. 15, Issue 2, p. 349-396 https://doi.org/10.1162/089976603762552951	journal	February 2003
The Null Space Problem II. Algorithms Coleman, Thomas F.; Pothen, Alex SIAM Journal on Algebraic Discrete Methods, Vol. 8, Issue 4 https://doi.org/10.1137/0608045	journal	October 1987
Sparse null basis computations in structural optimization Pothen, Alex Numerische Mathematik, Vol. 55, Issue 5 https://doi.org/10.1007/BF01398913	journal	September 1989
Analysis Operator Learning and its Application to Image Reconstruction Hawe, S.; Kleinsteuber, M.; Diepold, K. IEEE Transactions on Image Processing, Vol. 22, Issue 6 https://doi.org/10.1109/TIP.2013.2246175	journal	June 2013
Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications Wen, Bihan; Ravishankar, Saiprasad; Bresler, Yoram International Journal of Computer Vision, Vol. 114, Issue 2-3 https://doi.org/10.1007/s11263-014-0761-1	journal	October 2014
Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling Yaghoobi, Mehrdad; Nam, Sangnam; Gribonval, Remi IEEE Transactions on Signal Processing, Vol. 61, Issue 9 https://doi.org/10.1109/TSP.2013.2250968	journal	May 2013
Nonlinear Dimensionality Reduction by Locally Linear Embedding Roweis, S. T. Science, Vol. 290, Issue 5500 https://doi.org/10.1126/science.290.5500.2323	journal	December 2000
Sparse and Redundant Representation Modeling—What Next? Elad, Michael IEEE Signal Processing Letters, Vol. 19, Issue 12 https://doi.org/10.1109/LSP.2012.2224655	journal	December 2012
Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data Donoho, D. L.; Grimes, C. Proceedings of the National Academy of Sciences, Vol. 100, Issue 10 https://doi.org/10.1073/pnas.1031596100	journal	April 2003
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries Elad, Michael; Aharon, Michal IEEE Transactions on Image Processing, Vol. 15, Issue 12 https://doi.org/10.1109/TIP.2006.881969	journal	January 2006
Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model Rubinstein, Ron; Peleg, Tomer; Elad, Michael IEEE Transactions on Signal Processing, Vol. 61, Issue 3 https://doi.org/10.1109/TSP.2012.2226445	journal	February 2013
The cosparse analysis model and algorithms Nam, S.; Davies, M. E.; Elad, M. Applied and Computational Harmonic Analysis, Vol. 34, Issue 1 https://doi.org/10.1016/j.acha.2012.03.006	journal	January 2013
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Computing a Sparse Basis for the Null Space Gilbert, John R.; Heath, Michael T. SIAM Journal on Algebraic Discrete Methods, Vol. 8, Issue 3 https://doi.org/10.1137/0608037	journal	July 1987
The restricted isometry property and its implications for compressed sensing Candès, Emmanuel J. Comptes Rendus Mathematique, Vol. 346, Issue 9-10, p. 589-592 https://doi.org/10.1016/j.crma.2008.03.014	journal	May 2008

Similar Records

Bi-sparsity pursuit: A paradigm for robust subspace recovery

Journal Article · Thu May 24 20:00:00 EDT 2018 · Signal Processing · OSTI ID:1614542

BI-sparsity pursuit for robust subspace recovery

Journal Article · Mon Aug 31 20:00:00 EDT 2015 · Conference: IEEE conference on Image Processing (ICIP) · OSTI ID:1438406

Dictionary Learning with Accumulator Neurons

Conference · Mon Aug 01 00:00:00 EDT 2022 · OSTI ID:1893845

Related Subjects

97 MATHEMATICS AND COMPUTING
dictionary learning
sparse coding
sparse null space problem
union of subspaces

Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Citation Formats

References (21)

Similar Records

Related Subjects