Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Bian, Xiao; Krim, Hamid; Bronstein, Alex; Dai, Liyi

doi:10.1137/15M1030376

Title: Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Journal Article · Tue Aug 09 00:00:00 EDT 2016 · SIAM Journal on Imaging Sciences

DOI:https://doi.org/10.1137/15M1030376· OSTI ID:1437433

Bian, Xiao ^[1]; Krim, Hamid ^[1]; Bronstein, Alex ^[2]; Dai, Liyi ^[3]

North Carolina State Univ., Raleigh, NC (United States). Department of Electrical Engineering
Tel Aviv Univ., Ramat Aviv (Israel). School of Electrical Engineering
U.S. Army Research Offce, Research Triangle Park, NC (United States). Computing Sciences Division

Sparse models in dictionary learning have been successfully applied in a wide variety of machine learning and computer vision problems, and as a result have recently attracted increased research interest. Another interesting related problem based on linear equality constraints, namely the sparse null space (SNS) problem, first appeared in 1986 and has since inspired results on sparse basis pursuit. In this work, we investigate the relation between the SNS problem and the analysis dictionary learning (ADL) problem, and show that the SNS problem plays a central role, and may be utilized to solve dictionary learning problems. Moreover, we propose an efficient algorithm of sparse null space basis pursuit (SNS-BP) and extend it to a solution of ADL. Lastly, experimental results on numerical synthetic data and real-world data are further presented to validate the performance of our method.

Cite

Export

Save

Research Organization:: North Carolina State University, Raleigh, NC (United States). Consortium for Nonproliferation Enabling Capabilities (CNEC)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA), Office of Nonproliferation and Verification Research and Development (NA-22)

DOE Contract Number:: NA0002576

OSTI ID:: 1437433

Journal Information:: SIAM Journal on Imaging Sciences, Vol. 9, Issue 3; ISSN 1936-4954

Publisher:: Society for Industrial and Applied Mathematics (SIAM)

Country of Publication:: United States

Language:: English

References (21)

Stable signal recovery from incomplete and inaccurate measurements Candès, Emmanuel J.; Romberg, Justin K.; Tao, Terence Communications on Pure and Applied Mathematics, Vol. 59, Issue 8, p. 1207-1223 https://doi.org/10.1002/cpa.20124	journal	January 2006
Sparse null basis computations in structural optimization Pothen, Alex Numerische Mathematik, Vol. 55, Issue 5 https://doi.org/10.1007/BF01398913	journal	September 1989
Analysis Operator Learning and its Application to Image Reconstruction Hawe, S.; Kleinsteuber, M.; Diepold, K. IEEE Transactions on Image Processing, Vol. 22, Issue 6 https://doi.org/10.1109/TIP.2013.2246175	journal	June 2013
Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling Yaghoobi, Mehrdad; Nam, Sangnam; Gribonval, Remi IEEE Transactions on Signal Processing, Vol. 61, Issue 9 https://doi.org/10.1109/TSP.2013.2250968	journal	May 2013
Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data Donoho, D. L.; Grimes, C. Proceedings of the National Academy of Sciences, Vol. 100, Issue 10 https://doi.org/10.1073/pnas.1031596100	journal	April 2003
Filtering for texture classification: a comparative study Randen, T.; Husoy, J. H. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 21, Issue 4 https://doi.org/10.1109/34.761261	journal	April 1999
The cosparse analysis model and algorithms Nam, S.; Davies, M. E.; Elad, M. Applied and Computational Harmonic Analysis, Vol. 34, Issue 1 https://doi.org/10.1016/j.acha.2012.03.006	journal	January 2013
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries Elad, Michael; Aharon, Michal IEEE Transactions on Image Processing, Vol. 15, Issue 12 https://doi.org/10.1109/TIP.2006.881969	journal	January 2006
Nonlinear Dimensionality Reduction by Locally Linear Embedding Roweis, S. T. Science, Vol. 290, Issue 5500 https://doi.org/10.1126/science.290.5500.2323	journal	December 2000
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation Aharon, M.; Elad, M.; Bruckstein, A. IEEE Transactions on Signal Processing, Vol. 54, Issue 11 https://doi.org/10.1109/TSP.2006.881199	journal	November 2006
Insights Into Analysis Operator Learning: From Patch-Based Sparse Models to Higher Order MRFs IEEE Transactions on Image Processing, Vol. 23, Issue 3 https://doi.org/10.1109/TIP.2014.2299065	journal	March 2014
Sparse and Redundant Representation Modeling—What Next? Elad, Michael IEEE Signal Processing Letters, Vol. 19, Issue 12 https://doi.org/10.1109/LSP.2012.2224655	journal	December 2012
The restricted isometry property and its implications for compressed sensing Candès, Emmanuel J. Comptes Rendus Mathematique, Vol. 346, Issue 9-10, p. 589-592 https://doi.org/10.1016/j.crma.2008.03.014	journal	May 2008
On the Role of Sparse and Redundant Representations in Image Processing Elad, Michael; Figueiredo, Mário A. T.; Ma, Yi Proceedings of the IEEE, Vol. 98, Issue 6 https://doi.org/10.1109/JPROC.2009.2037655	journal	June 2010
Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model Rubinstein, Ron; Peleg, Tomer; Elad, Michael IEEE Transactions on Signal Processing, Vol. 61, Issue 3 https://doi.org/10.1109/TSP.2012.2226445	journal	February 2013
The Null Space Problem II. Algorithms Coleman, Thomas F.; Pothen, Alex SIAM Journal on Algebraic Discrete Methods, Vol. 8, Issue 4 https://doi.org/10.1137/0608045	journal	October 1987
Dictionary Learning Algorithms for Sparse Representation Kreutz-Delgado, Kenneth; Murray, Joseph F.; Rao, Bhaskar D. Neural Computation, Vol. 15, Issue 2, p. 349-396 https://doi.org/10.1162/089976603762552951	journal	February 2003
Computing a Sparse Basis for the Null Space Gilbert, John R.; Heath, Michael T. SIAM Journal on Algebraic Discrete Methods, Vol. 8, Issue 3 https://doi.org/10.1137/0608037	journal	July 1987
The Null Space Problem I. Complexity Coleman, Thomas F.; Pothen, Alex SIAM Journal on Algebraic Discrete Methods, Vol. 7, Issue 4 https://doi.org/10.1137/0607059	journal	October 1986
Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications Wen, Bihan; Ravishankar, Saiprasad; Bresler, Yoram International Journal of Computer Vision, Vol. 114, Issue 2-3 https://doi.org/10.1007/s11263-014-0761-1	journal	October 2014

Similar Records

Land cover classification in multispectral imagery using clustering of sparse approximations over learned feature dictionaries

Journal Article · Tue Dec 09 00:00:00 EST 2014 · Journal of Applied Remote Sensing · OSTI ID:1437433

Moody, Daniela I.; Brumby, Steven P.; Rowland, Joel C.; +1 more

Change detection and classification of land cover in multispectral satellite imagery using clustering of sparse approximations (CoSA) over learned feature dictionaries

Journal Article · Wed Oct 01 00:00:00 EDT 2014 · IEEE AIPR Conference proceedings · OSTI ID:1437433

Moody, Daniela I.; Brumby, Steven P.; Rowland, Joel C.; +2 more

WE-G-18A-04: 3D Dictionary Learning Based Statistical Iterative Reconstruction for Low-Dose Cone Beam CT Imaging

Journal Article · Sun Jun 15 00:00:00 EDT 2014 · Medical Physics · OSTI ID:1437433

Bai, T; Yan, H; Shi, F; +5 more

Related Subjects

97 MATHEMATICS AND COMPUTING
dictionary learning
sparse coding
sparse null space problem
union of subspaces

Title: Sparsity and Nullity: Paradigms for Analysis Dictionary Learning

Citation Formats

References (21)

Similar Records

Related Subjects