Sparse Overcomplete Denoising: Aggregation Versus Global Optimization

Carrera, Diego; Boracchi, Giacomo; Foi, Alessandro; Wohlberg, Brendt Egon

doi:10.1109/LSP.2017.2734119

Title: Sparse Overcomplete Denoising: Aggregation Versus Global Optimization

Abstract

Denoising is often addressed via sparse coding with respect to an overcomplete dictionary. There are two main approaches when the dictionary is composed of translates of an orthonormal basis. The first, traditionally employed by techniques such as wavelet cycle spinning, separately seeks sparsity w.r.t. each translate of the orthonormal basis, solving multiple partial optimizations and obtaining a collection of sparse approximations of the noise-free image, which are aggregated together to obtain a final estimate. The second approach, recently employed by convolutional sparse representations, instead seeks sparsity over the entire dictionary via a global optimization. It is tempting to view the former approach as providing a suboptimal solution of the latter. In this letter, we analyze whether global sparsity is a desirable property, and under what conditions the global optimization provides a better solution to the denoising problem. In particular, our experimental analysis shows that the two approaches attain comparable performance in case of natural images and global optimization outperforms the simpler aggregation of partial estimates only when the image admits an extremely sparse representation. Here, we explain this phenomenon by separately studying the bias and variance of these solutions, and by noting that the variance of the global solution increasesmore »« less

Authors:

Carrera, Diego ^[1]; Boracchi, Giacomo ^[1]; Foi, Alessandro ^[2];

^[3]

Politecnico di Milano, Milano (Italy)
Tampere Univ. of Technology, Tampere (Finland)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Publication Date:: Mon Jul 31 00:00:00 EDT 2017

Research Org.:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE Laboratory Directed Research and Development (LDRD) Program

OSTI Identifier:: 1467204

Report Number(s):: LA-UR-17-23531
Journal ID: ISSN 1070-9908

Grant/Contract Number:: AC52-06NA25396

Resource Type:: Accepted Manuscript

Journal Name:: IEEE Signal Processing Letters

Additional Journal Information:: Journal Volume: 24; Journal Issue: 10; Journal ID: ISSN 1070-9908

Publisher:: IEEE Signal Processing Society

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Computer Science; Information Science; Mathematics; Convolutional sparse coding; denoising; overcomplete representations; sparse representations

Citation Formats


                    Carrera, Diego, Boracchi, Giacomo, Foi, Alessandro, and Wohlberg, Brendt Egon. Sparse Overcomplete Denoising: Aggregation Versus Global Optimization.  United States: N. p., 2017. 
Web.  doi:10.1109/LSP.2017.2734119.

Copy to clipboard


                    Carrera, Diego, Boracchi, Giacomo, Foi, Alessandro, & Wohlberg, Brendt Egon. Sparse Overcomplete Denoising: Aggregation Versus Global Optimization.  United States.  https://doi.org/10.1109/LSP.2017.2734119

Copy to clipboard


                    Carrera, Diego, Boracchi, Giacomo, Foi, Alessandro, and Wohlberg, Brendt Egon. Mon .  
"Sparse Overcomplete Denoising: Aggregation Versus Global Optimization".  United States.  https://doi.org/10.1109/LSP.2017.2734119.  https://www.osti.gov/servlets/purl/1467204.

Copy to clipboard


                    
@article{osti_1467204,

  title        = {Sparse Overcomplete Denoising: Aggregation Versus Global Optimization},

  author       = {Carrera, Diego and Boracchi, Giacomo and Foi, Alessandro and Wohlberg, Brendt Egon},

  abstractNote = {Denoising is often addressed via sparse coding with respect to an overcomplete dictionary. There are two main approaches when the dictionary is composed of translates of an orthonormal basis. The first, traditionally employed by techniques such as wavelet cycle spinning, separately seeks sparsity w.r.t. each translate of the orthonormal basis, solving multiple partial optimizations and obtaining a collection of sparse approximations of the noise-free image, which are aggregated together to obtain a final estimate. The second approach, recently employed by convolutional sparse representations, instead seeks sparsity over the entire dictionary via a global optimization. It is tempting to view the former approach as providing a suboptimal solution of the latter. In this letter, we analyze whether global sparsity is a desirable property, and under what conditions the global optimization provides a better solution to the denoising problem. In particular, our experimental analysis shows that the two approaches attain comparable performance in case of natural images and global optimization outperforms the simpler aggregation of partial estimates only when the image admits an extremely sparse representation. Here, we explain this phenomenon by separately studying the bias and variance of these solutions, and by noting that the variance of the global solution increases very rapidly as the original signal becomes less and less sparse.},

  doi          = {10.1109/LSP.2017.2734119},

  journal      = {IEEE Signal Processing Letters},

  number       = 10,

  volume       = 24,

  place        = {United States},

  year         = {Mon Jul 31 00:00:00 EDT 2017},

  month        = {Mon Jul 31 00:00:00 EDT 2017}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/LSP.2017.2734119

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 6 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Works referencing / citing this record:

A Unified View on Patch Aggregation
journal, November 2019

Saint-Dizier, Alexandre; Delon, Julie; Bouveyron, Charles
Journal of Mathematical Imaging and Vision, Vol. 62, Issue 2
DOI: 10.1007/s10851-019-00921-z

Similar Records in DOE PAGES and OSTI.GOV collections:

Bayesian K-SVD Using Fast Variational Inference

Journal Article Serra, Juan G. ; Testa, Matteo ; Molina, Rafael ; ... - IEEE Transactions on Image Processing

Recent work in signal processing in general and image processing in particular deals with sparse representation related problems. Two such problems are of paramount importance: an overriding need for designing a well-suited overcomplete dictionary containing a redundant set of atoms— i.e., basis signals—and how to find a sparse representation of a given signal with respect to the chosen dictionary. Dictionary learning techniques, among which we find the popular K-singular value decomposition algorithm, tackle these problems by adapting a dictionary to a set of training data. A common drawback of such techniques is the need for parameter-tuning. In order to overcomemore »« less
https://doi.org/10.1109/TIP.2017.2681436
Image fusion using sparse overcomplete feature dictionaries

Patent Brumby, Steven P. ; Bettencourt, Luis ; Kenyon, Garrett T. ; ...

Approaches for deciding what individuals in a population of visual system "neurons" are looking for using sparse overcomplete feature dictionaries are provided. A sparse overcomplete feature dictionary may be learned for an image dataset and a local sparse representation of the image dataset may be built using the learned feature dictionary. A local maximum pooling operation may be applied on the local sparse representation to produce a translation-tolerant representation of the image dataset. An object may then be classified and/or clustered within the translation-tolerant representation of the image dataset using a supervised classification algorithm and/or an unsupervised clustering algorithm.
Full Text Available
Denoising Poisson phaseless measurements via orthogonal dictionary learning

Journal Article Chang, Huibin ; Marchesini, Stefano - Optics Express

Phaseless diffraction measurements recorded by CCD detectors are often affected by Poisson noise. In this paper, we propose a dictionary learning model by employing patches based sparsity in order to denoise such Poisson phaseless measurements. The model consists of three terms: (i) A representation term by an orthogonal dictionary, (ii) an L⁰ pseudo norm of the coefficient matrix, and (iii) a Kullback-Leibler divergence term to fit phaseless Poisson data. Fast alternating minimization method (AMM) and proximal alternating linearized minimization (PALM) are adopted to solve the proposed model, and especially the theoretical guarantee of the convergence of PALM is provided. Themore »« less
Cited by 7
https://doi.org/10.1364/OE.26.019773

Full Text Available
A mixed $$\ell_1$$ regularization approach for sparse simultaneous approximation of parameterized PDEs

Journal Article Webster, Clayton ; Tran, Hoang ; Dexter, Nick - Mathematical Modelling and Numerical Analysis

We introduce and assess a novel sparse polynomial technique for the simultaneous approximation of parameterized partial differential equations (PDEs) with deterministic and stochastic inputs. Our approach treats the numerical solution as a jointly sparse reconstruction problem through the reformulation of the standard basis pursuit denoising, where the set of jointly sparse vectors is infinite. To achieve global reconstruction of sparse solutions to parameterized elliptic PDEs over both physical and parametric domains, we combine the standard measurement scheme developed for compressed sensing in the context of bounded orthonormal systems with a novel mixed-norm basedmore »« less
Cited by 4
https://doi.org/10.1051/m2an/2019048

Full Text Available
Dictionary construction in sparse methods for image restoration

Journal Article Wohlberg, Brendt

Sparsity-based methods have achieved very good performance in a wide variety of image restoration problems, including denoising, inpainting, super-resolution, and source separation. These methods are based on the assumption that the image to be reconstructed may be represented as a superposition of a few known components, and the appropriate linear combination of components is estimated by solving an optimization such as Basis Pursuit De-Noising (BPDN). Considering that the K-SVD constructs a dictionary which has been optimised for mean performance over a training set, it is not too surprising that better performance can be achieved by selecting a custom dictionary formore »« less
Full Text Available

Similar Records

Title: Sparse Overcomplete Denoising: Aggregation Versus Global Optimization

Abstract

Citation Formats

A Unified View on Patch Aggregation journal, November 2019

A Unified View on Patch Aggregation
journal, November 2019