Provable Convergence of Plug-and-Play Priors With MMSE Denoisers

Xu, Xiaojian; Sun, Yu; Liu, Jiaming; Wohlberg, Brendt Egon; Kamilov, Ulugbek S.

doi:10.1109/lsp.2020.3006390

Provable Convergence of Plug-and-Play Priors With MMSE Denoisers

Journal Article · Wed Jul 01 00:00:00 EDT 2020 · IEEE Signal Processing Letters

DOI:https://doi.org/10.1109/lsp.2020.3006390· OSTI ID:1726206

^[1]; ^[1]; ^[1]; ^[2]; ^[1]

Washington Univ., St. Louis, MO (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Plug-and-play priors (PnP) is a methodology for regularized image reconstruction that specifies the prior through an image denoiser. While PnP algorithms are well understood for denoisers performing maximum a posteriori probability (MAP) estimation, they have not been analyzed for the minimum mean squared error (MMSE) denoisers. Here we address this gap by establishing the first theoretical convergence result for the iterative shrinkage/thresholding algorithm (ISTA) variant of PnP for MMSE denoisers. We show that the iterates produced by PnP-ISTA with an MMSE denoiser converge to a stationary point of some global cost function. We validate our analysis on sparse signal recovery in compressive sensing by comparing two types of denoisers, namely the exact MMSE denoiser and the approximate MMSE denoiser obtained by training a deep neural net.

View Accepted Manuscript (DOE)

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: 89233218CNA000001

OSTI ID:: 1726206

Report Number(s):: LA-UR--20-23774

Journal Information:: IEEE Signal Processing Letters, Journal Name: IEEE Signal Processing Letters Vol. 27; ISSN 1070-9908

Publisher:: IEEE Signal Processing SocietyCopyright Statement

Country of Publication:: United States

Language:: English

References (40)

Anisotropic Spatiotemporal Regularization in Compressive Video Recovery by Adaptively Modeling the Residual Errors as Correlated Noise Eslahi, Nasser; Foi, Alessandro 2018 IEEE 13th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP) https://doi.org/10.1109/ivmspw.2018.8448455	conference	June 2018
Bayesian Denoising: From MAP to MMSE Using Consistent Cycle Spinning Kazerouni, Abbas; Kamilov, Ulugbek S.; Bostan, Emrah IEEE Signal Processing Letters, Vol. 20, Issue 3 https://doi.org/10.1109/lsp.2013.2242061	journal	March 2013
Image Denoising Methods. A New Nonlocal Principle Buades, A.; Coll, B.; Morel, J. M. SIAM Review, Vol. 52, Issue 1 https://doi.org/10.1137/090773908	journal	January 2010
Probabilistic Reconstruction in Compressed Sensing: Algorithms, Phase Diagrams, and Threshold Achieving Matrices Krzakala, Florent; Mézard, Marc; Sausset, François arXiv https://doi.org/10.48550/arxiv.1206.3953	text	January 2012
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning Mairal, Julien arXiv https://doi.org/10.48550/arxiv.1402.4419	preprint	January 2014
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, Ingrid; Defrise, Michel; De Mol, Christine arXiv https://doi.org/10.48550/arxiv.math/0307152	preprint	January 2003
An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Daubechies, I.; Defrise, M.; De Mol, C. Communications on Pure and Applied Mathematics, Vol. 57, Issue 11 https://doi.org/10.1002/cpa.20042	journal	January 2004
Introductory Lectures on Convex Optimization Nesterov, Yurii Applied Optimization https://doi.org/10.1007/978-1-4419-8853-9	book	January 2004
A Stochastic Successive Minimization Method for Nonsmooth Nonconvex Optimization with Applications to Transceiver Design in Wireless Communication Networks Razaviyayn, Meisam; Sanjabi, Maziar; Luo, Zhi-Quan Mathematical Programming, Vol. 157, Issue 2 https://doi.org/10.1007/s10107-016-1021-7	journal	May 2016
Message-passing algorithms for compressed sensing Donoho, David L.; Maleki, Arian; Montanari, Andrea Proceedings of the National Academy of Sciences, Vol. 106, Issue 45 https://doi.org/10.1073/pnas.0909892106	journal	November 2009
Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices Krzakala, Florent; Mézard, Marc; Sausset, Francois Journal of Statistical Mechanics: Theory and Experiment, Vol. 2012, Issue 08 https://doi.org/10.1088/1742-5468/2012/08/P08009	journal	August 2012
Learning Deep CNN Denoiser Prior for Image Restoration Zhang, Kai; Zuo, Wangmeng; Gu, Shuhang 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.300	conference	July 2017
Plug-and-Play priors for model based reconstruction Venkatakrishnan, Singanallur V.; Bouman, Charles A.; Wohlberg, Brendt 2013 IEEE Global Conference on Signal and Information Processing (GlobalSIP) https://doi.org/10.1109/GlobalSIP.2013.6737048	conference	December 2013
Regularized Fourier Ptychography Using an Online Plug-and-play Algorithm Sun, Yu; Xu, Shiqi; Li, Yunzhe ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) https://doi.org/10.1109/ICASSP.2019.8683057	conference	May 2019
Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems Meinhardt, Tim; Moeller, Michael; Hazirbas, Caner 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.198	conference	October 2017
Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling Xu, Xiaojian; Liu, Jiaming; Sun, Yu 2020 54th Asilomar Conference on Signals, Systems, and Computers https://doi.org/10.1109/IEEECONF51394.2020.9443410	conference	November 2020
Generalized approximate message passing for estimation with random linear mixing Rangan, Sundeep 2011 IEEE International Symposium on Information Theory Proceedings https://doi.org/10.1109/ISIT.2011.6033942	conference	July 2011
Anisotropic Spatiotemporal Regularization in Compressive Video Recovery by Adaptively Modeling the Residual Errors as Correlated Noise Eslahi, Nasser; Foi, Alessandro 2018 IEEE 13th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP) https://doi.org/10.1109/IVMSPW.2018.8448455	conference	June 2018
Bayesian Denoising: From MAP to MMSE Using Consistent Cycle Spinning Kazerouni, Abbas; Kamilov, Ulugbek S.; Bostan, Emrah IEEE Signal Processing Letters, Vol. 20, Issue 3 https://doi.org/10.1109/LSP.2013.2242061	journal	March 2013
Primal-Dual Plug-and-Play Image Restoration Ono, Shunsuke IEEE Signal Processing Letters, Vol. 24, Issue 8 https://doi.org/10.1109/LSP.2017.2710233	journal	August 2017
A Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems Kamilov, Ulugbek S.; Mansour, Hassan; Wohlberg, Brendt IEEE Signal Processing Letters, Vol. 24, Issue 12 https://doi.org/10.1109/LSP.2017.2763583	journal	December 2017
A New Recurrent Plug-and-Play Prior Based on the Multiple Self-Similarity Network Song, Guangxiao; Sun, Yu; Liu, Jiaming IEEE Signal Processing Letters, Vol. 27 https://doi.org/10.1109/LSP.2020.2977214	journal	January 2020
Plug-and-Play ISTA Converges With Kernel Denoisers Gavaskar, Ruturaj G.; Chaudhury, Kunal N. IEEE Signal Processing Letters, Vol. 27 https://doi.org/10.1109/LSP.2020.2986643	journal	January 2020
Scene-Adapted plug-and-play algorithm with convergence guarantees Teodoro, Afonso M.; Bioucas-Dias, Jose M.; Figueiredo, Mario A. T. 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP) https://doi.org/10.1109/MLSP.2017.8168194	conference	September 2017
Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation Sreehari, Suhas; Venkatakrishnan, Singanallur V.; Wohlberg, Brendt IEEE Transactions on Computational Imaging https://doi.org/10.1109/TCI.2016.2599778	journal	January 2016
Plug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications Chan, Stanley H.; Wang, Xiran; Elgendy, Omar A. IEEE Transactions on Computational Imaging, Vol. 3, Issue 1 https://doi.org/10.1109/TCI.2016.2629286	journal	March 2017
Performance Analysis of Plug-and-Play ADMM: A Graph Signal Processing Perspective Chan, Stanley H. IEEE Transactions on Computational Imaging, Vol. 5, Issue 2 https://doi.org/10.1109/TCI.2019.2892123	journal	June 2019
An Online Plug-and-Play Algorithm for Regularized Image Reconstruction Sun, Yu; Wohlberg, Brendt; Kamilov, Ulugbek S. IEEE Transactions on Computational Imaging, Vol. 5, Issue 3 https://doi.org/10.1109/TCI.2019.2893568	journal	September 2019
An EM algorithm for wavelet-based image restoration Figueiredo, M. A. T.; Nowak, R. D. IEEE Transactions on Image Processing, Vol. 12, Issue 8 https://doi.org/10.1109/TIP.2003.814255	journal	August 2003
Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems Beck, A.; Teboulle, M. IEEE Transactions on Image Processing, Vol. 18, Issue 11 https://doi.org/10.1109/TIP.2009.2028250	journal	November 2009
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising Zhang, Kai; Zuo, Wangmeng; Chen, Yunjin IEEE Transactions on Image Processing, Vol. 26, Issue 7 https://doi.org/10.1109/TIP.2017.2662206	journal	July 2017
A Convergent Image Fusion Algorithm Using Scene-Adapted Gaussian-Mixture-Based Denoising Teodoro, Afonso M.; Bioucas-Dias, Jose M.; Figueiredo, Mario A. T. IEEE Transactions on Image Processing, Vol. 28, Issue 1 https://doi.org/10.1109/TIP.2018.2869727	journal	January 2019
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information Candes, E.J.; Romberg, J.; Tao, T. IEEE Transactions on Information Theory, Vol. 52, Issue 2, p. 489-509 https://doi.org/10.1109/TIT.2005.862083	journal	February 2006
Compressed sensing Donoho, D. L. IEEE Transactions on Information Theory, Vol. 52, Issue 4 https://doi.org/10.1109/TIT.2006.871582	journal	April 2006
Denoising Prior Driven Deep Neural Network for Image Restoration Dong, Weisheng; Wang, Peiyao; Yin, Wotao IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, Issue 10 https://doi.org/10.1109/TPAMI.2018.2873610	journal	October 2019
Should Penalized Least Squares Regression be Interpreted as Maximum A Posteriori Estimation? Gribonval, Rémi IEEE Transactions on Signal Processing, Vol. 59, Issue 5 https://doi.org/10.1109/TSP.2011.2107908	journal	May 2011
Regression Shrinkage and Selection Via the Lasso Tibshirani, Robert Journal of the Royal Statistical Society: Series B (Methodological), Vol. 58, Issue 1 https://doi.org/10.1111/j.2517-6161.1996.tb02080.x	journal	January 1996
Proximal Thresholding Algorithm for Minimization over Orthonormal Bases Combettes, Patrick L.; Pesquet, Jean-Christophe SIAM Journal on Optimization, Vol. 18, Issue 4 https://doi.org/10.1137/060669498	journal	January 2008
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning Mairal, Julien SIAM Journal on Optimization, Vol. 25, Issue 2 https://doi.org/10.1137/140957639	journal	January 2015
Plug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium Buzzard, Gregery T.; Chan, Stanley H.; Sreehari, Suhas SIAM Journal on Imaging Sciences, Vol. 11, Issue 3 https://doi.org/10.1137/17M1122451	journal	January 2018

Similar Records

Boosting the Performance of Plug-and-Play Priors via Denoiser Scaling

Conference · Sun Nov 01 00:00:00 EDT 2020 · OSTI ID:1823749

Scalable Plug-and-Play ADMM with Convergence Guarantees

Journal Article · Thu Jul 01 20:00:00 EDT 2021 · IEEE Transactions on Computational Imaging · OSTI ID:1825405

An Online Plug-and-Play Algorithm for Regularized Image Reconstruction

Journal Article · Wed Jan 16 19:00:00 EST 2019 · IEEE Transactions on Computational Imaging · OSTI ID:1544682

Related Subjects

97 MATHEMATICS AND COMPUTING
MMSE estimation
computational imaging
deep learning
inverse problems
iterative shrinkage/thresholding algorithm (ISTA)
plug-and-play priors

Provable Convergence of Plug-and-Play Priors With MMSE Denoisers

Citation Formats

References (40)

Similar Records

Related Subjects