Paired autoencoders for likelihood-free estimation in inverse problems

Chung, Matthias; Hart, Emma; Chung, Julianne; Peters, Bas; Haber, Eldad

doi:10.1088/2632-2153/ad95dd

Title: Paired autoencoders for likelihood-free estimation in inverse problems

Journal Article · 04 December 2024 · Machine Learning: Science and Technology

DOI: https://doi.org/10.1088/2632-2153/ad95dd · OSTI ID:2479795

;

; Haber, Eldad

Abstract We consider the solution of nonlinear inverse problems where the forward problem is a discretization of a partial differential equation. Such problems are notoriously difficult to solve in practice and require minimizing a combination of a data-fit term and a regularization term. The main computational bottleneck of typical algorithms is the direct estimation of the data misfit. Therefore, likelihood-free approaches have become appealing alternatives. Nonetheless, difficulties in generalization and limitations in accuracy have hindered their broader utility and applicability. In this work, we use a paired autoencoder framework as a likelihood-free estimator (LFE) for inverse problems. We show that the use of such an architecture allows us to construct a solution efficiently and to overcome some known open problems when using LFEs. In particular, our framework can assess the quality of the solution and improve on it if needed. We demonstrate the viability of our approach using examples from full waveform inversion and inverse electromagnetic imaging.

View Journal Article

Cite

Export

Save

Sponsoring Organization:: USDOE

OSTI ID:: 2479795

Journal Information:: Machine Learning: Science and Technology, Journal Name: Machine Learning: Science and Technology Journal Issue: 4 Vol. 5; ISSN 2632-2153

Publisher:: IOP PublishingCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (39)

A variable projection method for large-scale inverse problems with ℓ1 regularization Chung, Matthias; Renaut, Rosemary A. Applied Numerical Mathematics, Vol. 192 https://doi.org/10.1016/j.apnum.2023.06.015	journal	October 2023
Fourier-DeepONet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness Zhu, Min; Feng, Shihang; Lin, Youzuo Computer Methods in Applied Mechanics and Engineering, Vol. 416 https://doi.org/10.1016/j.cma.2023.116300	journal	November 2023
Solving electrical impedance tomography with deep learning Fan, Yuwei; Ying, Lexing Journal of Computational Physics, Vol. 404 https://doi.org/10.1016/j.jcp.2019.109119	journal	March 2020
Deep learning methods for solving linear inverse problems: Research directions and paradigms Bai, Yanna; Chen, Wei; Chen, Jie Signal Processing, Vol. 177 https://doi.org/10.1016/j.sigpro.2020.107729	journal	December 2020
Learned Regularizations for Multi‐Parameter Elastic Full Waveform Inversion Using Diffusion Models Taufik, Mohammad H.; Wang, Fu; Alkhalifah, Tariq Journal of Geophysical Research: Machine Learning and Computation, Vol. 1, Issue 1 https://doi.org/10.1029/2024JH000125	journal	March 2024
Novelty detection and neural network validation Bishop, C. M. IEE Proceedings - Vision, Image, and Signal Processing, Vol. 141, Issue 4 https://doi.org/10.1049/ip-vis:19941330	journal	January 1994
Likelihood-Free Parameter Estimation with Neural Bayes Estimators Sainsbury-Dale, Matthew; Zammit-Mangion, Andrew; Huser, Raphaël The American Statistician https://doi.org/10.1080/00031305.2023.2249522	journal	August 2023
Learning regularization functionals a supervised training approach Haber, E.; Tenorio, L. Inverse Problems, Vol. 19, Issue 3 https://doi.org/10.1088/0266-5611/19/3/309	journal	April 2003
Data analysis tools for uncertainty quantification of inverse problems Tenorio, L.; Andersson, F.; de Hoop, M. Inverse Problems, Vol. 27, Issue 4 https://doi.org/10.1088/0266-5611/27/4/045001	journal	March 2011
Solving ill-posed inverse problems using iterative deep neural networks Adler, Jonas; Öktem, Ozan Inverse Problems, Vol. 33, Issue 12 https://doi.org/10.1088/1361-6420/aa9581	journal	November 2017
Learning regularization parameters of inverse problems via deep neural networks Afkham, Babak Maboudi; Chung, Julianne; Chung, Matthias Inverse Problems, Vol. 37, Issue 10 https://doi.org/10.1088/1361-6420/ac245d	journal	September 2021
DRIP: deep regularizers for inverse problems Eliasof, Moshe; Haber, Eldad; Treister, Eran Inverse Problems, Vol. 40, Issue 1 https://doi.org/10.1088/1361-6420/ad0f3b	journal	December 2023
Anomaly Detection with Conditional Variational Autoencoders Pol, Adrian Alan; Berger, Victor; Germain, Cecile 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA) https://doi.org/10.1109/ICMLA.2019.00270	conference	December 2019
Deep Learning Techniques for Inverse Problems in Imaging Ongie, Gregory; Jalal, Ajil; Metzler, Christopher A. IEEE Journal on Selected Areas in Information Theory, Vol. 1, Issue 1 https://doi.org/10.1109/JSAIT.2020.2991563	journal	May 2020
Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods Lucas, Alice; Iliadis, Michael; Molina, Rafael IEEE Signal Processing Magazine, Vol. 35, Issue 1 https://doi.org/10.1109/MSP.2017.2760358	journal	January 2018
Underwater acoustic imaging Sutton, J. L. Proceedings of the IEEE, Vol. 67, Issue 4 https://doi.org/10.1109/PROC.1979.11283	journal	January 1979
InversionNet: An Efficient and Accurate Data-Driven Full Waveform Inversion Wu, Yue; Lin, Youzuo IEEE Transactions on Computational Imaging, Vol. 6 https://doi.org/10.1109/TCI.2019.2956866	journal	January 2020
Coupled Deep Autoencoder for Single Image Super-Resolution Zeng, Kun; Yu, Jun; Wang, Ruxin IEEE Transactions on Cybernetics, Vol. 47, Issue 1 https://doi.org/10.1109/TCYB.2015.2501373	journal	January 2017
Full-Waveform Inversion Using a Learned Regularization Sun, Pengpeng; Yang, Fangshu; Liang, Hongxian IEEE Transactions on Geoscience and Remote Sensing, Vol. 61 https://doi.org/10.1109/TGRS.2023.3322964	journal	January 2023
A Prior Regularized Full Waveform Inversion Using Generative Diffusion Models Wang, Fu; Huang, Xinquan; Alkhalifah, Tariq A. IEEE Transactions on Geoscience and Remote Sensing, Vol. 61 https://doi.org/10.1109/TGRS.2023.3337014	journal	January 2023
Deep Convolutional Neural Network for Inverse Problems in Imaging Jin, Kyong Hwan; McCann, Michael T.; Froustey, Emmanuel IEEE Transactions on Image Processing, Vol. 26, Issue 9 https://doi.org/10.1109/TIP.2017.2713099	journal	September 2017
Decoding by Linear Programming Candes, E. J.; Tao, T. IEEE Transactions on Information Theory, Vol. 51, Issue 12 https://doi.org/10.1109/TIT.2005.858979	journal	December 2005
A Survey on Generative Diffusion Models Cao, Hanqun; Tan, Cheng; Gao, Zhangyang IEEE Transactions on Knowledge and Data Engineering, Vol. 36, Issue 7 https://doi.org/10.1109/TKDE.2024.3361474	journal	July 2024
Deep D-Bar: Real-Time Electrical Impedance Tomography Imaging With Deep Neural Networks Hamilton, Sarah Jane; Hauptmann, A. IEEE Transactions on Medical Imaging, Vol. 37, Issue 10 https://doi.org/10.1109/TMI.2018.2828303	journal	October 2018
MoDL: Model-Based Deep Learning Architecture for Inverse Problems Aggarwal, Hemant K.; Mani, Merry P.; Jacob, Mathews IEEE Transactions on Medical Imaging, Vol. 38, Issue 2 https://doi.org/10.1109/TMI.2018.2865356	journal	February 2019
Model-based geostatistics Diggle, P. J.; Tawn, J. A.; Moyeed, R. A. Journal of the Royal Statistical Society: Series C (Applied Statistics), Vol. 47, Issue 3 https://doi.org/10.1111/1467-9876.00113	journal	March 1998
Inversion of 3-D DC resistivity data using an approximate inverse mapping Li, Yaoguo; Oldenburg, Douglas W. Geophysical Journal International, Vol. 116, Issue 3 https://doi.org/10.1111/j.1365-246X.1994.tb03277.x	journal	March 1994
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Bruckstein, Alfred M.; Donoho, David L.; Elad, Michael SIAM Review, Vol. 51, Issue 1, p. 34-81 https://doi.org/10.1137/060657704	journal	February 2009
Total Variation Regularization Strategies in Full-Waveform Inversion Esser, Ernie; Guasch, Lluis; van Leeuwen, Tristan SIAM Journal on Imaging Sciences, Vol. 11, Issue 1 https://doi.org/10.1137/17M111328X	journal	January 2018
Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Prior González, Mario; Almansa, Andrés; Tan, Pauline SIAM Journal on Imaging Sciences, Vol. 15, Issue 2 https://doi.org/10.1137/21M140225X	journal	June 2022
Robust Modeling with Erratic data Claerbout, Jon F.; Muir, Francis GEOPHYSICS, Vol. 38, Issue 5 https://doi.org/10.1190/1.1440378	journal	October 1973
Seismic waveform inversion in the frequency domain, Part 1: Theory and verification in a physical scale model Pratt, R. Gerhard GEOPHYSICS, Vol. 64, Issue 3 https://doi.org/10.1190/1.1444597	journal	May 1999
Fast full-wavefield seismic inversion using encoded sources Krebs, Jerome R.; Anderson, John E.; Hinkley, David GEOPHYSICS, Vol. 74, Issue 6 https://doi.org/10.1190/1.3230502	journal	November 2009
Projection methods and applications for seismic nonlinear inverse problems with multiple constraints Peters, Bas; Smithyman, Brendan R.; Herrmann, Felix J. GEOPHYSICS, Vol. 84, Issue 2 https://doi.org/10.1190/geo2018-0192.1	journal	March 2019
Reparameterized full-waveform inversion using deep neural networks He, Qinglong; Wang, Yanfei GEOPHYSICS, Vol. 86, Issue 1 https://doi.org/10.1190/geo2019-0382.1	journal	January 2021
Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification Zhu, Weiqiang; Xu, Kailai; Darve, Eric GEOPHYSICS, Vol. 87, Issue 1 https://doi.org/10.1190/geo2020-0933.1	journal	December 2021
WISE: Full-waveform variational inference via subsurface extensions Yin, Ziyi; Orozco, Rafael; Louboutin, Mathias GEOPHYSICS, Vol. 89, Issue 4 https://doi.org/10.1190/geo2023-0744.1	journal	May 2024
A Plug-and-Play Method for Controlled Text Generation Pascual, Damian; Egressy, Beni; Meister, Clara Findings of the Association for Computational Linguistics: EMNLP 2021 https://doi.org/10.18653/v1/2021.findings-emnlp.334	conference	January 2021
An over complete deep learning method for inverse problems Eliasof, Moshe; Haber, Eldad; Treister, Eran Foundations of Data Science, Vol. 7, Issue 1 https://doi.org/10.3934/fods.2024036	journal	January 2025

Similar Records

An autoencoder compression approach for accelerating large-scale inverse problems

Journal Article · 2023 · Inverse Problems · OSTI ID:2578691

Wittmer, Jonathan; Badger, Jacob; Sundar, Hari; +1 more

Deep convolutional autoencoders as generic feature extractors in seismological applications

Journal Article · 2021 · Artificial intelligence in geosciences · OSTI ID:1838599

Kong, Qingkai; Chiang, Andrea; Aguiar, Ana C.; +3 more

Bayesian Neural Network Variational Autoencoder Inverse Mapper (BNN-VAIM) and its application in Compton Form Factors extraction

Journal Article · 2024 · Journal of Instrumentation · OSTI ID:2578843

Hossen, MD; Alghamdi, Tareq; Almaeen, Manal; +1 more

Title: Paired autoencoders for likelihood-free estimation in inverse problems

Citation Formats

References (39)

Similar Records

Related Subjects