A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

Chavez, Tanny; Roberts, Eric J.; Zwart, Petrus H.; Hexemer, Alexander

doi:10.1107/S1600576722007105

Title: A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

Journal Article · Wed Sep 28 00:00:00 EDT 2022 · Journal of Applied Crystallography (Online)

DOI:https://doi.org/10.1107/S1600576722007105· OSTI ID:1890010

Chavez, Tanny; Roberts, Eric J.;

;

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

View Journal Article

Cite

Export

Save

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities Division; National Institutes of Health (NIH)

Grant/Contract Number:: AC02-05CH11231; 5R21GM129649-02

OSTI ID:: 1890010

Alternate ID(s):: OSTI ID: 1902686

Journal Information:: Journal of Applied Crystallography (Online), Journal Name: Journal of Applied Crystallography (Online) Vol. 55 Journal Issue: 5; ISSN 1600-5767

Publisher:: International Union of Crystallography (IUCr)Copyright Statement

Country of Publication:: Denmark

Language:: English

References (48)

Rank-One Matrix Approximation With ℓ_p-Norm for Image Inpainting Li, Xiao Peng; Liu, Qi; So, Hing Cheung IEEE Signal Processing Letters, Vol. 27 https://doi.org/10.1109/LSP.2020.2988596	journal	January 2020
Inpainting of Binary Images Using the Cahn–Hilliard Equation Bertozzi, Andrea L.; Esedoglu, Selim; Gillette, Alan IEEE Transactions on Image Processing, Vol. 16, Issue 1 https://doi.org/10.1109/TIP.2006.887728	journal	January 2007
Single-Image Fence Removal Using Deep Convolutional Neural Network Matsui, Takuro; Ikehara, Masaaki IEEE Access, Vol. 8 https://doi.org/10.1109/ACCESS.2019.2960087	journal	January 2020
Imaging the orientation of myelin sheaths in a non-stained histology slide of human brain Schulz, Georg; Deyhle, Hans; Bikis, Christos Precision Nanomedicine, Vol. 3, Issue 3 https://doi.org/10.33218/001c.17211	journal	September 2020
Image Inpainting: A Review Elharrouss, Omar; Almaadeed, Noor; Al-Maadeed, Somaya Neural Processing Letters, Vol. 51, Issue 2 https://doi.org/10.1007/s11063-019-10163-0	journal	December 2019
The perceptron: A probabilistic model for information storage and organization in the brain. Rosenblatt, F. Psychological Review, Vol. 65, Issue 6 https://doi.org/10.1037/h0042519	journal	January 1958
Nanomethods: Scanning X-ray scattering: Evaluating the nanostructure of human tissues Müller, Bert; Deyhle, Hans; Bradley, David A. European Journal of Nanomedicine, Vol. 3, Issue 1 https://doi.org/10.1515/EJNM.2010.3.1.30	journal	January 2010
On Surface Completion and Image Inpainting by Biharmonic Functions: Numerical Aspects Damelin, S. B.; Hoang, N. S. International Journal of Mathematics and Mathematical Sciences, Vol. 2018 https://doi.org/10.1155/2018/3950312	journal	January 2018
Nanostructure-specific X-ray tomography reveals myelin levels, integrity and axon orientations in mouse and human nervous tissue Georgiadis, Marios; Schroeter, Aileen; Gao, Zirui Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-22719-7	journal	May 2021
A SAXS/WAXS/GISAXS Beamline with Multilayer Monochromator Hexemer, Alexander; Bras, Wim; Glossinger, James Journal of Physics: Conference Series, Vol. 247 https://doi.org/10.1088/1742-6596/247/1/012007	journal	October 2010
UNet++: A Nested U-Net Architecture for Medical Image Segmentation Zhou, Zongwei; Rahman Siddiquee, Md Mahfuzur; Tajbakhsh, Nima Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support https://doi.org/10.1007/978-3-030-00889-5_1	book	January 2018
A comprehensive review of past and present image inpainting methods Jam, Jireh; Kendrick, Connah; Walker, Kevin Computer Vision and Image Understanding, Vol. 203 https://doi.org/10.1016/j.cviu.2020.103147	journal	February 2021
Membrane interactions of non-membrane targeting antibiotics: The case of aminoglycosides, macrolides, and fluoroquinolones Khondker, Adree; Bider, Renée-Claude; Passos-Gastaldo, Isabella Biochimica et Biophysica Acta (BBA) - Biomembranes, Vol. 1863, Issue 1 https://doi.org/10.1016/j.bbamem.2020.183448	journal	January 2021
Robust depth map inpainting using superpixels and non-local Gauss–Markov random field prior Satapathy, Sukla; Sahay, Rajiv Ranjan Signal Processing: Image Communication, Vol. 98 https://doi.org/10.1016/j.image.2021.116378	journal	October 2021
Noise reduction and mask removal neural network for X-ray single-particle imaging Bellisario, Alfredo; Maia, Filipe R. N. C.; Ekeberg, Tomas Journal of Applied Crystallography, Vol. 55, Issue 1 https://doi.org/10.1107/S1600576721012371	journal	February 2022
The relationship between ultra-small-angle X-ray scattering and viscosity measurements of casein micelles in skim milk concentrates Christiansen, Morten V.; Smith, Gregory N.; Brok, Erik S. Food Research International, Vol. 147 https://doi.org/10.1016/j.foodres.2021.110451	journal	September 2021
Convolutional neural networks for grazing incidence x-ray scattering patterns: thin film structure identification Liu, Shuai; Melton, Charles N.; Venkatakrishnan, Singanallur MRS Communications, Vol. 9, Issue 02 https://doi.org/10.1557/mrc.2019.26	journal	March 2019
Complex biomembrane mimetics on the sub-nanometer scale Heberle, Frederick A.; Pabst, Georg Biophysical Reviews, Vol. 9, Issue 4 https://doi.org/10.1007/s12551-017-0275-5	journal	July 2017
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation Çiçek, Özgün; Abdulkadir, Ahmed; Lienkamp, Soeren S. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2016 https://doi.org/10.1007/978-3-319-46723-8_49	conference	January 2016
Protein X-ray Crystallography and Drug Discovery Maveyraud, Laurent; Mourey, Lionel Molecules, Vol. 25, Issue 5 https://doi.org/10.3390/molecules25051030	journal	February 2020
Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs Xu, Qi; Zhang, Ming; Gu, Zonghua Neurocomputing, Vol. 328 https://doi.org/10.1016/j.neucom.2018.03.080	journal	February 2019
The Sub-Nanometer Scale as a New Focus in Nanoscience Ni, Bing; Shi, Yuang; Wang, Xun Advanced Materials, Vol. 30, Issue 43 https://doi.org/10.1002/adma.201802031	journal	July 2018
Research on image Inpainting algorithm of improved GAN based on two-discriminations networks Chen, Yuantao; Zhang, Haopeng; Liu, Linwu Applied Intelligence, Vol. 51, Issue 6 https://doi.org/10.1007/s10489-020-01971-2	journal	November 2020
ipyChord: a package for evaluating small-angle X-ray scattering data of fiber symmetry Li, Xuke Journal of Applied Crystallography, Vol. 54, Issue 2 https://doi.org/10.1107/S1600576721001369	journal	March 2021
Skimmed milk structural dynamics during high hydrostatic pressure processing from in situ SAXS Yang, Shuailing; Tyler, Arwen I. I.; Ahrné, Lilia Food Research International, Vol. 147 https://doi.org/10.1016/j.foodres.2021.110527	journal	September 2021
The fast azimuthal integration Python library: pyFAI Ashiotis, Giannis; Deschildre, Aurore; Nawaz, Zubair Journal of Applied Crystallography, Vol. 48, Issue 2 https://doi.org/10.1107/S1600576715004306	journal	March 2015
Data Management at the Advanced Photon Source Schwarz, Nicholas; Veseli, Siniša; Jarosz, Dariusz Synchrotron Radiation News, Vol. 32, Issue 3 https://doi.org/10.1080/08940886.2019.1608120	journal	May 2019
ResNet Autoencoders for Unsupervised Feature Learning From High-Dimensional Data: Deep Models Resistant to Performance Degradation Wickramasinghe, Chathurika S.; Marino, Daniel L.; Manic, Milos IEEE Access, Vol. 9 https://doi.org/10.1109/ACCESS.2021.3064819	journal	January 2021
A mixed-scale dense convolutional neural network for image analysis Pelt, Daniël M.; Sethian, James A. Proceedings of the National Academy of Sciences, Vol. 115, Issue 2 https://doi.org/10.1073/pnas.1715832114	journal	December 2017
Loss Functions for Image Restoration With Neural Networks Zhao, Hang; Gallo, Orazio; Frosio, Iuri IEEE Transactions on Computational Imaging, Vol. 3, Issue 1 https://doi.org/10.1109/TCI.2016.2644865	journal	March 2017
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation Zhou, Zongwei; Siddiquee, Md Mahfuzur Rahman; Tajbakhsh, Nima IEEE Transactions on Medical Imaging, Vol. 39, Issue 6 https://doi.org/10.1109/TMI.2019.2959609	journal	June 2020
Polyoxometalate Clusters: Sub-nanometer Building Blocks for Construction of Advanced Materials Liu, Qingda; Wang, Xun Matter, Vol. 2, Issue 4 https://doi.org/10.1016/j.matt.2020.01.020	journal	April 2020
Applications of synchrotron powder X-ray diffractometry in drug substance and drug product characterization Munjal, Bhushan; Suryanarayanan, Raj TrAC Trends in Analytical Chemistry, Vol. 136 https://doi.org/10.1016/j.trac.2021.116181	journal	March 2021
An unsupervised learning method with convolutional auto-encoder for vessel trajectory similarity computation Liang, Maohan; Liu, Ryan Wen; Li, Shichen Ocean Engineering, Vol. 225 https://doi.org/10.1016/j.oceaneng.2021.108803	journal	April 2021
Feasibility of imaging amyloid in the brain using small-angle x-ray scattering Choi, Mina; Dahal, Eshan; Badano, Aldo Biomedical Physics & Engineering Express, Vol. 7, Issue 1 https://doi.org/10.1088/2057-1976/ab501c	journal	November 2020
Machine learning on neutron and x-ray scattering and spectroscopies Chen, Zhantao; Andrejevic, Nina; Drucker, Nathan C. Chemical Physics Reviews, Vol. 2, Issue 3 https://doi.org/10.1063/5.0049111	journal	September 2021
A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images Cui, Yuxin; Zhang, Guiying; Liu, Zhonghao Medical & Biological Engineering & Computing, Vol. 57, Issue 9 https://doi.org/10.1007/s11517-019-02008-8	journal	July 2019
Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition Fukushima, Kunihiko; Miyake, Sei Competition and Cooperation in Neural Nets https://doi.org/10.1007/978-3-642-46466-9_18	book	January 1982
Improving Tomographic Reconstruction from Limited Data Using Mixed-Scale Dense Convolutional Neural Networks Pelt, Daniël; Batenburg, Kees; Sethian, James Journal of Imaging, Vol. 4, Issue 11 https://doi.org/10.3390/jimaging4110128	journal	October 2018
Multiscale methods in drug design bridge chemical and biological complexity in the search for cures Amaro, Rommie E.; Mulholland, Adrian J. Nature Reviews Chemistry, Vol. 2, Issue 4 https://doi.org/10.1038/s41570-018-0148	journal	April 2018
Gaussian processes for autonomous data acquisition at large-scale synchrotron and neutron facilities Noack, Marcus M.; Zwart, Petrus H.; Ushizima, Daniela M. Nature Reviews Physics, Vol. 3, Issue 10 https://doi.org/10.1038/s42254-021-00345-y	journal	July 2021
Amyloid structure exhibits polymorphism on multiple length scales in human brain tissue Liu, Jiliang; Costantino, Isabel; Venugopalan, Nagarajan Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep33079	journal	September 2016
Ab initio structure determination from experimental fluctuation X-ray scattering data Pande, Kanupriya; Donatelli, Jeffrey J.; Malmerberg, Erik Proceedings of the National Academy of Sciences, Vol. 115, Issue 46 https://doi.org/10.1073/pnas.1812064115	journal	October 2018
Gradient-based learning applied to document recognition Lecun, Y.; Bottou, L.; Bengio, Y. Proceedings of the IEEE, Vol. 86, Issue 11 https://doi.org/10.1109/5.726791	journal	January 1998
Learning on the Edge: Investigating Boundary Filters in CNNs Innamorati, Carlo; Ritschel, Tobias; Weyrich, Tim International Journal of Computer Vision, Vol. 128, Issue 4 https://doi.org/10.1007/s11263-019-01223-y	journal	October 2019
Introducing Hann windows for reducing edge-effects in patch-based image segmentation Pielawski, Nicolas; Wählby, Carolina PLOS ONE, Vol. 15, Issue 3 https://doi.org/10.1371/journal.pone.0229839	journal	March 2020
Healing X-ray scattering images Liu, Jiliang; Lhermitte, Julien; Tian, Ye IUCrJ, Vol. 4, Issue 4 https://doi.org/10.1107/S2052252517006212	journal	May 2017
Scale-space approximated convolutional neural networks for retinal vessel segmentation Noh, Kyoung Jin; Park, Sang Jun; Lee, Soochahn Computer Methods and Programs in Biomedicine, Vol. 178 https://doi.org/10.1016/j.cmpb.2019.06.030	journal	September 2019

Similar Records

DLSIA: Deep Learning for Scientific Image Analysis

Journal Article · Thu Mar 21 00:00:00 EDT 2024 · Journal of Applied Crystallography (Online) · OSTI ID:1890010

Roberts, Eric J.; Chavez, Tanny; Hexemer, Alexander; +1 more

Imaging through glass diffusers using densely connected convolutional networks

Journal Article · Fri Jul 06 00:00:00 EDT 2018 · Optica · OSTI ID:1890010

Li, Shuai; Deng, Mo; Lee, Justin; +2 more

Integrate Light-Weight Deep Learning Tools with Internet of Things

Technical Report · Mon Sep 30 00:00:00 EDT 2019 · OSTI ID:1890010

Sun, Yu; Yu, Dantong

Related Subjects

97 MATHEMATICS AND COMPUTING
X-ray scattering
image inpainting
deep learning
mixed-scale dense networks
tunable U-Nets

Title: A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

Citation Formats

References (48)

Similar Records

Related Subjects