Deep learning enhanced super-resolution x-ray fluorescence microscopy by a dual-branch network

Zheng, Xiaoyin; Kankanallu, Varun R.; Lo, Chang-An; Pattammattel, Ajith; Chu, Yong; Chen-Wiegart, Yu-Chen Karen; Huang, Xiaojing

doi:10.1364/optica.503398

Deep learning enhanced super-resolution x-ray fluorescence microscopy by a dual-branch network

Journal Article · Thu Jan 25 00:00:00 EST 2024 · Optica

DOI:https://doi.org/10.1364/optica.503398· OSTI ID:2283313

Zheng, Xiaoyin ^[1]; Kankanallu, Varun R. ^[1]; Lo, Chang-An ^[1]; Pattammattel, Ajith ^[2]; Chu, Yong ^[2]; Chen-Wiegart, Yu-Chen Karen ^[3]; Huang, Xiaojing ^[2]

Stony Brook University, NY (United States)
Brookhaven National Laboratory (BNL), Upton, NY (United States). National Synchrotron Light Source II (NSLS-II)
Stony Brook University, NY (United States); Brookhaven National Laboratory (BNL), Upton, NY (United States). National Synchrotron Light Source II (NSLS-II)

X-ray fluorescence (XRF) microscopy is a powerful technique for quantifying the distribution of elements in complex materials, which makes it a crucial imaging technique across a wide range of disciplines in physical and biological sciences, including chemistry, materials science, microbiology, and geosciences. However, as a scanning microscopy technique, the spatial resolution of XRF imaging is inherently constrained by the x-ray probe profile and scanning step size. Here we propose a dual-branch machine learning (ML) model, which can extract scale-variant features and bypass abundant low-frequency information separately, to enhance the spatial resolution of the XRF images by mitigating the effects of blurring from the probe profile. The model is trained by simulated natural images, and a two-stage training strategy is used to overcome the domain gap between the natural images and experimental data. The tomography reconstruction from enhanced XRF projections shows an improvement in resolution by a scale factor of four and reveals distinct internal features invisible in low-resolution XRF within a battery sample. This study offers a promising approach for obtaining high-resolution XRF imaging from its low-resolution version, paving the way for future investigations in a broader range of disciplines and materials.

View Accepted Manuscript (DOE)

Research Organization:: Brookhaven National Laboratory (BNL), Upton, NY (United States); Stony Brook University, NY (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities (SUF)

Grant/Contract Number:: SC0012673; SC0012704

OSTI ID:: 2283313

Report Number(s):: BNL--225251-2024-JAAM

Journal Information:: Optica, Journal Name: Optica Journal Issue: 2 Vol. 11; ISSN 2334-2536

Publisher:: Optical Society of AmericaCopyright Statement

Country of Publication:: United States

Language:: English

References (32)

Perceptual Losses for Real-Time Style Transfer and Super-Resolution Johnson, Justin; Alahi, Alexandre; Fei-Fei, Li Computer Vision – ECCV 2016 https://doi.org/10.1007/978-3-319-46475-6_43	book	September 2016
Handbook of Practical X-Ray Fluorescence Analysis Beckhoff, Burkhard; Kanngießer, habil. Birgit; Langhoff, Norbert https://doi.org/10.1007/978-3-540-36722-2	book	June 2006
The theoretical derivation of fluorescent X-ray intensities from mixtures Sherman, Jacob Spectrochimica Acta, Vol. 7, p. 283-306 https://doi.org/10.1016/0371-1951(55)80041-0	journal	January 1955
Fourier shell correlation threshold criteria van Heel, Marin; Schatz, Michael Journal of Structural Biology, Vol. 151, Issue 3 https://doi.org/10.1016/j.jsb.2005.05.009	journal	September 2005
Hard X-ray fluorescence tomography—an emerging tool for structural visualization de Jonge, Martin D.; Vogt, Stefan Current Opinion in Structural Biology, Vol. 20, Issue 5 https://doi.org/10.1016/j.sbi.2010.09.002	journal	October 2010
Simultaneous Elucidation of Solid and Solution Manganese Environments via Multiphase Operando Extended X-ray Absorption Fine Structure Spectroscopy in Aqueous Zn/MnO₂ Batteries Wu, Daren; Housel, Lisa M.; King, Steven T. Journal of the American Chemical Society, Vol. 144, Issue 51 https://doi.org/10.1021/jacs.2c09477	journal	December 2022
Reconstructing state mixtures from diffraction measurements Thibault, Pierre; Menzel, Andreas Nature, Vol. 494, Issue 7435 https://doi.org/10.1038/nature11806	journal	February 2013
Reversible aqueous zinc/manganese oxide energy storage from conversion reactions Pan, Huilin; Shao, Yuyan; Yan, Pengfei Nature Energy, Vol. 1, Issue 5 https://doi.org/10.1038/nenergy.2016.39	journal	April 2016
Resolution-enhanced X-ray fluorescence microscopy via deep residual networks Wu, Longlong; Bak, Seongmin; Shin, Youngho npj Computational Materials, Vol. 9, Issue 1 https://doi.org/10.1038/s41524-023-00995-9	journal	March 2023
X-ray ptychographic and fluorescence microscopy of frozen-hydrated cells using continuous scanning Deng, Junjing; Vine, David J.; Chen, Si Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/s41598-017-00569-y	journal	March 2017
High-energy, high-resolution, fly-scan X-ray phase tomography Wang, Hongchang; Atwood, Robert C.; Pankhurst, Matthew James Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-45561-w	journal	June 2019
Fly-scan ptychography Huang, Xiaojing; Lauer, Kenneth; Clark, Jesse N. Scientific Reports, Vol. 5, Issue 1 https://doi.org/10.1038/srep09074	journal	March 2015
Bubble-template-assisted synthesis of hollow fullerene-like MoS ₂ nanocages as a lithium ion battery anode material Zuo, Xiaoxia; Chang, Kun; Zhao, Jing Journal of Materials Chemistry A, Vol. 4, Issue 1 https://doi.org/10.1039/C5TA06869J	journal	January 2016
Quantitative temporally and spatially resolved X-ray fluorescence microprobe characterization of the manganese dissolution-deposition mechanism in aqueous Zn/α-MnO ₂ batteries Wu, Daren; Housel, Lisa M.; Kim, Sung Joo Energy & Environmental Science, Vol. 13, Issue 11 https://doi.org/10.1039/D0EE02168G	journal	January 2020
Elucidating a dissolution–deposition reaction mechanism by multimodal synchrotron X-ray characterization in aqueous Zn/MnO₂ batteries Kankanallu, Varun R.; Zheng, Xiaoyin; Leschev, Denis Energy & Environmental Science, Vol. 16, Issue 6 https://doi.org/10.1039/D2EE03731A	journal	January 2023
An evaluation of high energy bremsstrahlung background in point-projection x-ray radiography experiments Krauland, C. M.; Jarrott, L. C.; Drake, R. P. Review of Scientific Instruments, Vol. 83, Issue 10 https://doi.org/10.1063/1.4738649	journal	October 2012
Achieving high spatial resolution in a large field-of-view using lensless x-ray imaging Jiang, Yi; Deng, Junjing; Yao, Yudong Applied Physics Letters, Vol. 119, Issue 12 https://doi.org/10.1063/5.0067197	journal	September 2021
Multimodal hard x-ray imaging with resolution approaching 10 nm for studies in material science Yan, Hanfei; Bouet, Nathalie; Zhou, Juan Nano Futures, Vol. 2, Issue 1 https://doi.org/10.1088/2399-1984/aab25d	journal	March 2018
TomoPy: a framework for the analysis of synchrotron tomographic data Gürsoy, Dogˇa; De Carlo, Francesco; Xiao, Xianghui Journal of Synchrotron Radiation, Vol. 21, Issue 5 https://doi.org/10.1107/S1600577514013939	journal	August 2014
Design and performance of an X-ray scanning microscope at the Hard X-ray Nanoprobe beamline of NSLS-II Nazaretski, E.; Yan, H.; Lauer, K. Journal of Synchrotron Radiation, Vol. 24, Issue 6 https://doi.org/10.1107/S1600577517011183	journal	October 2017
Convolutional Sparse Coding for Image Super-Resolution Gu, Shuhang; Zuo, Wangmeng; Xie, Qi 2015 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2015.212	conference	December 2015
MemNet: A Persistent Memory Network for Image Restoration Tai, Ying; Yang, Jian; Liu, Xiaoming 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.486	conference	October 2017
Guided Attention Inference Network Li, Kunpeng; Wu, Ziyan; Peng, Kuan-Chuan IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, Issue 12 https://doi.org/10.1109/TPAMI.2019.2921543	journal	December 2020
Superresolution imaging via ptychography Maiden, Andrew M.; Humphry, Martin J.; Zhang, Fucai Journal of the Optical Society of America A, Vol. 28, Issue 4 https://doi.org/10.1364/JOSAA.28.000604	journal	January 2011
Simultaneous X-ray fluorescence and ptychographic microscopy of Cyclotella meneghiniana Vine, D. J.; Pelliccia, D.; Holzner, C. Optics Express, Vol. 20, Issue 16 https://doi.org/10.1364/OE.20.018287	journal	January 2012
Adam: A Method for Stochastic Optimization Kingma, Diederik P.; Ba, Jimmy arXiv https://doi.org/10.48550/arXiv.1412.6980	preprint	January 2014
Image Super-Resolution Using Deep Convolutional Networks Dong, Chao; Loy, Chen Change; He, Kaiming arXiv https://doi.org/10.48550/arXiv.1501.00092	preprint	January 2015
Loss Functions for Neural Networks for Image Processing Zhao, Hang; Gallo, Orazio; Frosio, Iuri arXiv https://doi.org/10.48550/arXiv.1511.08861	preprint	January 2015
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network Shi, Wenzhe; Caballero, Jose; Huszár, Ferenc arXiv https://doi.org/10.48550/arXiv.1609.05158	preprint	January 2016
Enhanced Deep Residual Networks for Single Image Super-Resolution Lim, Bee; Son, Sanghyun; Kim, Heewon arXiv https://doi.org/10.48550/arXiv.1707.02921	preprint	January 2017
Deep Learning using Rectified Linear Units (ReLU) Agarap, Abien Fred arXiv https://doi.org/10.48550/arXiv.1803.08375	preprint	January 2018
Multi-scale deep neural networks for real image super-resolution Gao, Shangqi; Zhuang, Xiahai arXiv https://doi.org/10.48550/arXiv.1904.10698	preprint	January 2019