Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks
Abstract
Abstract As a critical component of coherent X-ray diffraction imaging (CDI), phase retrieval has been extensively applied in X-ray structural science to recover the 3D morphological information inside measured particles. Despite meeting all the oversampling requirements of Sayre and Shannon, current phase retrieval approaches still have trouble achieving a unique inversion of experimental data in the presence of noise. Here, we propose to overcome this limitation by incorporating a 3D Machine Learning (ML) model combining (optional) supervised learning with transfer learning. The trained ML model can rapidly provide an immediate result with high accuracy which could benefit real-time experiments, and the predicted result can be further refined with transfer learning. More significantly, the proposed ML model can be used without any prior training to learn the missing phases of an image based on minimization of an appropriate ‘loss function’ alone. We demonstrate significantly improved performance with experimental Bragg CDI data over traditional iterative phase retrieval algorithms.
- Publication Date:
- Research Org.:
- Brookhaven National Lab. (BNL), Upton, NY (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES); National Science Foundation (NSF)
- OSTI Identifier:
- 1828093
- Alternate Identifier(s):
- OSTI ID: 1823628
- Report Number(s):
- BNL-222182-2021-JAAM
Journal ID: ISSN 2057-3960; 175; PII: 644
- Grant/Contract Number:
- SC0012704; No. DE-SC0012704; AC02-06CH11357; DMR-9724294
- Resource Type:
- Published Article
- Journal Name:
- npj Computational Materials
- Additional Journal Information:
- Journal Name: npj Computational Materials Journal Volume: 7 Journal Issue: 1; Journal ID: ISSN 2057-3960
- Publisher:
- Nature Publishing Group
- Country of Publication:
- United Kingdom
- Language:
- English
- Subject:
- 75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND SUPERFLUIDITY; Applied physics; Characterization and analytical techniques; Computational methods; Imaging techniques
Citation Formats
Wu, Longlong, Yoo, Shinjae, Suzana, Ana F., Assefa, Tadesse A., Diao, Jiecheng, Harder, Ross J., Cha, Wonsuk, and Robinson, Ian K. Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks. United Kingdom: N. p., 2021.
Web. doi:10.1038/s41524-021-00644-z.
Wu, Longlong, Yoo, Shinjae, Suzana, Ana F., Assefa, Tadesse A., Diao, Jiecheng, Harder, Ross J., Cha, Wonsuk, & Robinson, Ian K. Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks. United Kingdom. https://doi.org/10.1038/s41524-021-00644-z
Wu, Longlong, Yoo, Shinjae, Suzana, Ana F., Assefa, Tadesse A., Diao, Jiecheng, Harder, Ross J., Cha, Wonsuk, and Robinson, Ian K. Thu .
"Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks". United Kingdom. https://doi.org/10.1038/s41524-021-00644-z.
@article{osti_1828093,
title = {Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks},
author = {Wu, Longlong and Yoo, Shinjae and Suzana, Ana F. and Assefa, Tadesse A. and Diao, Jiecheng and Harder, Ross J. and Cha, Wonsuk and Robinson, Ian K.},
abstractNote = {Abstract As a critical component of coherent X-ray diffraction imaging (CDI), phase retrieval has been extensively applied in X-ray structural science to recover the 3D morphological information inside measured particles. Despite meeting all the oversampling requirements of Sayre and Shannon, current phase retrieval approaches still have trouble achieving a unique inversion of experimental data in the presence of noise. Here, we propose to overcome this limitation by incorporating a 3D Machine Learning (ML) model combining (optional) supervised learning with transfer learning. The trained ML model can rapidly provide an immediate result with high accuracy which could benefit real-time experiments, and the predicted result can be further refined with transfer learning. More significantly, the proposed ML model can be used without any prior training to learn the missing phases of an image based on minimization of an appropriate ‘loss function’ alone. We demonstrate significantly improved performance with experimental Bragg CDI data over traditional iterative phase retrieval algorithms.},
doi = {10.1038/s41524-021-00644-z},
journal = {npj Computational Materials},
number = 1,
volume = 7,
place = {United Kingdom},
year = {2021},
month = {10}
}
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1038/s41524-021-00644-z
https://doi.org/10.1038/s41524-021-00644-z
Other availability
Search WorldCat to find libraries that may hold this journal
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Deep learning for tomographic image reconstruction
journal, December 2020
- Wang, Ge; Ye, Jong Chul; De Man, Bruno
- Nature Machine Intelligence, Vol. 2, Issue 12
Three-dimensional imaging of dislocation dynamics during the hydriding phase transformation
journal, January 2017
- Ulvestad, A.; Welland, M. J.; Cha, W.
- Nature Materials, Vol. 16, Issue 5
Maximum-likelihood refinement for coherent diffractive imaging
journal, June 2012
- Thibault, P.; Guizar-Sicairos, M.
- New Journal of Physics, Vol. 14, Issue 6
Coherent X-ray diffraction imaging of strain at the nanoscale
journal, April 2009
- Robinson, Ian; Harder, Ross
- Nature Materials, Vol. 8, Issue 4
Quantum optics: Science and technology in a new light
journal, April 2015
- Walmsley, I. A.
- Science, Vol. 348, Issue 6234
Three-dimensional imaging of strain in a single ZnO nanorod
journal, December 2009
- Newton, Marcus C.; Leake, Steven J.; Harder, Ross
- Nature Materials, Vol. 9, Issue 2
General approaches for shear-correcting coordinate transformations in Bragg coherent diffraction imaging. Part II
journal, March 2020
- Li, P.; Maddali, S.; Pateras, A.
- Journal of Applied Crystallography, Vol. 53, Issue 2
Three-Dimensional Imaging of Microstructure in Au Nanocrystals
journal, April 2003
- Williams, G. J.; Pfeifer, M. A.; Vartanyants, I. A.
- Physical Review Letters, Vol. 90, Issue 17
Bragg coherent diffractive imaging of single-grain defect dynamics in polycrystalline films
journal, May 2017
- Yau, Allison; Cha, Wonsuk; Kanan, Matthew W.
- Science, Vol. 356, Issue 6339
Three-dimensional strain dynamics govern the hysteresis in heterogeneous catalysis
journal, September 2020
- Passos, Aline R.; Rochet, Amélie; Manente, Luiza M.
- Nature Communications, Vol. 11, Issue 1
Perspectives on single particle imaging with x rays at the advent of high repetition rate x-ray free electron laser sources
journal, July 2020
- Bielecki, Johan; Maia, Filipe R. N. C.; Mancuso, Adrian P.
- Structural Dynamics, Vol. 7, Issue 4
Phase retrieval algorithms: a comparison
journal, January 1982
- Fienup, J. R.
- Applied Optics, Vol. 21, Issue 15
Coherent X-Ray Diffraction Imaging and Characterization of Strain in Silicon-on-Insulator Nanostructures
journal, June 2014
- Xiong, Gang; Moutanabbir, Oussama; Reiche, Manfred
- Advanced Materials, Vol. 26, Issue 46
Bragg coherent diffraction imaging by simultaneous reconstruction of multiple diffraction peaks
journal, January 2021
- Gao, Yuan; Huang, Xiaojing; Yan, Hanfei
- Physical Review B, Vol. 103, Issue 1
A MHz-repetition-rate hard X-ray free-electron laser driven by a superconducting linear accelerator
journal, May 2020
- Decking, W.; Abeghyan, S.; Abramian, P.
- Nature Photonics, Vol. 14, Issue 6
Complex imaging of phase domains by deep neural networks
journal, January 2021
- Wu, Longlong; Juhas, Pavol; Yoo, Shinjae
- IUCrJ, Vol. 8, Issue 1
Three-dimensional mapping of a deformation field inside a nanocrystal
journal, July 2006
- Pfeifer, Mark A.; Williams, Garth J.; Vartanyants, Ivan A.
- Nature, Vol. 442, Issue 7098
Single-particle imaging without symmetry constraints at an X-ray free-electron laser
journal, September 2018
- Rose, Max; Bobkov, Sergey; Ayyer, Kartik
- IUCrJ, Vol. 5, Issue 6, p. 727-736
Relaxed averaged alternating reflections for diffraction imaging
journal, November 2004
- Luke, D. Russell
- Inverse Problems, Vol. 21, Issue 1
Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization
journal, January 2002
- Bauschke, Heinz H.; Combettes, Patrick L.; Luke, D. Russell
- Journal of the Optical Society of America A, Vol. 19, Issue 7
Monte Carlo Calculation for Electromagnetic-Wave Scattering from Random Rough Surfaces
journal, May 1984
- Garcia, N.; Stoll, E.
- Physical Review Letters, Vol. 52, Issue 20
Deep neural networks in real-time coherent diffraction imaging
journal, January 2021
- Harder, Ross
- IUCrJ, Vol. 8, Issue 1
Ultrafast Three-Dimensional Imaging of Lattice Dynamics in Individual Gold Nanocrystals
journal, May 2013
- Clark, J. N.; Beitra, L.; Xiong, G.
- Science, Vol. 341, Issue 6141
Phase retrieval by iterated projections
journal, January 2003
- Elser, Veit
- Journal of the Optical Society of America A, Vol. 20, Issue 1
Real-time coherent diffraction inversion using deep generative networks
journal, November 2018
- Cherukara, Mathew J.; Nashed, Youssef S. G.; Harder, Ross J.
- Scientific Reports, Vol. 8, Issue 1
General approaches for shear-correcting coordinate transformations in Bragg coherent diffraction imaging. Part I
journal, March 2020
- Maddali, S.; Li, P.; Pateras, A.
- Journal of Applied Crystallography, Vol. 53, Issue 2
Coherent lensless X-ray imaging
journal, November 2010
- Chapman, Henry N.; Nugent, Keith A.
- Nature Photonics, Vol. 4, Issue 12
Invited Article: A unified evaluation of iterative projection algorithms for phase retrieval
journal, January 2007
- Marchesini, S.
- Review of Scientific Instruments, Vol. 78, Issue 1
Phase Retrieval with Application to Optical Imaging: A contemporary overview
journal, May 2015
- Shechtman, Yoav; Eldar, Yonina C.; Cohen, Oren
- IEEE Signal Processing Magazine, Vol. 32, Issue 3
Time-Resolved X-Ray Microscopy for Materials Science
journal, July 2019
- Wen, Haidan; Cherukara, Mathew J.; Holt, Martin V.
- Annual Review of Materials Research, Vol. 49, Issue 1
Reconstruction of the Shapes of Gold Nanocrystals Using Coherent X-Ray Diffraction
journal, October 2001
- Robinson, I. K.; Vartanyants, I. A.; Williams, G. J.
- Physical Review Letters, Vol. 87, Issue 19
Orientation variation of surface strain
journal, September 2007
- Harder, R.; Pfeifer, M. A.; Williams, G. J.
- Physical Review B, Vol. 76, Issue 11
Femtosecond diffractive imaging with a soft-X-ray free-electron laser
journal, November 2006
- Chapman, Henry N.; Barty, Anton; Bogan, Michael J.
- Nature Physics, Vol. 2, Issue 12
Broadband coherent diffractive imaging
journal, July 2020
- Huijts, Julius; Fernandez, Sara; Gauthier, David
- Nature Photonics, Vol. 14, Issue 10
Uniqueness of solutions to two-dimensional fourier phase problems for localized and positive images
journal, February 1984
- Bates, R. H. T.
- Computer Vision, Graphics, and Image Processing, Vol. 25, Issue 2
Ultrafast single-shot diffraction imaging of nanoscale dynamics
journal, June 2008
- Barty, Anton; Boutet, Sébastien; Bogan, Michael J.
- Nature Photonics, Vol. 2, Issue 7
Adaptive 3D convolutional neural network-based reconstruction method for 3D coherent diffraction imaging
journal, November 2020
- Scheinker, Alexander; Pokharel, Reeju
- Journal of Applied Physics, Vol. 128, Issue 18
X-ray image reconstruction from a diffraction pattern alone
journal, October 2003
- Marchesini, S.; He, H.; Chapman, H. N.
- Physical Review B, Vol. 68, Issue 14