Deep-Learning Electron Diffractive Imaging

Chang, Dillan J.; O’Leary, Colum M.; Su, Cong; Jacobs, Daniel A.; Kahn, Salman; Zettl, Alex; Ciston, Jim; Ercius, Peter; Miao, Jianwei

doi:10.1103/PhysRevLett.130.016101

Title: Deep-Learning Electron Diffractive Imaging

Journal Article · Fri Jan 06 00:00:00 UTC 2023 · Physical Review Letters

DOI: https://doi.org/10.1103/PhysRevLett.130.016101 · OSTI ID:2568362

Chang, Dillan J. ^[1]; O’Leary, Colum M. ^[1]; Su, Cong ^[2]; Jacobs, Daniel A. ^[1]; Kahn, Salman ^[2]; Zettl, Alex ^[2]; Ciston, Jim ^[3]; Ercius, Peter ^[3];

^[1]

Univ. of California, Los Angeles, CA (United States)
University of California, Berkeley, CA (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); Kavli Energy NanoScience Institute, Berkeley, CA (United States)
Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). Molecular Foundry

Here, we report the development of deep-learning coherent electron diffractive imaging at subangstrom resolution using convolutional neural networks (CNNs) trained with only simulated data. We experimentally demonstrate this method by applying the trained CNNs to recover the phase images from electron diffraction patterns of twisted hexagonal boron nitride, monolayer graphene, and a gold nanoparticle with comparable quality to those reconstructed by a conventional ptychographic algorithm. Fourier ring correlation between the CNN and ptychographic images indicates the achievement of a resolution in the range of 0.70 and 0.55 Å. We further develop CNNs to recover the probe function from the experimental data. The ability to replace iterative algorithms with CNNs and perform real-time atomic imaging from coherent diffraction patterns is expected to find applications in the physical and biological sciences.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Univ. of California, Los Angeles, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division (MSE)

Grant/Contract Number:: AC02-05CH11231; SC0010378

OSTI ID:: 2568362

Journal Information:: Physical Review Letters, Journal Name: Physical Review Letters Journal Issue: 1 Vol. 130; ISSN 1079-7114; ISSN 0031-9007

Publisher:: American Physical Society (APS)Copyright Statement

Country of Publication:: United States

Language:: English

References (40)

Advanced Computing in Electron Microscopy Kirkland, Earl J. https://doi.org/10.1007/978-1-4419-6533-2	book	January 2010
Optimal Determination of Particle Orientation, Absolute Hand, and Contrast Loss in Single-particle Electron Cryomicroscopy Rosenthal, Peter B.; Henderson, Richard Journal of Molecular Biology, Vol. 333, Issue 4 https://doi.org/10.1016/j.jmb.2003.07.013	journal	October 2003
Deep learning in neural networks: An overview Schmidhuber, Jürgen Neural Networks, Vol. 61 https://doi.org/10.1016/j.neunet.2014.09.003	journal	January 2015
An improved ptychographical phase retrieval algorithm for diffractive imaging Maiden, Andrew M.; Rodenburg, John M. Ultramicroscopy, Vol. 109, Issue 10 https://doi.org/10.1016/j.ultramic.2009.05.012	journal	September 2009
Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution Pennycook, Timothy J.; Lupini, Andrew R.; Yang, Hao Ultramicroscopy, Vol. 151, p. 160-167 https://doi.org/10.1016/j.ultramic.2014.09.013	journal	April 2015
Contrast transfer and noise considerations in focused-probe electron ptychography O’Leary, Colum M.; Martinez, Gerardo T.; Liberti, Emanuela Ultramicroscopy, Vol. 221 https://doi.org/10.1016/j.ultramic.2020.113189	journal	February 2021
Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens Miao, Jianwei; Charalambous, Pambos; Kirz, Janos Nature, Vol. 400, Issue 6742 https://doi.org/10.1038/22498	journal	July 1999
Ptychographic X-ray computed tomography at the nanoscale Dierolf, Martin; Menzel, Andreas; Thibault, Pierre Nature, Vol. 467, Issue 7314 https://doi.org/10.1038/nature09419	journal	September 2010
Electron tomography at 2.4-ångström resolution Scott, M. C.; Chen, Chien-Chun; Mecklenburg, Matthew Nature, Vol. 483, Issue 7390 https://doi.org/10.1038/nature10934	journal	March 2012
High-resolution non-destructive three-dimensional imaging of integrated circuits Holler, Mirko; Guizar-Sicairos, Manuel; Tsai, Esther H. R. Nature, Vol. 543, Issue 7645 https://doi.org/10.1038/nature21698	journal	March 2017
Simultaneous atomic-resolution electron ptychography and Z-contrast imaging of light and heavy elements in complex nanostructures Yang, H.; Rutte, R. N.; Jones, L. Nature Communications, Vol. 7, Issue 1 https://doi.org/10.1038/ncomms12532	journal	August 2016
Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging Humphry, M. J.; Kraus, B.; Hurst, A. C. Nature Communications, Vol. 3, Issue 1 https://doi.org/10.1038/ncomms1733	journal	January 2012
Chemical composition mapping with nanometre resolution by soft X-ray microscopy Shapiro, David A.; Yu, Young-Sang; Tyliszczak, Tolek Nature Photonics, Vol. 8, Issue 10 https://doi.org/10.1038/nphoton.2014.207	journal	September 2014
Subwavelength coherent imaging of periodic samples using a 13.5 nm tabletop high-harmonic light source Gardner, Dennis F.; Tanksalvala, Michael; Shanblatt, Elisabeth R. Nature Photonics, Vol. 11, Issue 4 https://doi.org/10.1038/nphoton.2017.33	journal	March 2017
Electron ptychographic microscopy for three-dimensional imaging Gao, Si; Wang, Peng; Zhang, Fucai Nature Communications, Vol. 8, Issue 1 https://doi.org/10.1038/s41467-017-00150-1	journal	July 2017
Low-dose phase retrieval of biological specimens using cryo-electron ptychography Zhou, Liqi; Song, Jingdong; Kim, Judy S. Nature Communications, Vol. 11, Issue 1 https://doi.org/10.1038/s41467-020-16391-6	journal	June 2020
Three-dimensional atomic packing in amorphous solids with liquid-like structure Yuan, Yakun; Kim, Dennis S.; Zhou, Jihan Nature Materials, Vol. 21, Issue 1 https://doi.org/10.1038/s41563-021-01114-z	journal	October 2021
Tuning colour centres at a twisted hexagonal boron nitride interface Su, Cong; Zhang, Fang; Kahn, Salman Nature Materials, Vol. 21, Issue 8 https://doi.org/10.1038/s41563-022-01303-4	journal	July 2022
Electron ptychography of 2D materials to deep sub-ångström resolution Jiang, Yi; Chen, Zhen; Han, Yimo Nature, Vol. 559, Issue 7714 https://doi.org/10.1038/s41586-018-0298-5	journal	July 2018
Determining the three-dimensional atomic structure of an amorphous solid Yang, Yao; Zhou, Jihan; Zhu, Fan Nature, Vol. 592, Issue 7852 https://doi.org/10.1038/s41586-021-03354-0	journal	March 2021
Electron Ptychographic Diffractive Imaging of Boron Atoms in LaB6 Crystals Wang, Peng; Zhang, Fucai; Gao, Si Scientific Reports, Vol. 7, Issue 1 https://doi.org/10.1038/s41598-017-02778-x	journal	June 2017
Multi-slice ptychographic tomography Li, Peng; Maiden, Andrew Scientific Reports, Vol. 8, Issue 1 https://doi.org/10.1038/s41598-018-20530-x	journal	February 2018
AI-enabled high-resolution scanning coherent diffraction imaging Cherukara, Mathew J.; Zhou, Tao; Nashed, Youssef Applied Physics Letters, Vol. 117, Issue 4 https://doi.org/10.1063/5.0013065	journal	July 2020
X-ray linear dichroic ptychography Lo, Yuan Hung; Zhou, Jihan; Rana, Arjun Proceedings of the National Academy of Sciences, Vol. 118, Issue 3 https://doi.org/10.1073/pnas.2019068118	journal	January 2021
Ptychographic atomic electron tomography: Towards three-dimensional imaging of individual light atoms in materials Chang, Dillan J.; Kim, Dennis S.; Rana, Arjun Physical Review B, Vol. 102, Issue 17 https://doi.org/10.1103/PhysRevB.102.174101	journal	November 2020
Potential of Attosecond Coherent Diffractive Imaging Rana, Arjun; Zhang, Jianhua; Pham, Minh Physical Review Letters, Vol. 125, Issue 8 https://doi.org/10.1103/PhysRevLett.125.086101	journal	August 2020
Hard-X-Ray Lensless Imaging of Extended Objects Rodenburg, J. M.; Hurst, A. C.; Cullis, A. G. Physical Review Letters, Vol. 98, Issue 3 https://doi.org/10.1103/PhysRevLett.98.034801	journal	January 2007
Beugung im inhomogenen Primärstrahlwellenfeld. I. Prinzip einer Phasenmessung von Elektronenbeungungsinterferenzen Hoppe, W. Acta Crystallographica Section A, Vol. 25, Issue 4 https://doi.org/10.1107/S0567739469001045	journal	July 1969
Complex imaging of phase domains by deep neural networks Wu, Longlong; Juhas, Pavol; Yoo, Shinjae IUCrJ, Vol. 8, Issue 1 https://doi.org/10.1107/S2052252520013780	journal	January 2021
Real-Time Interactive 4D-STEM Phase-Contrast Imaging From Electron Event Representation Data: Less computation with the right representation Pelz, Philipp M.; Johnson, Ian; Ophus, Colin IEEE Signal Processing Magazine, Vol. 39, Issue 1 https://doi.org/10.1109/MSP.2021.3120981	journal	January 2022
Image Super-Resolution Using Deep Convolutional Networks Dong, Chao; Loy, Chen Change; He, Kaiming IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 38, Issue 2 https://doi.org/10.1109/TPAMI.2015.2439281	journal	February 2016
Correlative 3D x-ray fluorescence and ptychographic tomography of frozen-hydrated green algae Deng, Junjing; Lo, Yuan Hung; Gallagher-Jones, Marcus Science Advances, Vol. 4, Issue 11 https://doi.org/10.1126/sciadv.aau4548	journal	November 2018
High-Resolution Scanning X-ray Diffraction Microscopy Thibault, P.; Dierolf, M.; Menzel, A. Science, Vol. 321, Issue 5887 https://doi.org/10.1126/science.1158573	journal	July 2008
Beyond crystallography: Diffractive imaging using coherent x-ray light sources Miao, J.; Ishikawa, T.; Robinson, I. K. Science, Vol. 348, Issue 6234 https://doi.org/10.1126/science.aaa1394	journal	April 2015
Atomic electron tomography: 3D structures without crystals Miao, J.; Ercius, P.; Billinge, S. J. L. Science, Vol. 353, Issue 6306 https://doi.org/10.1126/science.aaf2157	journal	September 2016
Electron ptychography achieves atomic-resolution limits set by lattice vibrations Chen, Zhen; Jiang, Yi; Shao, Yu-Tsun Science, Vol. 372, Issue 6544 https://doi.org/10.1126/science.abg2533	journal	May 2021
Phase retrieval from the magnitude of the Fourier transforms of nonperiodic objects Miao, J.; Sayre, D.; Chapman, H. N. Journal of the Optical Society of America A, Vol. 15, Issue 6 https://doi.org/10.1364/JOSAA.15.001662	journal	January 1998
On the interplay between physical and content priors in deep learning for computational imaging Deng, Mo; Li, Shuai; Zhang, Zhengyun Optics Express, Vol. 28, Issue 16 https://doi.org/10.1364/OE.395204	journal	January 2020
Lensless computational imaging through deep learning Sinha, Ayan; Lee, Justin; Li, Shuai Optica, Vol. 4, Issue 9 https://doi.org/10.1364/OPTICA.4.001117	journal	January 2017
Deep learning microscopy Rivenson, Yair; Göröcs, Zoltán; Günaydin, Harun Optica, Vol. 4, Issue 11 https://doi.org/10.1364/OPTICA.4.001437	journal	January 2017

Similar Records

Deep-Learning Electron Diffractive Imaging.

Journal Article · Sun Jan 01 04:00:00 UTC 2023 · Physical review letters · OSTI ID:1924475

Chang, Dillan J; O'Leary, Colum M; Su, Cong; +6 more

Achieving sub-0.5-angstrom–resolution ptychography in an uncorrected electron microscope

Journal Article · Thu Feb 22 00:00:00 UTC 2024 · Science · OSTI ID:2323561

Nguyen, Kayla X.; Jiang, Yi; Lee, Chia-Hao; +4 more

Real-time coherent diffraction inversion using deep generative networks

Journal Article · Thu Nov 08 00:00:00 UTC 2018 · Scientific Reports · OSTI ID:1619487

Cherukara, Mathew J.; Nashed, Youssef S. G.; Harder, Ross J.

Related Subjects

47 OTHER INSTRUMENTATION
convolutional neural networks
electron diffraction
electron microscopy

Title: Deep-Learning Electron Diffractive Imaging

Citation Formats

References (40)

Similar Records

Related Subjects