Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks

Zhang, Chenyu; Feng, Jie; DaCosta, Luis Rangel; Voyles, Paul. M.

doi:10.1016/j.ultramic.2019.112921

Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks

Journal Article · Sun Mar 01 00:00:00 EST 2020 · Ultramicroscopy

DOI:https://doi.org/10.1016/j.ultramic.2019.112921· OSTI ID:1593819

Zhang, Chenyu ^[1]; Feng, Jie ^[2]; DaCosta, Luis Rangel ^[2]; Voyles, Paul. M. ^[2]

Univ. of Wisconsin, Madison, WI (United States). Dept. of Materials Science and Engineering; University of Wisconsin - Madison, Department of Materials Science and Engineering
Univ. of Wisconsin, Madison, WI (United States). Dept. of Materials Science and Engineering

Two types of convolutional neural network (CNN) models, a discrete classification network and a continuous regression network, were trained to determine local sample thickness from convergent beam diffraction (CBED) patterns of SrTiO₃ collected in a scanning transmission electron microscope (STEM) at atomic column resolution. Acquisition of atomic resolution CBED patterns for this purpose requires careful balancing of CBED feature size in pixels, acquisition speed, and detector dynamic range. The training datasets were derived from multislice simulations, which must be convolved with incoherent source broadening. Sample thicknesses were also determined using quantitative high-angle annular dark-field (HAADF) STEM images acquired simultaneously. The regression CNN performed well on sample thinner than 35 nm, with 70% of the CNN results within 1 nm of HAADF thickness, and 1.0 nm overall root mean square error between the two measurements. The classification CNN was trained for a thicknesses up to 100 nm and yielded 66% of CNN results within one classification increment of 2 nm of HAADF thickness. Our approach depends on methods from computer vision including transfer learning and image augmentation.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of Wisconsin, Madison, WI (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22)

Contributing Organization:: Wisconsin REU program

Grant/Contract Number:: FG02-08ER46547

OSTI ID:: 1593819

Alternate ID(s):: OSTI ID: 1592812

Journal Information:: Ultramicroscopy, Journal Name: Ultramicroscopy Journal Issue: C Vol. 210; ISSN 0304-3991

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (41)

Supporting user-oriented analysis for multi-view domain-specific visual languages Guerra, Esther; de Lara, Juan; Malizia, Alessio Information and Software Technology, Vol. 51, Issue 4 https://doi.org/10.1016/j.infsof.2008.09.005	journal	April 2009
Applications of direct detection device in transmission electron microscopy Jin, Liang; Milazzo, Anna-Clare; Kleinfelder, Stuart Journal of Structural Biology, Vol. 161, Issue 3 https://doi.org/10.1016/j.jsb.2007.10.007	journal	March 2008
Position averaged convergent beam electron diffraction: Theory and applications LeBeau, James M.; Findlay, Scott D.; Allen, Leslie J. Ultramicroscopy, Vol. 110, Issue 2 https://doi.org/10.1016/j.ultramic.2009.10.001	journal	January 2010
Measurement of effective source distribution and its importance for quantitative interpretation of STEM images Dwyer, C.; Erni, R.; Etheridge, J. Ultramicroscopy, Vol. 110, Issue 8 https://doi.org/10.1016/j.ultramic.2010.01.007	journal	July 2010
Comparison of optimal performance at 300keV of three direct electron detectors for use in low dose electron microscopy McMullan, G.; Faruqi, A. R.; Clare, D. Ultramicroscopy, Vol. 147 https://doi.org/10.1016/j.ultramic.2014.08.002	journal	December 2014
Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution Pennycook, Timothy J.; Lupini, Andrew R.; Yang, Hao Ultramicroscopy, Vol. 151, p. 160-167 https://doi.org/10.1016/j.ultramic.2014.09.013	journal	April 2015
Efficient phase contrast imaging in STEM using a pixelated detector. Part II: Optimisation of imaging conditions Yang, Hao; Pennycook, Timothy J.; Nellist, Peter D. Ultramicroscopy, Vol. 151 https://doi.org/10.1016/j.ultramic.2014.10.013	journal	April 2015
Characterizing the response of a scintillator-based detector to single electrons Sang, Xiahan; LeBeau, James M. Ultramicroscopy, Vol. 161 https://doi.org/10.1016/j.ultramic.2015.11.008	journal	February 2016
Optimizing disk registration algorithms for nanobeam electron diffraction strain mapping Pekin, Thomas C.; Gammer, Christoph; Ciston, Jim Ultramicroscopy, Vol. 176 https://doi.org/10.1016/j.ultramic.2016.12.021	journal	May 2017
Accuracy and precision of thickness determination from position-averaged convergent beam electron diffraction patterns using a single-parameter metric Pollock, J. A.; Weyland, M.; Taplin, D. J. Ultramicroscopy, Vol. 181 https://doi.org/10.1016/j.ultramic.2017.05.001	journal	October 2017
A deep convolutional neural network to analyze position averaged convergent beam electron diffraction patterns Xu, W.; LeBeau, J. M. Ultramicroscopy, Vol. 188 https://doi.org/10.1016/j.ultramic.2018.03.004	journal	May 2018
Direct determination of structural heterogeneity in metallic glasses using four-dimensional scanning transmission electron microscopy Im, Soohyun; Chen, Zhen; Johnson, Jared M. Ultramicroscopy, Vol. 195 https://doi.org/10.1016/j.ultramic.2018.09.005	journal	December 2018
Identifying and Correcting Scan Noise and Drift in the Scanning Transmission Electron Microscope Jones, Lewys; Nellist, Peter D. Microscopy and Microanalysis, Vol. 19, Issue 4 https://doi.org/10.1017/S1431927613001402	journal	May 2013
Direct Visualization of Local Electromagnetic Field Structures by Scanning Transmission Electron Microscopy Shibata, Naoya; Findlay, Scott D.; Matsumoto, Takao Accounts of Chemical Research, Vol. 50, Issue 7 https://doi.org/10.1021/acs.accounts.7b00123	journal	July 2017
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem ACS Nano, Vol. 11, Issue 12 https://doi.org/10.1021/acsnano.7b07504	journal	October 2016
Standardless Atom Counting in Scanning Transmission Electron Microscopy LeBeau, James M.; Findlay, Scott D.; Allen, Leslie J. Nano Letters, Vol. 10, Issue 11 https://doi.org/10.1021/nl102025s	journal	November 2010
Simultaneous atomic-resolution electron ptychography and Z-contrast imaging of light and heavy elements in complex nanostructures Yang, H.; Rutte, R. N.; Jones, L. Nature Communications, Vol. 7, Issue 1 https://doi.org/10.1038/ncomms12532	journal	August 2016
Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W. Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms5155	journal	June 2014
Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction Müller, Knut; Krause, Florian F.; Béché, Armand Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms6653	journal	December 2014
Electron counting and beam-induced motion correction enable near-atomic-resolution single-particle cryo-EM Li, Xueming; Mooney, Paul; Zheng, Shawn Nature Methods, Vol. 10, Issue 6 https://doi.org/10.1038/nmeth.2472	journal	May 2013
Electron ptychographic microscopy for three-dimensional imaging Gao, Si; Wang, Peng; Zhang, Fucai Nature Communications, Vol. 8, Issue 1 https://doi.org/10.1038/s41467-017-00150-1	journal	July 2017
Electron ptychography of 2D materials to deep sub-ångström resolution Jiang, Yi; Chen, Zhen; Han, Yimo Nature, Vol. 559, Issue 7714 https://doi.org/10.1038/s41586-018-0298-5	journal	July 2018
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography Jesse, S.; Chi, M.; Belianinov, A. Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep26348	journal	May 2016
Jointed magnetic skyrmion lattices at a small-angle grain boundary directly visualized by advanced electron microscopy Matsumoto, Takao; So, Yeong-Gi; Kohno, Yuji Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep35880	journal	October 2016
Method to measure spatial coherence of subangstrom electron beams Dwyer, Christian; Erni, Rolf; Etheridge, Joanne Applied Physics Letters, Vol. 93, Issue 2 https://doi.org/10.1063/1.2957648	journal	July 2008
Determining ferroelectric polarity at the nanoscale LeBeau, James M.; D’Alfonso, Adrian J.; Wright, Nicholas J. Applied Physics Letters, Vol. 98, Issue 5 https://doi.org/10.1063/1.3549300	journal	January 2011
Nanoscale quantification of octahedral tilts in perovskite films Hwang, Jinwoo; Zhang, Jack Y.; Son, Junwoo Applied Physics Letters, Vol. 100, Issue 19 https://doi.org/10.1063/1.4714734	journal	May 2012
Strain mapping at nanometer resolution using advanced nano-beam electron diffraction Ozdol, V. B.; Gammer, C.; Jin, X. G. Applied Physics Letters, Vol. 106, Issue 25 https://doi.org/10.1063/1.4922994	journal	June 2015
Non-spectroscopic composition measurements of SrTiO ₃ -La _0.7 Sr _0.3 MnO ₃ multilayers using scanning convergent beam electron diffraction Ophus, Colin; Ercius, Peter; Huijben, Mark Applied Physics Letters, Vol. 110, Issue 6 https://doi.org/10.1063/1.4975932	journal	February 2017
Non-rigid registration and non-local principle component analysis to improve electron microscopy spectrum images Yankovich, Andrew B.; Zhang, Chenyu; Oh, Albert Nanotechnology, Vol. 27, Issue 36 https://doi.org/10.1088/0957-4484/27/36/364001	journal	August 2016
A pnCCD-based, fast direct single electron imaging camera for TEM and STEM Ryll, H.; Simson, M.; Hartmann, R. Journal of Instrumentation, Vol. 11, Issue 04 https://doi.org/10.1088/1748-0221/11/04/P04006	journal	April 2016
Quantitative Atomic Resolution Scanning Transmission Electron Microscopy LeBeau, James M.; Findlay, Scott D.; Allen, Leslie J. Physical Review Letters, Vol. 100, Issue 20 https://doi.org/10.1103/PhysRevLett.100.206101	journal	May 2008
Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties Xie, Tian; Grossman, Jeffrey C. Physical Review Letters, Vol. 120, Issue 14 https://doi.org/10.1103/PhysRevLett.120.145301	journal	April 2018
Computation in electron microscopy Kirkland, Earl J. Acta Crystallographica Section A Foundations and Advances, Vol. 72, Issue 1 https://doi.org/10.1107/S205327331501757X	journal	January 2016
A survey of decision tree classifier methodology Safavian, S. R.; Landgrebe, D. IEEE Transactions on Systems, Man, and Cybernetics, Vol. 21, Issue 3 https://doi.org/10.1109/21.97458	journal	January 1991
Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning Shin, Hoo-Chang; Roth, Holger R.; Gao, Mingchen IEEE Transactions on Medical Imaging, Vol. 35, Issue 5 https://doi.org/10.1109/TMI.2016.2528162	journal	May 2016
High-precision scanning transmission electron microscopy at coarse pixel sampling for reduced electron dose Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, Wolfgang Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0003-9	journal	March 2015
Poisson noise removal from high-resolution STEM images based on periodic block matching Mevenkamp, Niklas; Binev, Peter; Dahmen, Wolfgang Advanced Structural and Chemical Imaging, Vol. 1, Issue 1 https://doi.org/10.1186/s40679-015-0004-8	journal	March 2015
A streaming multi-GPU implementation of image simulation algorithms for scanning transmission electron microscopy Pryor, Alan; Ophus, Colin; Miao, Jianwei Advanced Structural and Chemical Imaging, Vol. 3, Issue 1 https://doi.org/10.1186/s40679-017-0048-z	journal	October 2017
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography Somnath, Suhas; Jesse, Stephen; Belianinov, Alex Oak Ridge Leadership Computing Facility; Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States) https://doi.org/10.13139/olcf/1463599	dataset	January 2018
Convolutional Neural Networks for Sentence Classification Kim, Yoon Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) https://doi.org/10.3115/v1/D14-1181	conference	January 2014

Similar Records

Convolutional neural network identification of galaxy post-mergers in UNIONS using IllustrisTNG

Journal Article · Thu Mar 18 20:00:00 EDT 2021 · Monthly Notices of the Royal Astronomical Society · OSTI ID:1806345

Classifying Radio Galaxies with the Convolutional Neural Network

Journal Article · Thu Jun 01 00:00:00 EDT 2017 · Astrophysical Journal, Supplement Series · OSTI ID:22661117

Classifying T cell activity in autofluorescence intensity images with convolutional neural networks

Journal Article · Sat Dec 14 19:00:00 EST 2019 · Journal of Biophotonics · OSTI ID:1623563

Related Subjects

36 MATERIALS SCIENCE
Convergent beam electron diffraction
Convolutional neural network
Deep learning
Machine learning
Scanning transmission electron microscopy

Atomic resolution convergent beam electron diffraction analysis using convolutional neural networks

Citation Formats

References (41)

Linked Research (1)

Similar Records

Related Subjects