skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations

Journal Article · · ACS Nano
ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [3]; ORCiD logo [3]; ORCiD logo [3]; ORCiD logo [3]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Science (CNMS) and Inst. for Functional Imaging of Materials
  2. Univ. of Tennessee, Knoxville, TN (United States). Bredesen Center for Interdisciplinary Research and Graduate Education; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Science (CNMS) and Inst. for Functional Imaging of Materials
  3. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Science (CNMS)

Recent advances in scanning transmission electron and scanning probe microscopies have opened unprecedented opportunities in probing the materials structural parameters and various functional properties in real space with an angstrom-level precision. This progress has been accompanied by exponential increase in the size and quality of datasets produced by microscopic and spectroscopic experimental techniques. These developments necessitate adequate methods for extracting relevant physical and chemical information from the large datasets, for which a priori information on the structures of various atomic configurations and lattice defects is limited or absent. Here we demonstrate an application of deep neural networks to extracting information from atomically resolved images including location of the atomic species and type of defects. We develop a “weakly-supervised” approach that uses information on the coordinates of all atomic species in the image, extracted via a deep neural network, to identify a rich variety of defects that are not part of an initial training set. We further apply our approach to interpret complex atomic and defect transformation, including switching between different coordination of silicon dopants in graphene as a function of time, formation of peculiar silicon dimer with mixed 3-fold and 4-fold coordination, and the motion of molecular “rotor”. In conclusion, this deep learning based approach resembles logic of a human operator, but can be scaled leading to significant shift in the way of extracting and analyzing information from raw experimental data.

Research Organization:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division
Grant/Contract Number:
AC05-00OR22725
OSTI ID:
1427646
Journal Information:
ACS Nano, Vol. 11, Issue 12; ISSN 1936-0851
Publisher:
American Chemical Society (ACS)Copyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 193 works
Citation information provided by
Web of Science

References (41)

Atomic-Level Sculpting of Crystalline Oxides: Toward Bulk Nanofabrication with Single Atomic Plane Precision journal October 2015
Machine learning approaches in medical image analysis: From detection to diagnosis journal October 2016
Deep learning journal May 2015
Computational insights and the observation of SiC nanograin assembly: towards 2D silicon carbide journal June 2017
Oxygen-Vacancy-Induced Polar Behavior in (LaFeO 3 ) 2 /(SrFeO 3 ) Superlattices journal April 2014
Observing Atomic Collapse Resonances in Artificial Nuclei on Graphene journal March 2013
The perceptron: A probabilistic model for information storage and organization in the brain. journal January 1958
Fire up the atom forge journal November 2016
Lateral Hopping of Molecules Induced by Excitation of Internal Vibration Mode journal March 2002
Interplay of chemical disorder and electronic inhomogeneity in unconventional superconductors journal January 2013
Complex structural dynamics of nanocatalysts revealed in Operando conditions by correlated imaging and spectroscopy probes journal June 2015
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation journal December 2017
Isoelectronic Tungsten Doping in Monolayer MoSe 2 for Carrier Type Modulation journal July 2016
Effective Cleaning of Hexagonal Boron Nitride for Graphene Devices journal January 2012
Scanning Tunneling Spectroscopy journal July 2009
Single-atom fabrication with electron and ion beams: From surfaces and two-dimensional materials toward three-dimensional atom-by-atom assembly journal September 2017
Learning phase transitions by confusion journal February 2017
Machine learning phases of matter journal February 2017
Large scale atmospheric pressure chemical vapor deposition of graphene journal April 2013
Structural Defects in Graphene journal November 2010
Atom-by-atom structural and chemical analysis by annular dark-field electron microscopy journal March 2010
Machine learning in cell biology – teaching computers to recognize phenotypes journal November 2013
Direct Imaging of Covalent Bond Structure in Single-Molecule Chemical Reactions journal May 2013
Placing single atoms in graphene with a scanning transmission electron microscope journal September 2017
Surface faceting and elemental diffusion behaviour at atomic scale for alloy nanoparticles during in situ annealing journal November 2015
Observation of polar vortices in oxide superlattices journal January 2016
Controllable shrinking and shaping of silicon nitride nanopores under electron irradiation journal April 2007
Orbital Occupancy and Charge Doping in Iron-Based Superconductors journal July 2014
Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts journal June 2014
Direct Determination of the Chemical Bonding of Individual Impurities in Graphene journal November 2012
Fully Convolutional Networks for Semantic Segmentation journal April 2017
Direct Observation of Dopant Atom Diffusion in a Bulk Semiconductor Crystal Enhanced by a Large Size Mismatch journal October 2014
Functional and Spectroscopic Measurements with Scanning Tunneling Microscopy journal July 2008
Suppression of Defects and Deep Levels Using Isoelectronic Tungsten Substitution in Monolayer MoSe 2 journal October 2016
Searching for exotic particles in high-energy physics with deep learning journal July 2014
Identifying local structural states in atomic imaging by computer vision journal November 2016
Direct Observation of Atomic Dynamics and Silicon Doping at a Topological Defect in Graphene journal June 2014
Mechanochemistry Induced Using Force Exerted by a Functionalized Microscope Tip journal August 2017
Structure and bonding at the atomic scale by scanning transmission electron microscopy journal April 2009
Probing the Bonding and Electronic Structure of Single Atom Dopants in Graphene with Electron Energy Loss Spectroscopy journal January 2013
Mitigating e-beam-induced hydrocarbon deposition on graphene for atomic-scale scanning transmission electron microscopy studies
  • Dyck, Ondrej; Kim, Songkil; Kalinin, Sergei V.
  • Journal of Vacuum Science & Technology B, Nanotechnology and Microelectronics: Materials, Processing, Measurement, and Phenomena, Vol. 36, Issue 1 https://doi.org/10.1116/1.5003034
journal January 2018

Cited By (45)

Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks journal May 2019
Investigation of Polymer Coatings Formed by Polyvinyl Alcohol and Silver Nanoparticles on Copper Surface in Acid Medium by Means of Deep Convolutional Neural Networks journal February 2019
General Resolution Enhancement Method in Atomic Force Microscopy Using Deep Learning journal September 2018
Layer dependence of graphene-diamene phase transition in epitaxial and exfoliated few-layer graphene using machine learning journal June 2019
Design of visual communication based on deep learning approaches journal April 2019
Analyzing machine learning models to accelerate generation of fundamental materials insights journal March 2019
Electron-Beam Manipulation of Silicon Dopants in Graphene text January 2017
Automated defect analysis in electron microscopic images journal July 2018
Materials science in the artificial intelligence age: high-throughput library generation, machine learning, and a pathway from correlations to the underpinning physics journal July 2019
Unsupervised learning of ferroic variants from atomically resolved STEM images preprint January 2021
Deep Learning Analysis of Defect and Phase Evolution During Electron Beam Induced Transformations in WS2 text January 2018
Layer dependence of graphene-diamene phase transition in epitaxial and exfoliated few-layer graphene using machine learning preprint January 2019
Investigation of AFM-based machining of ferroelectric thin films at the nanoscale journal January 2020
Tap water fingerprinting using a convolutional neural network built from images of the coffee-ring effect journal January 2020
Deep neural networks for understanding noisy data applied to physical property extraction in scanning probe microscopy journal February 2019
Building and exploring libraries of atomic defects in graphene: Scanning transmission electron and scanning tunneling microscopy study journal September 2019
Charting the low-loss region in Electron Energy Loss Spectroscopy with machine learning preprint January 2020
Atom-by-atom fabrication with electron beams journal June 2019
TEMImageNet Training Library and AtomSegNet Deep-Learning Models for High-Precision Atom Segmentation, Localization, Denoising, and Super-Resolution Processing of Atomic-Resolution Images preprint January 2020
A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images journal July 2018
Robust multi-scale multi-feature deep learning for atomic and defect identification in Scanning Tunneling Microscopy on H-Si(100) 2x1 surface preprint January 2020
Deep learning analysis of defect and phase evolution during electron beam-induced transformations in WS2 journal February 2019
3D Crystal Image Analysis based on Fast Synchrosqueezed Transforms text January 2018
Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels journal September 2019
Atomic mechanisms for the Si atom dynamics in graphene: chemical transformations at the edge and in the bulk text January 2019
A deep learning approach for determining the chiral indices of carbon nanotubes from high-resolution transmission electron microscopy images text January 2020
A machine perspective of atomic defects in scanning transmission electron microscopy journal August 2019
Building ferroelectric from the bottom up: The machine learning analysis of the atomic-scale ferroelectric distortions journal July 2019
Off-the-shelf deep learning is not enough: parsimony, Bayes and causality preprint January 2020
Deep Learning Enabled Strain Mapping of Single-Atom Defects in 2D Transition Metal Dichalcogenides with Sub-picometer Precision text January 2020
Data-Driven Materials Investigations: The Next Frontier in Understanding and Predicting Fatigue Behavior journal May 2018
Inkjet-printed unclonable quantum dot fluorescent anti-counterfeiting labels with artificial intelligence authentication journal June 2019
Epitaxial TiO x Surface in Ferroelectric BaTiO 3 : Native Structure and Dynamic Patterning at the Atomic Scale journal June 2019
An artificial intelligence atomic force microscope enabled by machine learning journal January 2018
Deep data analytics for genetic engineering of diatoms linking genotype to phenotype via machine learning journal June 2019
Deep learning in nano-photonics: inverse design and beyond text January 2020
Robust recognition and exploratory analysis of crystal structures via Bayesian deep learning journal October 2021
Atomic Mechanisms for the Si Atom Dynamics in Graphene: Chemical Transformations at the Edge and in the Bulk journal November 2019
Artificial Intelligent Atomic Force Microscope Enabled by Machine Learning text January 2018
The Rise of Catalyst Informatics: Towards Catalyst Genomics journal January 2019
Revealing the chemical bonding in adatoms arrays via machine learning of 3D scanning tunneling spectroscopy data text January 2021
Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks preprint January 2018
Deep learning for accelerated all-dielectric metasurface design journal January 2019
Quantifying the Dynamics of Protein Self-Organization Using Deep Learning Analysis of Atomic Force Microscopy Data journal December 2020
Mapping mesoscopic phase evolution during E-beam induced transformations via deep learning of atomically resolved images journal June 2018