DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography

Abstract

Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.

Authors:
 [1];  [1];  [1];  [2];  [1];  [3];  [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences
  2. Florida State Univ., Tallahassee, FL (United States)
  3. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Materials Sciences and Technology Division
Publication Date:
Research Org.:
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Materials Sciences & Engineering Division; USDOE Laboratory Directed Research and Development (LDRD) Program
OSTI Identifier:
1261280
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
Scientific Reports
Additional Journal Information:
Journal Volume: 6; Journal ID: ISSN 2045-2322
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE; imaging techniques; transmission electron microscopy

Citation Formats

Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States: N. p., 2016. Web. doi:10.1038/srep26348.
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., & Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States. https://doi.org/10.1038/srep26348
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Mon . "Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography". United States. https://doi.org/10.1038/srep26348. https://www.osti.gov/servlets/purl/1261280.
@article{osti_1261280,
title = {Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography},
author = {Jesse, S. and Chi, M. and Belianinov, A. and Beekman, C. and Kalinin, S. V. and Borisevich, A. Y. and Lupini, A. R.},
abstractNote = {Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.},
doi = {10.1038/srep26348},
journal = {Scientific Reports},
number = ,
volume = 6,
place = {United States},
year = {Mon May 23 00:00:00 EDT 2016},
month = {Mon May 23 00:00:00 EDT 2016}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 48 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
journal, December 2014

  • Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
  • APL Materials, Vol. 2, Issue 12
  • DOI: 10.1063/1.4902996

Ptychographic microscope for three-dimensional imaging
journal, January 2014

  • Godden, T. M.; Suman, R.; Humphry, M. J.
  • Optics Express, Vol. 22, Issue 10
  • DOI: 10.1364/OE.22.012513

Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy
journal, February 2009


A Strain-Driven Morphotropic Phase Boundary in BiFeO3
journal, November 2009

  • Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
  • Science, Vol. 326, Issue 5955, p. 977-980
  • DOI: 10.1126/science.1177046

Aberration measurement using the Ronchigram contrast transfer function
journal, June 2010


Mapping internal structure of coal by confocal micro-Raman spectroscopy and scanning microwave microscopy
journal, June 2014


Ptychography and Related Diffractive Imaging Methods
book, May 2008


Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
journal, March 2013


Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
journal, January 2012

  • Humphry, M. J.; Kraus, B.; Hurst, A. C.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1733

STEM imaging of 47-pm-separated atomic columns by a spherical aberration-corrected electron microscope with a 300-kV cold field emission gun
journal, June 2009

  • Sawada, H.; Tanishiro, Y.; Ohashi, N.
  • Journal of Electron Microscopy, Vol. 58, Issue 6
  • DOI: 10.1093/jmicro/dfp030

Big–deep–smart data in imaging for guiding materials design
journal, September 2015

  • Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
  • Nature Materials, Vol. 14, Issue 10
  • DOI: 10.1038/nmat4395

Mechanisms for microstructure enhancement in flux-assisted growth of barium titanate on sapphire
journal, March 2014

  • Burch, Matthew J.; Li, Jing; Harris, David T.
  • Journal of Materials Research, Vol. 29, Issue 7
  • DOI: 10.1557/jmr.2014.59

Suppression of Octahedral Tilts and Associated Changes in Electronic Properties at Epitaxial Oxide Heterostructure Interfaces
journal, August 2010


Complete information acquisition in dynamic force microscopy
journal, March 2015

  • Belianinov, Alexei; Kalinin, Sergei V.; Jesse, Stephen
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms7550

Letter to the Editor: Limitations to the Measurement of Oxygen Concentrations by HRTEM Imposed by Surface Roughness
journal, March 2005

  • Lupini, Andrew R.; Chisholm, Matthew F.; van Benthem, Klaus
  • Microscopy and Microanalysis, Vol. 11, Issue 2
  • DOI: 10.1017/S1431927605210309

Direct observation of ferroelectric field effect and vacancy-controlled screening at the BiFeO3/LaxSr1−xMnO3 interface
journal, August 2014

  • Kim, Young-Min; Morozovska, Anna; Eliseev, Eugene
  • Nature Materials, Vol. 13, Issue 11
  • DOI: 10.1038/nmat4058

Constraining Data Mining with Physical Models: Voltage- and Oxygen Pressure-Dependent Transport in Multiferroic Nanostructures
journal, August 2015


Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction
journal, December 2014

  • Müller, Knut; Krause, Florian F.; Béché, Armand
  • Nature Communications, Vol. 5, Issue 1
  • DOI: 10.1038/ncomms6653

Combined Scanning Transmission Electron Microscopy Tilt- and Focal Series
journal, February 2014

  • Dahmen, Tim; Baudoin, Jean-Pierre; Lupini, Andrew R.
  • Microscopy and Microanalysis, Vol. 20, Issue 2
  • DOI: 10.1017/S1431927614000075

Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
journal, March 2011


Spontaneous Vortex Nanodomain Arrays at Ferroelectric Heterointerfaces
journal, February 2011

  • Nelson, Christopher T.; Winchester, Benjamin; Zhang, Yi
  • Nano Letters, Vol. 11, Issue 2
  • DOI: 10.1021/nl1041808

Graphene engineering by neon ion beams
journal, February 2016


Prominent electrochromism through vacancy-order melting in a complex oxide
journal, January 2012

  • Seidel, J.; Luo, W.; Suresha, S. J.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1799

3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography
journal, September 2003


Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
journal, June 2014

  • Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
  • Nature Communications, Vol. 5, Issue 1
  • DOI: 10.1038/ncomms5155

Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
journal, April 2015


Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006


Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
journal, July 2013


Full information acquisition in piezoresponse force microscopy
journal, December 2015

  • Somnath, Suhas; Belianinov, Alexei; Kalinin, Sergei V.
  • Applied Physics Letters, Vol. 107, Issue 26
  • DOI: 10.1063/1.4938482

Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
journal, May 2014

  • Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
  • ACS Nano, Vol. 8, Issue 6
  • DOI: 10.1021/nn502029b

Comment on "Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly"
journal, May 2003


Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979

  • Hartigan, J. A.; Wong, M. A.
  • Applied Statistics, Vol. 28, Issue 1
  • DOI: 10.2307/2346830

Control of Octahedral Tilts and Magnetic Properties of Perovskite Oxide Heterostructures by Substrate Symmetry
journal, November 2010


Identification of phases, symmetries and defects through local crystallography
journal, July 2015

  • Belianinov, Alex; He, Qian; Kravchenko, Mikhail
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms8801

Big data in reciprocal space: Sliding fast Fourier transforms for determining periodicity
journal, March 2015

  • Vasudevan, Rama K.; Belianinov, Alex; Gianfrancesco, Anthony G.
  • Applied Physics Letters, Vol. 106, Issue 9
  • DOI: 10.1063/1.4914016

Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
journal, February 2016


Stress-induced R M A M C T symmetry changes in BiFeO 3 films
journal, April 2011


Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
journal, December 2006

  • Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
  • Nature Materials, Vol. 6, Issue 1
  • DOI: 10.1038/nmat1808

Big-Data Reflection High Energy Electron Diffraction Analysis for Understanding Epitaxial Film Growth Processes
journal, October 2014

  • Vasudevan, Rama K.; Tselev, Alexander; Baddorf, Arthur P.
  • ACS Nano, Vol. 8, Issue 10
  • DOI: 10.1021/nn504730n

Atomically Resolved Mapping of Polarization and Electric Fields Across Ferroelectric/Oxide Interfaces by Z-contrast Imaging
journal, April 2011

  • Chang, Hye Jung; Kalinin, Sergei V.; Morozovska, Anna N.
  • Advanced Materials, Vol. 23, Issue 21
  • DOI: 10.1002/adma.201004641

Blind analysis: Hide results to seek the truth
journal, October 2015

  • MacCoun, Robert; Perlmutter, Saul
  • Nature, Vol. 526, Issue 7572
  • DOI: 10.1038/526187a

Detection of Single Atoms and Buried Defects in Three Dimensions by Aberration-Corrected Electron Microscope with 0.5-Å Information Limit
journal, September 2008


Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
journal, August 2012

  • Kim, Young-Min; He, Jun; Biegalski, Michael D.
  • Nature Materials, Vol. 11, Issue 10
  • DOI: 10.1038/nmat3393

Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
journal, March 2013


Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
journal, July 2013


Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006


Aberration measurement using the Ronchigram contrast transfer function
journal, June 2010


Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
journal, April 2015


Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
journal, May 2014

  • Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
  • ACS Nano, Vol. 8, Issue 6
  • DOI: 10.1021/nn502029b

Blind analysis: Hide results to seek the truth
journal, October 2015

  • MacCoun, Robert; Perlmutter, Saul
  • Nature, Vol. 526, Issue 7572
  • DOI: 10.1038/526187a

Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
journal, January 2012

  • Humphry, M. J.; Kraus, B.; Hurst, A. C.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1733

Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
journal, June 2014

  • Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
  • Nature Communications, Vol. 5, Issue 1
  • DOI: 10.1038/ncomms5155

Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
journal, December 2006

  • Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
  • Nature Materials, Vol. 6, Issue 1
  • DOI: 10.1038/nmat1808

Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
journal, August 2012

  • Kim, Young-Min; He, Jun; Biegalski, Michael D.
  • Nature Materials, Vol. 11, Issue 10
  • DOI: 10.1038/nmat3393

Big–deep–smart data in imaging for guiding materials design
journal, September 2015

  • Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
  • Nature Materials, Vol. 14, Issue 10
  • DOI: 10.1038/nmat4395

Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
journal, December 2014

  • Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
  • APL Materials, Vol. 2, Issue 12
  • DOI: 10.1063/1.4902996

Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
journal, February 2016


Graphene engineering by neon ion beams
journal, February 2016


Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly
journal, April 2001


A Strain-Driven Morphotropic Phase Boundary in BiFeO3
journal, November 2009

  • Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
  • Science, Vol. 326, Issue 5955, p. 977-980
  • DOI: 10.1126/science.1177046

Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
journal, March 2011


Erratum: Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
journal, August 2015

  • Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
  • Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
  • DOI: 10.1186/s40679-015-0011-9

Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979

  • Hartigan, J. A.; Wong, M. A.
  • Applied Statistics, Vol. 28, Issue 1
  • DOI: 10.2307/2346830

Works referencing / citing this record:

LiberTEM/LiberTEM: 0.4.1
software, February 2020


Direct and continuous strain control of catalysts with tunable battery electrode materials
journal, November 2016


LiberTEM/LiberTEM: 0.3.0
software, December 2019


Frontiers in strain-engineered multifunctional ferroic materials
journal, August 2016

  • Agar, Joshua C.; Pandya, Shishir; Xu, Ruijuan
  • MRS Communications, Vol. 6, Issue 3
  • DOI: 10.1557/mrc.2016.29

LiberTEM/LiberTEM: 0.2.0
software, October 2019


LiberTEM/LiberTEM-blobfinder: 0.4.0
software, February 2020


Automated defect analysis in electron microscopic images
journal, July 2018


LiberTEM/LiberTEM-blobfinder: 0.4.0
software, February 2020


Machine Detection of Enhanced Electromechanical Energy Conversion in PbZr 0.2 Ti 0.8 O 3 Thin Films
journal, May 2018


LiberTEM/LiberTEM: 0.2.2
software, October 2019


Machine learning-based multidomain processing for texture-based image segmentation and analysis
journal, January 2020

  • Borodinov, Nikolay; Tsai, Wan-Yu; Korolkov, Vladimir V.
  • Applied Physics Letters, Vol. 116, Issue 4
  • DOI: 10.1063/1.5135328

Manifold learning of four-dimensional scanning transmission electron microscopy
journal, January 2019


Functional Materials Under Stress: In Situ TEM Observations of Structural Evolution
journal, November 2019


Ptychography 4.0: 0.1.0
software, January 2021


Ptychography 4.0: 0.1.0
software, July 2021


LiberTEM/LiberTEM: 0.2.1
software, October 2019


LiberTEM/LiberTEM: 0.4.0
software, February 2020


Smart machine learning or discovering meaningful physical and chemical contributions through dimensional stacking
journal, August 2019

  • Griffin, Lee A.; Gaponenko, Iaroslav; Zhang, Shujun
  • npj Computational Materials, Vol. 5, Issue 1
  • DOI: 10.1038/s41524-019-0222-z

LiberTEM/LiberTEM: 0.5.1
software, August 2020


LiberTEM/LiberTEM: 0.5.0
software, April 2020


Correcting the linear and nonlinear distortions for atomically resolved STEM spectrum and diffraction imaging
journal, January 2018

  • Wang, Yi; Suyolcu, Y. Eren; Salzberger, Ute
  • Microscopy, Vol. 67, Issue suppl_1
  • DOI: 10.1093/jmicro/dfy002

Nonuniform Sparse Data Clustering Cascade Algorithm Based on Dynamic Cumulative Entropy
journal, January 2016

  • Li, Ning; Gu, Yunxia; Deng, Zhongliang
  • Mathematical Problems in Engineering, Vol. 2016
  • DOI: 10.1155/2016/5707692