skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography

Abstract

Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.

Authors:
 [1];  [1];  [1];  [2];  [1];  [3];  [3]
  1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Center for Nanophase Materials Sciences
  2. Florida State Univ., Tallahassee, FL (United States)
  3. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Inst. for Functional Imaging of Materials; Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Materials Sciences and Technology Division
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22); Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
OSTI Identifier:
1261280
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
Scientific Reports
Additional Journal Information:
Journal Volume: 6; Journal ID: ISSN 2045-2322
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE; imaging techniques; transmission electron microscopy

Citation Formats

Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States: N. p., 2016. Web. doi:10.1038/srep26348.
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., & Lupini, A. R. Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography. United States. doi:10.1038/srep26348.
Jesse, S., Chi, M., Belianinov, A., Beekman, C., Kalinin, S. V., Borisevich, A. Y., and Lupini, A. R. Mon . "Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography". United States. doi:10.1038/srep26348. https://www.osti.gov/servlets/purl/1261280.
@article{osti_1261280,
title = {Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography},
author = {Jesse, S. and Chi, M. and Belianinov, A. and Beekman, C. and Kalinin, S. V. and Borisevich, A. Y. and Lupini, A. R.},
abstractNote = {Electron microscopy is undergoing a transition; from the model of producing only a few micrographs, through the current state where many images and spectra can be digitally recorded, to a new mode where very large volumes of data (movies, ptychographic and multi-dimensional series) can be rapidly obtained. In this paper, we discuss the application of so-called “big-data” methods to high dimensional microscopy data, using unsupervised multivariate statistical techniques, in order to explore salient image features in a specific example of BiFeO3 domains. Remarkably, k-means clustering reveals domain differentiation despite the fact that the algorithm is purely statistical in nature and does not require any prior information regarding the material, any coexisting phases, or any differentiating structures. While this is a somewhat trivial case, this example signifies the extraction of useful physical and structural information without any prior bias regarding the sample or the instrumental modality. Further interpretation of these types of results may still require human intervention. Finally, however, the open nature of this algorithm and its wide availability, enable broad collaborations and exploratory work necessary to enable efficient data analysis in electron microscopy.},
doi = {10.1038/srep26348},
journal = {Scientific Reports},
number = ,
volume = 6,
place = {United States},
year = {2016},
month = {5}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 5 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Detection of Single Atoms and Buried Defects in Three Dimensions by Aberration-Corrected Electron Microscope with 0.5-Å Information Limit
journal, September 2008


Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
journal, June 2014

  • Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
  • Nature Communications, Vol. 5, Issue 1
  • DOI: 10.1038/ncomms5155

Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
journal, August 2012

  • Kim, Young-Min; He, Jun; Biegalski, Michael D.
  • Nature Materials, Vol. 11, Issue 10
  • DOI: 10.1038/nmat3393

STEM imaging of 47-pm-separated atomic columns by a spherical aberration-corrected electron microscope with a 300-kV cold field emission gun
journal, June 2009

  • Sawada, H.; Tanishiro, Y.; Ohashi, N.
  • Journal of Electron Microscopy, Vol. 58, Issue 6
  • DOI: 10.1093/jmicro/dfp030

Direct observation of ferroelectric field effect and vacancy-controlled screening at the BiFeO3/LaxSr1−xMnO3 interface
journal, August 2014

  • Kim, Young-Min; Morozovska, Anna; Eliseev, Eugene
  • Nature Materials, Vol. 13, Issue 11
  • DOI: 10.1038/nmat4058

Atomically Resolved Mapping of Polarization and Electric Fields Across Ferroelectric/Oxide Interfaces by Z-contrast Imaging
journal, April 2011

  • Chang, Hye Jung; Kalinin, Sergei V.; Morozovska, Anna N.
  • Advanced Materials, Vol. 23, Issue 21
  • DOI: 10.1002/adma.201004641

Spontaneous Vortex Nanodomain Arrays at Ferroelectric Heterointerfaces
journal, February 2011

  • Nelson, Christopher T.; Winchester, Benjamin; Zhang, Yi
  • Nano Letters, Vol. 11, Issue 2
  • DOI: 10.1021/nl1041808

Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
journal, December 2006

  • Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
  • Nature Materials, Vol. 6, Issue 1
  • DOI: 10.1038/nmat1808

Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
journal, March 2011


Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
journal, March 2013


Suppression of Octahedral Tilts and Associated Changes in Electronic Properties at Epitaxial Oxide Heterostructure Interfaces
journal, August 2010


Control of Octahedral Tilts and Magnetic Properties of Perovskite Oxide Heterostructures by Substrate Symmetry
journal, November 2010


3D electron microscopy in the physical sciences: the development of Z-contrast and EFTEM tomography
journal, September 2003


Combined Scanning Transmission Electron Microscopy Tilt- and Focal Series
journal, February 2014

  • Dahmen, Tim; Baudoin, Jean-Pierre; Lupini, Andrew R.
  • Microscopy and Microanalysis, Vol. 20, Issue 2
  • DOI: 10.1017/S1431927614000075

Ptychographic microscope for three-dimensional imaging
journal, January 2014

  • Godden, T. M.; Suman, R.; Humphry, M. J.
  • Optics Express, Vol. 22, Issue 10
  • DOI: 10.1364/OE.22.012513

Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
journal, January 2012

  • Humphry, M. J.; Kraus, B.; Hurst, A. C.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1733

Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction
journal, December 2014

  • Müller, Knut; Krause, Florian F.; Béché, Armand
  • Nature Communications, Vol. 5, Issue 1
  • DOI: 10.1038/ncomms6653

Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
journal, April 2015


Blind analysis: Hide results to seek the truth
journal, October 2015

  • MacCoun, Robert; Perlmutter, Saul
  • Nature, Vol. 526, Issue 7572
  • DOI: 10.1038/526187a

Big–deep–smart data in imaging for guiding materials design
journal, September 2015

  • Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
  • Nature Materials, Vol. 14, Issue 10
  • DOI: 10.1038/nmat4395

Comment on "Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly"
journal, May 2003


Letter to the Editor: Limitations to the Measurement of Oxygen Concentrations by HRTEM Imposed by Surface Roughness
journal, March 2005

  • Lupini, Andrew R.; Chisholm, Matthew F.; van Benthem, Klaus
  • Microscopy and Microanalysis, Vol. 11, Issue 2
  • DOI: 10.1017/S1431927605210309

Complete information acquisition in dynamic force microscopy
journal, March 2015

  • Belianinov, Alexei; Kalinin, Sergei V.; Jesse, Stephen
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms7550

Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
journal, February 2016


Full information acquisition in piezoresponse force microscopy
journal, December 2015

  • Somnath, Suhas; Belianinov, Alexei; Kalinin, Sergei V.
  • Applied Physics Letters, Vol. 107, Issue 26
  • DOI: 10.1063/1.4938482

Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
journal, July 2013


Stress-induced R M A M C T symmetry changes in BiFeO 3 films
journal, April 2011


Prominent electrochromism through vacancy-order melting in a complex oxide
journal, January 2012

  • Seidel, J.; Luo, W.; Suresha, S. J.
  • Nature Communications, Vol. 3, Issue 1
  • DOI: 10.1038/ncomms1799

A Strain-Driven Morphotropic Phase Boundary in BiFeO3
journal, November 2009

  • Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
  • Science, Vol. 326, Issue 5955, p. 977-980
  • DOI: 10.1126/science.1177046

Aberration measurement using the Ronchigram contrast transfer function
journal, June 2010


Mechanisms for microstructure enhancement in flux-assisted growth of barium titanate on sapphire
journal, March 2014

  • Burch, Matthew J.; Li, Jing; Harris, David T.
  • Journal of Materials Research, Vol. 29, Issue 7
  • DOI: 10.1557/jmr.2014.59

Big-Data Reflection High Energy Electron Diffraction Analysis for Understanding Epitaxial Film Growth Processes
journal, October 2014

  • Vasudevan, Rama K.; Tselev, Alexander; Baddorf, Arthur P.
  • ACS Nano, Vol. 8, Issue 10
  • DOI: 10.1021/nn504730n

Big data in reciprocal space: Sliding fast Fourier transforms for determining periodicity
journal, March 2015

  • Vasudevan, Rama K.; Belianinov, Alex; Gianfrancesco, Anthony G.
  • Applied Physics Letters, Vol. 106, Issue 9
  • DOI: 10.1063/1.4914016

Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
journal, December 2014

  • Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
  • APL Materials, Vol. 2, Issue 12
  • DOI: 10.1063/1.4902996

Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
journal, October 2006


Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy
journal, February 2009


Graphene engineering by neon ion beams
journal, February 2016


Identification of phases, symmetries and defects through local crystallography
journal, July 2015

  • Belianinov, Alex; He, Qian; Kravchenko, Mikhail
  • Nature Communications, Vol. 6, Issue 1
  • DOI: 10.1038/ncomms8801

Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
journal, May 2014

  • Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
  • ACS Nano, Vol. 8, Issue 6
  • DOI: 10.1021/nn502029b

Constraining Data Mining with Physical Models: Voltage- and Oxygen Pressure-Dependent Transport in Multiferroic Nanostructures
journal, August 2015


Mapping internal structure of coal by confocal micro-Raman spectroscopy and scanning microwave microscopy
journal, June 2014


Algorithm AS 136: A K-Means Clustering Algorithm
journal, January 1979

  • Hartigan, J. A.; Wong, M. A.
  • Applied Statistics, Vol. 28, Issue 1
  • DOI: 10.2307/2346830

    Works referencing / citing this record:

    Automated defect analysis in electron microscopic images
    journal, July 2018


    Extensive deep neural networks for transferring small scale learning to large scale systems
    journal, January 2019

    • Mills, Kyle; Ryczko, Kevin; Luchak, Iryna
    • Chemical Science, Vol. 10, Issue 15
    • DOI: 10.1039/c8sc04578j

    Atomically Resolved Mapping of Polarization and Electric Fields Across Ferroelectric/Oxide Interfaces by Z-contrast Imaging
    journal, April 2011

    • Chang, Hye Jung; Kalinin, Sergei V.; Morozovska, Anna N.
    • Advanced Materials, Vol. 23, Issue 21
    • DOI: 10.1002/adma.201004641

    Interplay of Octahedral Tilts and Polar Order in BiFeO 3 Films
    journal, March 2013


    Phase Transitions, Phase Coexistence, and Piezoelectric Switching Behavior in Highly Strained BiFeO 3 Films
    journal, July 2013


    Mapping internal structure of coal by confocal micro-Raman spectroscopy and scanning microwave microscopy
    journal, June 2014


    Mapping chemical and bonding information using multivariate analysis of electron energy-loss spectrum images
    journal, October 2006


    Aberration measurement using the Ronchigram contrast transfer function
    journal, June 2010


    Efficient phase contrast imaging in STEM using a pixelated detector. Part 1: Experimental demonstration at atomic resolution
    journal, April 2015


    Constraining Data Mining with Physical Models: Voltage- and Oxygen Pressure-Dependent Transport in Multiferroic Nanostructures
    journal, August 2015


    Spontaneous Vortex Nanodomain Arrays at Ferroelectric Heterointerfaces
    journal, February 2011

    • Nelson, Christopher T.; Winchester, Benjamin; Zhang, Yi
    • Nano Letters, Vol. 11, Issue 2
    • DOI: 10.1021/nl1041808

    Deep Data Analysis of Conductive Phenomena on Complex Oxide Interfaces: Physics from Data Mining
    journal, May 2014

    • Strelcov, Evgheni; Belianinov, Alexei; Hsieh, Ying-Hui
    • ACS Nano, Vol. 8, Issue 6
    • DOI: 10.1021/nn502029b

    Big-Data Reflection High Energy Electron Diffraction Analysis for Understanding Epitaxial Film Growth Processes
    journal, October 2014

    • Vasudevan, Rama K.; Tselev, Alexander; Baddorf, Arthur P.
    • ACS Nano, Vol. 8, Issue 10
    • DOI: 10.1021/nn504730n

    Blind analysis: Hide results to seek the truth
    journal, October 2015

    • MacCoun, Robert; Perlmutter, Saul
    • Nature, Vol. 526, Issue 7572
    • DOI: 10.1038/526187a

    Ptychographic electron microscopy using high-angle dark-field scattering for sub-nanometre resolution imaging
    journal, January 2012

    • Humphry, M. J.; Kraus, B.; Hurst, A. C.
    • Nature Communications, Vol. 3, Issue 1
    • DOI: 10.1038/ncomms1733

    Prominent electrochromism through vacancy-order melting in a complex oxide
    journal, January 2012

    • Seidel, J.; Luo, W.; Suresha, S. J.
    • Nature Communications, Vol. 3, Issue 1
    • DOI: 10.1038/ncomms1799

    Picometre-precision analysis of scanning transmission electron microscopy images of platinum nanocatalysts
    journal, June 2014

    • Yankovich, Andrew B.; Berkels, Benjamin; Dahmen, W.
    • Nature Communications, Vol. 5, Issue 1
    • DOI: 10.1038/ncomms5155

    Atomic electric fields revealed by a quantum mechanical approach to electron picodiffraction
    journal, December 2014

    • Müller, Knut; Krause, Florian F.; Béché, Armand
    • Nature Communications, Vol. 5, Issue 1
    • DOI: 10.1038/ncomms6653

    Complete information acquisition in dynamic force microscopy
    journal, March 2015

    • Belianinov, Alexei; Kalinin, Sergei V.; Jesse, Stephen
    • Nature Communications, Vol. 6, Issue 1
    • DOI: 10.1038/ncomms7550

    Identification of phases, symmetries and defects through local crystallography
    journal, July 2015

    • Belianinov, Alex; He, Qian; Kravchenko, Mikhail
    • Nature Communications, Vol. 6, Issue 1
    • DOI: 10.1038/ncomms8801

    Unit-cell scale mapping of ferroelectricity and tetragonality in epitaxial ultrathin ferroelectric films
    journal, December 2006

    • Jia, Chun-Lin; Nagarajan, Valanoor; He, Jia-Qing
    • Nature Materials, Vol. 6, Issue 1
    • DOI: 10.1038/nmat1808

    Probing oxygen vacancy concentration and homogeneity in solid-oxide fuel-cell cathode materials on the subunit-cell level
    journal, August 2012

    • Kim, Young-Min; He, Jun; Biegalski, Michael D.
    • Nature Materials, Vol. 11, Issue 10
    • DOI: 10.1038/nmat3393

    Direct observation of ferroelectric field effect and vacancy-controlled screening at the BiFeO3/LaxSr1−xMnO3 interface
    journal, August 2014

    • Kim, Young-Min; Morozovska, Anna; Eliseev, Eugene
    • Nature Materials, Vol. 13, Issue 11
    • DOI: 10.1038/nmat4058

    Big–deep–smart data in imaging for guiding materials design
    journal, September 2015

    • Kalinin, Sergei V.; Sumpter, Bobby G.; Archibald, Richard K.
    • Nature Materials, Vol. 14, Issue 10
    • DOI: 10.1038/nmat4395

    Research Update: Spatially resolved mapping of electronic structure on atomic level by multivariate statistical analysis
    journal, December 2014

    • Belianinov, Alex; Ganesh, Panchapakesan; Lin, Wenzhi
    • APL Materials, Vol. 2, Issue 12
    • DOI: 10.1063/1.4902996

    Big data in reciprocal space: Sliding fast Fourier transforms for determining periodicity
    journal, March 2015

    • Vasudevan, Rama K.; Belianinov, Alex; Gianfrancesco, Anthony G.
    • Applied Physics Letters, Vol. 106, Issue 9
    • DOI: 10.1063/1.4914016

    Full information acquisition in piezoresponse force microscopy
    journal, December 2015

    • Somnath, Suhas; Belianinov, Alexei; Kalinin, Sergei V.
    • Applied Physics Letters, Vol. 107, Issue 26
    • DOI: 10.1063/1.4938482

    Principal component and spatial correlation analysis of spectroscopic-imaging data in scanning probe microscopy
    journal, February 2009


    Multifrequency spectrum analysis using fully digital G Mode-Kelvin probe force microscopy
    journal, February 2016


    Graphene engineering by neon ion beams
    journal, February 2016


    STEM imaging of 47-pm-separated atomic columns by a spherical aberration-corrected electron microscope with a 300-kV cold field emission gun
    journal, June 2009

    • Sawada, H.; Tanishiro, Y.; Ohashi, N.
    • Journal of Electron Microscopy, Vol. 58, Issue 6
    • DOI: 10.1093/jmicro/dfp030

    Comment on "Single Crystals of Single-Walled Carbon Nanotubes Formed by Self-Assembly"
    journal, May 2003


    A Strain-Driven Morphotropic Phase Boundary in BiFeO3
    journal, November 2009

    • Zeches, R. J.; Rossell, M. D.; Zhang, J. X.
    • Science, Vol. 326, Issue 5955, p. 977-980
    • DOI: 10.1126/science.1177046

    Direct Observation of Continuous Electric Dipole Rotation in Flux-Closure Domains in Ferroelectric Pb(Zr,Ti)O3
    journal, March 2011


    Mechanisms for microstructure enhancement in flux-assisted growth of barium titanate on sapphire
    journal, March 2014

    • Burch, Matthew J.; Li, Jing; Harris, David T.
    • Journal of Materials Research, Vol. 29, Issue 7
    • DOI: 10.1557/jmr.2014.59

    Algorithm AS 136: A K-Means Clustering Algorithm
    journal, January 1979

    • Hartigan, J. A.; Wong, M. A.
    • Applied Statistics, Vol. 28, Issue 1
    • DOI: 10.2307/2346830

    Automated defect analysis in electron microscopic images
    journal, July 2018


    Extensive deep neural networks for transferring small scale learning to large scale systems
    journal, January 2019

    • Mills, Kyle; Ryczko, Kevin; Luchak, Iryna
    • Chemical Science, Vol. 10, Issue 15
    • DOI: 10.1039/c8sc04578j