skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Automated defect analysis in electron microscopic images

Abstract

Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a rangemore » of defect types, materials, and electron microscopic techniques.« less

Authors:
ORCiD logo [1]; ORCiD logo [2]; ORCiD logo [1]
  1. Univ. of Wisconsin, Madison, WI (United States)
  2. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Publication Date:
Research Org.:
Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
1462882
Grant/Contract Number:  
AC05-00OR22725
Resource Type:
Accepted Manuscript
Journal Name:
npj Computational Materials
Additional Journal Information:
Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2057-3960
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
47 OTHER INSTRUMENTATION

Citation Formats

Li, Wei, Field, Kevin G., and Morgan, Dane. Automated defect analysis in electron microscopic images. United States: N. p., 2018. Web. doi:10.1038/s41524-018-0093-8.
Li, Wei, Field, Kevin G., & Morgan, Dane. Automated defect analysis in electron microscopic images. United States. doi:10.1038/s41524-018-0093-8.
Li, Wei, Field, Kevin G., and Morgan, Dane. Wed . "Automated defect analysis in electron microscopic images". United States. doi:10.1038/s41524-018-0093-8. https://www.osti.gov/servlets/purl/1462882.
@article{osti_1462882,
title = {Automated defect analysis in electron microscopic images},
author = {Li, Wei and Field, Kevin G. and Morgan, Dane},
abstractNote = {Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a range of defect types, materials, and electron microscopic techniques.},
doi = {10.1038/s41524-018-0093-8},
journal = {npj Computational Materials},
number = 1,
volume = 4,
place = {United States},
year = {2018},
month = {7}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 7 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Deep learning
journal, May 2015

  • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
  • Nature, Vol. 521, Issue 7553
  • DOI: 10.1038/nature14539

Face Description with Local Binary Patterns: Application to Face Recognition
journal, December 2006

  • Ahonen, T.; Hadid, A.; Pietikainen, M.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 12
  • DOI: 10.1109/TPAMI.2006.244

A practical implementation of face detection by using Matlab cascade object detector
conference, October 2015

  • Alionte, Elena; Lazar, Corneliu
  • 2015 19th International Conference on System Theory, Control and Computing (ICSTCC)
  • DOI: 10.1109/ICSTCC.2015.7321390

Image driven machine learning methods for microstructure recognition
journal, October 2016


Robust Real-Time Face Detection
journal, May 2004


Histograms of Oriented Gradients for Human Detection
conference, January 2005

  • Dalal, N.; Triggs, B.
  • 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
  • DOI: 10.1109/CVPR.2005.177

Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures
journal, July 2017


A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection
conference, August 2014

  • Sharifara, Ali; Mohd Rahim, Mohd Shafry; Anisi, Yasaman
  • 2014 International Symposium on Biometrics and Security Technologies (ISBAST)
  • DOI: 10.1109/ISBAST.2014.7013097

Rethinking the Inception Architecture for Computer Vision
conference, June 2016

  • Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2016.308

Characterization of irradiation defect structures and densities by transmission electron microscopy
journal, February 2015

  • Kirk, Marquis; Yi, Xiaoou; Jenkins, Michael
  • Journal of Materials Research, Vol. 30, Issue 9
  • DOI: 10.1557/jmr.2015.19

Fully convolutional networks for semantic segmentation
conference, June 2015

  • Long, Jonathan; Shelhamer, Evan; Darrell, Trevor
  • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2015.7298965

Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa
journal, November 2017


Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography
journal, May 2016

  • Jesse, S.; Chi, M.; Belianinov, A.
  • Scientific Reports, Vol. 6, Issue 1
  • DOI: 10.1038/srep26348

Inferring low-dimensional microstructure representations using convolutional neural networks
journal, November 2017


Watersheds in digital spaces: an efficient algorithm based on immersion simulations
journal, June 1991

  • Vincent, L.; Soille, P.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, Issue 6
  • DOI: 10.1109/34.87344

Structural materials for fission & fusion energy
journal, November 2009


A Method for Detection and Classification of Glass Defects in Low Resolution Images
conference, August 2011

  • Zhao, Jie; Kong, Qing-Jie; Zhao, Xu
  • Graphics (ICIG), 2011 Sixth International Conference on Image and Graphics
  • DOI: 10.1109/ICIG.2011.187

Shape matching and object recognition using shape contexts
journal, April 2002

  • Belongie, S.; Malik, J.; Puzicha, J.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 4
  • DOI: 10.1109/34.993558

TEM characterization of dislocation loops in irradiated bcc Fe-based steels
journal, March 2013


Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images
journal, October 2011


Diffraction contrast STEM of dislocations: Imaging and simulations
journal, August 2011


Face Detection with the Faster R-CNN
conference, May 2017

  • Jiang, Huaizu; Learned-Miller, Erik
  • 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)
  • DOI: 10.1109/FG.2017.82

Object Detection with Discriminatively Trained Part-Based Models
journal, September 2010

  • Felzenszwalb, P. F.; Girshick, R. B.; McAllester, D.
  • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, Issue 9
  • DOI: 10.1109/TPAMI.2009.167

Rapid object detection using a boosted cascade of simple features
conference, January 2001

  • Viola, P.; Jones, M.
  • Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001
  • DOI: 10.1109/CVPR.2001.990517

Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy
journal, January 2017


Application of STEM characterization for investigating radiation effects in BCC Fe-based alloys
journal, April 2015

  • Parish, Chad M.; Field, Kevin G.; Certain, Alicia G.
  • Journal of Materials Research, Vol. 30, Issue 9
  • DOI: 10.1557/jmr.2015.32

Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys
journal, October 2015


Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
journal, May 2015

  • Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
  • Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
  • DOI: 10.1186/s40679-015-0006-6

    Works referencing / citing this record:

    Mask R-CNN
    conference, October 2017

    • He, Kaiming; Gkioxari, Georgia; Dollar, Piotr
    • 2017 IEEE International Conference on Computer Vision (ICCV)
    • DOI: 10.1109/iccv.2017.322

    Better matching with fewer features: The selection of useful features in large database recognition problems
    conference, September 2009

    • Turcot, Panu; Lowe, David G.
    • 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops
    • DOI: 10.1109/iccvw.2009.5457541

    Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
    journal, May 2015

    • Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
    • Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
    • DOI: 10.1186/s40679-015-0006-6

    Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images
    journal, October 2011


    Watersheds in digital spaces: an efficient algorithm based on immersion simulations
    journal, June 1991

    • Vincent, L.; Soille, P.
    • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, Issue 6
    • DOI: 10.1109/34.87344

    Application of STEM characterization for investigating radiation effects in BCC Fe-based alloys
    journal, April 2015

    • Parish, Chad M.; Field, Kevin G.; Certain, Alicia G.
    • Journal of Materials Research, Vol. 30, Issue 9
    • DOI: 10.1557/jmr.2015.32

    Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa
    journal, November 2017


    Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography
    journal, May 2016

    • Jesse, S.; Chi, M.; Belianinov, A.
    • Scientific Reports, Vol. 6, Issue 1
    • DOI: 10.1038/srep26348

    Deep learning
    journal, May 2015

    • LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
    • Nature, Vol. 521, Issue 7553
    • DOI: 10.1038/nature14539

    Diffraction contrast STEM of dislocations: Imaging and simulations
    journal, August 2011


    TEM characterization of dislocation loops in irradiated bcc Fe-based steels
    journal, March 2013


    Shape matching and object recognition using shape contexts
    journal, April 2002

    • Belongie, S.; Malik, J.; Puzicha, J.
    • IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 4
    • DOI: 10.1109/34.993558

    Image driven machine learning methods for microstructure recognition
    journal, October 2016


    Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy
    journal, January 2017


    Object recognition from local scale-invariant features
    conference, January 1999


    Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures
    journal, July 2017


    Characterization of irradiation defect structures and densities by transmission electron microscopy
    journal, February 2015

    • Kirk, Marquis; Yi, Xiaoou; Jenkins, Michael
    • Journal of Materials Research, Vol. 30, Issue 9
    • DOI: 10.1557/jmr.2015.19

    Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys
    journal, October 2015