Automated defect analysis in electron microscopic images
Abstract
Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a rangemore »
- Authors:
-
- Univ. of Wisconsin, Madison, WI (United States)
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Publication Date:
- Research Org.:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Org.:
- USDOE
- OSTI Identifier:
- 1462882
- Grant/Contract Number:
- AC05-00OR22725
- Resource Type:
- Accepted Manuscript
- Journal Name:
- npj Computational Materials
- Additional Journal Information:
- Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2057-3960
- Publisher:
- Nature Publishing Group
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 47 OTHER INSTRUMENTATION
Citation Formats
Li, Wei, Field, Kevin G., and Morgan, Dane. Automated defect analysis in electron microscopic images. United States: N. p., 2018.
Web. doi:10.1038/s41524-018-0093-8.
Li, Wei, Field, Kevin G., & Morgan, Dane. Automated defect analysis in electron microscopic images. United States. https://doi.org/10.1038/s41524-018-0093-8
Li, Wei, Field, Kevin G., and Morgan, Dane. Wed .
"Automated defect analysis in electron microscopic images". United States. https://doi.org/10.1038/s41524-018-0093-8. https://www.osti.gov/servlets/purl/1462882.
@article{osti_1462882,
title = {Automated defect analysis in electron microscopic images},
author = {Li, Wei and Field, Kevin G. and Morgan, Dane},
abstractNote = {Electron microscopy and defect analysis are a cornerstone of materials science, as they offer detailed insights on the microstructure and performance of a wide range of materials and material systems. Building a robust and flexible platform for automated defect recognition and classification in electron microscopy will result in the completion of analysis orders of magnitude faster after images are recorded, or even online during image acquisition. Automated analysis has the potential to be significantly more efficient, accurate, and repeatable than human analysis, and it can scale with the increasingly important methods of automated data generation. Herein, an automated recognition tool is developed based on a computer vison–based approach; it sequentially applies a cascade object detector, convolutional neural network, and local image analysis methods. We demonstrate that the automated tool performs as well as or better than manual human detection in terms of recall and precision and achieves quantitative image/defect analysis metrics close to the human average. The proposed approach works for images of varying contrast, brightness, and magnification. Furthermore, these promising results suggest that this and similar approaches are worth exploring for detecting multiple defect types and have the potential to locate, classify, and measure quantitative features for a range of defect types, materials, and electron microscopic techniques.},
doi = {10.1038/s41524-018-0093-8},
journal = {npj Computational Materials},
number = 1,
volume = 4,
place = {United States},
year = {2018},
month = {7}
}
Web of Science
Works referenced in this record:
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
Face Description with Local Binary Patterns: Application to Face Recognition
journal, December 2006
- Ahonen, T.; Hadid, A.; Pietikainen, M.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 12
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
journal, October 2016
- Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem
- ACS Nano, Vol. 11, Issue 12
A practical implementation of face detection by using Matlab cascade object detector
conference, October 2015
- Alionte, Elena; Lazar, Corneliu
- 2015 19th International Conference on System Theory, Control and Computing (ICSTCC)
Image driven machine learning methods for microstructure recognition
journal, October 2016
- Chowdhury, Aritra; Kautz, Elizabeth; Yener, Bülent
- Computational Materials Science, Vol. 123
Robust Real-Time Face Detection
journal, May 2004
- Viola, Paul; Jones, Michael J.
- International Journal of Computer Vision, Vol. 57, Issue 2
Histograms of Oriented Gradients for Human Detection
conference, January 2005
- Dalal, N.; Triggs, B.
- 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)
Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures
journal, July 2017
- DeCost, Brian L.; Francis, Toby; Holm, Elizabeth A.
- Acta Materialia, Vol. 133
A general review of human face detection including a study of neural networks and Haar feature-based cascade classifier in face detection
conference, August 2014
- Sharifara, Ali; Mohd Rahim, Mohd Shafry; Anisi, Yasaman
- 2014 International Symposium on Biometrics and Security Technologies (ISBAST)
Object recognition from local scale-invariant features
conference, January 1999
- Lowe, D. G.
- Proceedings of the Seventh IEEE International Conference on Computer Vision
Rethinking the Inception Architecture for Computer Vision
conference, June 2016
- Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Characterization of irradiation defect structures and densities by transmission electron microscopy
journal, February 2015
- Kirk, Marquis; Yi, Xiaoou; Jenkins, Michael
- Journal of Materials Research, Vol. 30, Issue 9
Fully convolutional networks for semantic segmentation
conference, June 2015
- Long, Jonathan; Shelhamer, Evan; Darrell, Trevor
- 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa
journal, November 2017
- Field, Kevin G.; Briggs, Samuel A.; Sridharan, Kumar
- Journal of Nuclear Materials, Vol. 495
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography
journal, May 2016
- Jesse, S.; Chi, M.; Belianinov, A.
- Scientific Reports, Vol. 6, Issue 1
Inferring low-dimensional microstructure representations using convolutional neural networks
journal, November 2017
- Lubbers, Nicholas; Lookman, Turab; Barros, Kipton
- Physical Review E, Vol. 96, Issue 5
Watersheds in digital spaces: an efficient algorithm based on immersion simulations
journal, June 1991
- Vincent, L.; Soille, P.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, Issue 6
Structural materials for fission & fusion energy
journal, November 2009
- Zinkle, Steven J.; Busby, Jeremy T.
- Materials Today, Vol. 12, Issue 11
A Method for Detection and Classification of Glass Defects in Low Resolution Images
conference, August 2011
- Zhao, Jie; Kong, Qing-Jie; Zhao, Xu
- Graphics (ICIG), 2011 Sixth International Conference on Image and Graphics
Shape matching and object recognition using shape contexts
journal, April 2002
- Belongie, S.; Malik, J.; Puzicha, J.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, Issue 4
TEM characterization of dislocation loops in irradiated bcc Fe-based steels
journal, March 2013
- Yao, B.; Edwards, D. J.; Kurtz, R. J.
- Journal of Nuclear Materials, Vol. 434, Issue 1-3
Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images
journal, October 2011
- Kreshuk, Anna; Straehle, Christoph N.; Sommer, Christoph
- PLoS ONE, Vol. 6, Issue 10
Diffraction contrast STEM of dislocations: Imaging and simulations
journal, August 2011
- Phillips, P. J.; Brandes, M. C.; Mills, M. J.
- Ultramicroscopy, Vol. 111, Issue 9-10
Face Detection with the Faster R-CNN
conference, May 2017
- Jiang, Huaizu; Learned-Miller, Erik
- 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)
Object Detection with Discriminatively Trained Part-Based Models
journal, September 2010
- Felzenszwalb, P. F.; Girshick, R. B.; McAllester, D.
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 32, Issue 9
Rapid object detection using a boosted cascade of simple features
conference, January 2001
- Viola, P.; Jones, M.
- Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001
Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy
journal, January 2017
- Field, Kevin G.; Briggs, Samuel A.; Hu, Xunxiang
- Journal of Nuclear Materials, Vol. 483
Application of STEM characterization for investigating radiation effects in BCC Fe-based alloys
journal, April 2015
- Parish, Chad M.; Field, Kevin G.; Certain, Alicia G.
- Journal of Materials Research, Vol. 30, Issue 9
Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys
journal, October 2015
- Field, Kevin G.; Hu, Xunxiang; Littrell, Kenneth C.
- Journal of Nuclear Materials, Vol. 465
Mask R-CNN
conference, October 2017
- He, Kaiming; Gkioxari, Georgia; Dollar, Piotr
- 2017 IEEE International Conference on Computer Vision (ICCV)
Big data and deep data in scanning and electron microscopies: deriving functionality from multidimensional data sets
journal, May 2015
- Belianinov, Alex; Vasudevan, Rama; Strelcov, Evgheni
- Advanced Structural and Chemical Imaging, Vol. 1, Issue 1
A deep convolutional neural network to analyze position averaged convergent beam electron diffraction patterns
journal, May 2018
- Xu, W.; LeBeau, J. M.
- Ultramicroscopy, Vol. 188
Better matching with fewer features: The selection of useful features in large database recognition problems
conference, September 2009
- Turcot, Panu; Lowe, David G.
- 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops
Mask R-CNN
journal, February 2020
- He, Kaiming; Gkioxari, Georgia; Dollar, Piotr
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, Issue 2
Fully Convolutional Networks for Semantic Segmentation
journal, April 2017
- Shelhamer, Evan; Long, Jonathan; Darrell, Trevor
- IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, Issue 4
Robust real-time face detection
conference, January 2001
- Viola, P.; Jones, M.
- Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001
Rethinking the Inception Architecture for Computer Vision
preprint, January 2015
- Szegedy, Christian; Vanhoucke, Vincent; Ioffe, Sergey
- arXiv
Fully Convolutional Networks for Semantic Segmentation
preprint, January 2016
- Shelhamer, Evan; Long, Jonathan; Darrell, Trevor
- arXiv
A Deep Convolutional Neural Network to Analyze Position Averaged Convergent Beam Electron Diffraction Patterns
text, January 2017
- Xu, Weizong; LeBeau, James M.
- arXiv
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations
text, January 2018
- Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem
- arXiv
Detection of Open Loop Defects in STEM Images of Irradiation-Damaged Alloys – Source Code for Detection and Image Dataset
dataset, January 2018
- Li, Wei; Field, Kevin G.; Morgan, Dane
- Materials Data Facility
Works referencing / citing this record:
Interplay Between Grain Boundaries and Radiation Damage
journal, March 2019
- Barr, Christopher M.; El-Atwani, Osman; Kaoumi, Djamel
- JOM, Vol. 71, Issue 4
Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels
journal, September 2019
- Roberts, Graham; Haile, Simon Y.; Sainju, Rajat
- Scientific Reports, Vol. 9, Issue 1
Machine-learning-assisted screening of pure-silica zeolites for effective removal of linear siloxanes and derivatives
journal, January 2020
- Lin, Shiru; Wang, Yekun; Zhao, Yinghe
- Journal of Materials Chemistry A, Vol. 8, Issue 6
Data-driven approach for synchrotron X-ray Laue microdiffraction scan analysis
journal, October 2019
- Song, Yintao; Tamura, Nobumichi; Zhang, Chenbo
- Acta Crystallographica Section A Foundations and Advances, Vol. 75, Issue 6