Search Menu
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information
Search Scholarly Publications

Title: Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Journal Article · · Scientific Reports
 [1];  [2];  [3]; ORCiD logo [1];  [4]; ORCiD logo [5]
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  2. Western Washington Univ., Bellingham, WA (United States)
  3. Univ. of Connecticut, Storrs, CT (United States)
  4. Western Washington Univ., Bellingham, WA (United States); Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  5. Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Univ. of Connecticut, Storrs, CT (United States)
+ Show Author Affiliations

Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.

Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Fusion Energy Sciences (FES) (SC-24)
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
1559992
Report Number(s):
PNNL-SA--146546
Journal Information:
Scientific Reports, Journal Name: Scientific Reports Journal Issue: 1 Vol. 9; ISSN 2045-2322
Publisher:
Nature Publishing GroupCopyright Statement
Country of Publication:
United States
Language:
English

References (60)

Densely Connected Convolutional Networks preprint January 2016
Phase stability and structural defects in high-temperature Mo–Si–B alloys journal October 2008
Survey on semantic image segmentation techniques conference December 2017
Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation journal January 2016
Advanced Steel Microstructural Classification by Deep Learning Methods journal February 2018
A computer vision approach for automated analysis and classification of microstructural image data journal December 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation
  • Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas
  • Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28
book November 2015
Image driven machine learning methods for microstructure recognition journal October 2016
Practical Recommendations for Gradient-Based Training of Deep Architectures book January 2012
A Method for Detection and Classification of Glass Defects in Low Resolution Images conference August 2011
Towards bend-contour-free dislocation imaging via diffraction contrast STEM journal October 2018
The formation and interpretation of defect images from crystalline materials in a scanning transmission electron microscope journal January 1976
Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics journal December 2017
Analytical Transmission Electron Microscopy datapaper January 2009
Advanced Steel Microstructural Classification by Deep Learning Methods journal February 2018
A review of semantic segmentation using deep neural networks journal November 2017
A comparative study of texture measures with classification based on featured distributions journal January 1996
A Kinematical Theory of Diffraction Contrast of Electron Transmission Microscope Images of Dislocations and other Defects
  • Hirsch, P. B.; Howie, A.; Whelan, M. J.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 252, Issue 1017 https://doi.org/10.1098/rsta.1960.0013
journal May 1960
AxonDeepSeg: automatic axon and myelin segmentation from microscopy data using convolutional neural networks journal February 2018
Phase stability and structural defects in high-temperature Mo–Si–B alloys journal October 2008
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation conference July 2017
Analytical Transmission Electron Microscopy journal August 2005
Diffraction contrast STEM of dislocations: Imaging and simulations journal August 2011
Dislocation Movement Through Random Arrays of Obstacles journal February 1967
A Kinematical Theory of Diffraction Contrast of Electron Transmission Microscope Images of Dislocations and other Defects
  • Hirsch, P. B.; Howie, A.; Whelan, M. J.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 252, Issue 1017 https://doi.org/10.1098/rsta.1960.0013
journal May 1960
High throughput quantitative metallography for complex microstructures using deep learning: A case study in ultrahigh carbon steel text January 2018
A comparative study of texture measures with classification based on featured distributions journal January 1996
A computer vision approach for automated analysis and classification of microstructural image data journal December 2015
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations text January 2018
A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images journal July 2018
Fully Convolutional Networks for Semantic Segmentation preprint January 2016
Automated defect analysis in electron microscopic images journal July 2018
Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics journal December 2017
The relationship between grain boundary structure, defect mobility and grain boundary sink efficiency journal March 2015
Enhancing radiation tolerance by controlling defect mobility and migration pathways in multicomponent single-phase alloys journal December 2016
Advanced Steel Microstructural Classification by Deep Learning Methods text January 2017
Best practices for convolutional neural networks applied to visual document analysis conference January 2003
A systematic error in the determination of dislocation densities in thin films journal September 1961
Dislocation movement through random arrays of obstacles journal October 1966
Image Enhancement via Subimage Histogram Equalization Based on Mean and Variance journal January 2017
Image driven machine learning methods for microstructure recognition journal October 2016
Disconnections and other defects associated with twin interfaces journal October 2016
The relationship between grain boundary structure, defect mobility and grain boundary sink efficiency journal March 2015
Fully convolutional networks for semantic segmentation conference June 2015
Image Enhancement via Subimage Histogram Equalization Based on Mean and Variance journal January 2017
Enhancing radiation tolerance by controlling defect mobility and migration pathways in multicomponent single-phase alloys journal December 2016
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations journal October 2016
Steel defect classification with Max-Pooling Convolutional Neural Networks
  • Masci, Jonathan; Meier, Ueli; Ciresan, Dan
  • 2012 International Joint Conference on Neural Networks (IJCNN 2012 - Brisbane), The 2012 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN.2012.6252468
conference June 2012
Dislocation movement through random arrays of obstacles journal October 1966
High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel journal February 2019
A deep learning approach to identify local structures in atomic-resolution transmission electron microscopy images text January 2018
Diffraction contrast STEM of dislocations: Imaging and simulations journal August 2011
A systematic error in the determination of dislocation densities in thin films journal September 1961
Densely Connected Convolutional Networks conference July 2017
Fully Convolutional Networks for Semantic Segmentation journal April 2017
The formation and interpretation of defect images from crystalline materials in a scanning transmission electron microscope journal January 1976
Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation preprint January 2015
Classification of corrosion defects in NiAl bronze through image analysis journal November 2010
Importance-Aware Semantic Segmentation for Autonomous Vehicles journal January 2019
Control and Use of Defects in Materials journal August 1998

Cited By (1)

Learning-based Defect Recognition for Quasi-Periodic Microscope Images text January 2020

Similar Records

Related Subjects

36 MATERIALS SCIENCE
machine leaning
STEM Imaging
convolutional neural network
semantic segmentation