skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Abstract

Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statisticallymore » meaningful quantification of materials defects.« less

Authors:
 [1];  [2];  [3]; ORCiD logo [1];  [4]; ORCiD logo [5]
+ Show Author Affiliations
  1. Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  2. Western Washington Univ., Bellingham, WA (United States)
  3. Univ. of Connecticut, Storrs, CT (United States)
  4. Western Washington Univ., Bellingham, WA (United States); Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
  5. Pacific Northwest National Lab. (PNNL), Richland, WA (United States); Univ. of Connecticut, Storrs, CT (United States)
Publication Date:
Research Org.:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Fusion Energy Sciences (FES) (SC-24)
OSTI Identifier:
1559992
Report Number(s):
PNNL-SA-146546
Journal ID: ISSN 2045-2322
Grant/Contract Number:  
AC05-76RL01830
Resource Type:
Accepted Manuscript
Journal Name:
Scientific Reports
Additional Journal Information:
Journal Volume: 9; Journal Issue: 1; Journal ID: ISSN 2045-2322
Publisher:
Nature Publishing Group
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE; machine leaning; STEM Imaging; convolutional neural network; semantic segmentation

Citation Formats

Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., and Zhu, Yuanyuan. Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels. United States: N. p., 2019. Web. doi:10.1038/s41598-019-49105-0.
Copy to clipboard
Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., & Zhu, Yuanyuan. Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels. United States. doi:10.1038/s41598-019-49105-0.
Copy to clipboard
Roberts, Graham, Haile, Simon, Sainju, Rajat, Edwards, Danny J., Hutchinson, Brian J., and Zhu, Yuanyuan. Wed . "Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels". United States. doi:10.1038/s41598-019-49105-0. https://www.osti.gov/servlets/purl/1559992.
Copy to clipboard
@article{osti_1559992,
title = {Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels},
author = {Roberts, Graham and Haile, Simon and Sainju, Rajat and Edwards, Danny J. and Hutchinson, Brian J. and Zhu, Yuanyuan},
abstractNote = {Crystalline materials exhibit long-range ordered lattice unit, within which resides nonperiodic structural features called defects. These crystallographic defects play a vital role in determining the physical and mechanical properties of a wide range of material systems. While computer vision has demonstrated success in recognizing feature patterns in images with well defined contrast, automated identification of nanometer scale crystallographic defects in electron micrographs governed by complex contrast mechanism is still a challenging task. Here, building upon an advanced defect imaging mode that offers high feature clarity, we introduce DefectSegNet - a new convolutional neural network (CNN) architecture that performs semantic segmentation of three common crystallographic defects in structural alloys: dislocation lines, precipitates and voids. Results from supervised training of a small set of high-quality defect images of steels show high pixel-wise accuracy across all three types of defects: 91.60 ± 1.77% on dislocations, 93.39 ± 1.00% on precipitates, and 98.85 ± 0.56% on voids. We discuss the sources of uncertainties in CNN prediction, and the training data in terms of feature density, representation and homogeneity and their effects on deep learning performance. Further defect quantification using DefectSegNet prediction outperforms human expert average, presenting a promising new workflow for fast and statistically meaningful quantification of materials defects.},
doi = {10.1038/s41598-019-49105-0},
journal = {Scientific Reports},
number = 1,
volume = 9,
place = {United States},
year = {2019},
month = {9}
}
Copy to clipboard
Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI:  10.1038/s41598-019-49105-0
Copyright Statement
Other availability
Search WorldCat Search WorldCat to find libraries that may hold this journal
Save / Share:
Export Metadata
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.

Works referenced in this record:

Fully convolutional networks for semantic segmentation
conference, June 2015

  • Long, Jonathan; Shelhamer, Evan; Darrell, Trevor
  • 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2015.7298965

Densely Connected Convolutional Networks
conference, July 2017

  • Huang, Gao; Liu, Zhuang; Maaten, Laurens van der
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • DOI: 10.1109/CVPR.2017.243

Classification of corrosion defects in NiAl bronze through image analysis
journal, November 2010

Enhancing radiation tolerance by controlling defect mobility and migration pathways in multicomponent single-phase alloys
journal, December 2016

  • Lu, Chenyang; Niu, Liangliang; Chen, Nanjun
  • Nature Communications, Vol. 7, Issue 1
  • DOI: 10.1038/ncomms13564

Advanced Steel Microstructural Classification by Deep Learning Methods
journal, February 2018

The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation
conference, July 2017

  • Jegou, Simon; Drozdzal, Michal; Vazquez, David
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • DOI: 10.1109/CVPRW.2017.156

A Method for Detection and Classification of Glass Defects in Low Resolution Images
conference, August 2011

  • Zhao, Jie; Kong, Qing-Jie; Zhao, Xu
  • Graphics (ICIG), 2011 Sixth International Conference on Image and Graphics
  • DOI: 10.1109/ICIG.2011.187

A review of semantic segmentation using deep neural networks
journal, November 2017

  • Guo, Yanming; Liu, Yu; Georgiou, Theodoros
  • International Journal of Multimedia Information Retrieval, Vol. 7, Issue 2
  • DOI: 10.1007/s13735-017-0141-z

High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel
journal, February 2019

Image driven machine learning methods for microstructure recognition
journal, October 2016

Steel defect classification with Max-Pooling Convolutional Neural Networks
conference, June 2012

  • Masci, Jonathan; Meier, Ueli; Ciresan, Dan
  • 2012 International Joint Conference on Neural Networks (IJCNN 2012 - Brisbane), The 2012 International Joint Conference on Neural Networks (IJCNN)
  • DOI: 10.1109/IJCNN.2012.6252468

A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images
journal, July 2018

  • Madsen, Jacob; Liu, Pei; Kling, Jens
  • Advanced Theory and Simulations, Vol. 1, Issue 8
  • DOI: 10.1002/adts.201800037

Beyond pixels: A comprehensive survey from bottom-up to semantic image segmentation and cosegmentation
journal, January 2016

  • Zhu, Hongyuan; Meng, Fanman; Cai, Jianfei
  • Journal of Visual Communication and Image Representation, Vol. 34
  • DOI: 10.1016/j.jvcir.2015.10.012

The relationship between grain boundary structure, defect mobility and grain boundary sink efficiency
journal, March 2015

  • Uberuaga, Blas Pedro; Vernon, Louis J.; Martinez, Enrique
  • Scientific Reports, Vol. 5, Issue 1
  • DOI: 10.1038/srep09095

Diffraction contrast STEM of dislocations: Imaging and simulations
journal, August 2011

Dislocation movement through random arrays of obstacles
journal, October 1966

Control and Use of Defects in Materials
journal, August 1998

AxonDeepSeg: automatic axon and myelin segmentation from microscopy data using convolutional neural networks
journal, February 2018

A systematic error in the determination of dislocation densities in thin films
journal, September 1961

Microstructure recognition using convolutional neural networks for prediction of ionic conductivity in ceramics
journal, December 2017

A Kinematical Theory of Diffraction Contrast of Electron Transmission Microscope Images of Dislocations and other Defects
journal, May 1960

  • Hirsch, P. B.; Howie, A.; Whelan, M. J.
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 252, Issue 1017
  • DOI: 10.1098/rsta.1960.0013

Phase stability and structural defects in high-temperature Mo–Si–B alloys
journal, October 2008

A comparative study of texture measures with classification based on featured distributions
journal, January 1996

Automated defect analysis in electron microscopic images
journal, July 2018

Towards bend-contour-free dislocation imaging via diffraction contrast STEM
journal, October 2018

Best practices for convolutional neural networks applied to visual document analysis
conference, January 2003

  • Simard, P. Y.; Steinkraus, D.; Platt, J. C.
  • Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings.
  • DOI: 10.1109/ICDAR.2003.1227801

Image Enhancement via Subimage Histogram Equalization Based on Mean and Variance
journal, January 2017

  • Zhuang, Liyun; Guan, Yepeng
  • Computational Intelligence and Neuroscience, Vol. 2017
  • DOI: 10.1155/2017/6029892

Disconnections and other defects associated with twin interfaces
journal, October 2016

Survey on semantic image segmentation techniques
conference, December 2017

  • Sevak, Jay S.; Kapadia, Aerika D.; Chavda, Jaiminkumar B.
  • 2017 International Conference on Intelligent Sustainable Systems (ICISS)
  • DOI: 10.1109/ISS1.2017.8389420

Importance-Aware Semantic Segmentation for Autonomous Vehicles
journal, January 2019

  • Chen, Bike; Gong, Chen; Yang, Jian
  • IEEE Transactions on Intelligent Transportation Systems, Vol. 20, Issue 1
  • DOI: 10.1109/TITS.2018.2801309

Analytical Transmission Electron Microscopy
journal, August 2005

A computer vision approach for automated analysis and classification of microstructural image data
journal, December 2015

    [ × clear filter / sort ]

    Similar Records in DOE PAGES and OSTI.GOV collections: