Performance and limitations of deep learning semantic segmentation of multiple defects in transmission electron micrographs

Jacobs, Ryan; Shen, Mingren; Liu, Yuhan; Hao, Wei; Li, Xiaoshan; He, Ruoyu; Greaves, Jacob R. C.; Wang, Donglin; Xie, Zeming; Huang, Zitong; Wang, Chao; Field, Kevin G.; Morgan, Dane

doi:10.1016/j.xcrp.2022.100876

Performance and limitations of deep learning semantic segmentation of multiple defects in transmission electron micrographs

Journal Article · 01 May 2022 · Cell Reports Physical Science

DOI:https://doi.org/10.1016/j.xcrp.2022.100876· OSTI ID:1981794

; Shen, Mingren; Liu, Yuhan; Hao, Wei; Li, Xiaoshan; He, Ruoyu; Greaves, Jacob R. C.; Wang, Donglin; Xie, Zeming; Huang, Zitong; Wang, Chao; Field, Kevin G.; Morgan, Dane

Not provided.

Research Organization:: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). High Flux Isotope Reactor (HFIR)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC05-00OR22725

OSTI ID:: 1981794

Journal Information:: Cell Reports Physical Science, Vol. 3, Issue 5; ISSN 2666-3864

Publisher:: Elsevier

Country of Publication:: United States

Language:: English

References (40)

Work-hardening and work-softening of face-centred cubic metal crystals Seeger, A.; Diehl, J.; Mader, S. Philosophical Magazine, Vol. 2, Issue 15 https://doi.org/10.1080/14786435708243823	journal	March 1957
NIH Image to ImageJ: 25 years of image analysis Schneider, Caroline A.; Rasband, Wayne S.; Eliceiri, Kevin W. Nature Methods, Vol. 9, Issue 7 https://doi.org/10.1038/nmeth.2089	journal	June 2012
Big Data Analytics for Scanning Transmission Electron Microscopy Ptychography Jesse, S.; Chi, M.; Belianinov, A. Scientific Reports, Vol. 6, Issue 1 https://doi.org/10.1038/srep26348	journal	May 2016
The Pascal Visual Object Classes Challenge: A Retrospective Everingham, Mark; Eslami, S. M. Ali; Van Gool, Luc International Journal of Computer Vision, Vol. 111, Issue 1 https://doi.org/10.1007/s11263-014-0733-5	journal	June 2014
ImageNet Large Scale Visual Recognition Challenge Russakovsky, Olga; Deng, Jia; Su, Hao International Journal of Computer Vision, Vol. 115, Issue 3 https://doi.org/10.1007/s11263-015-0816-y	journal	April 2015
Deep Learning for Generic Object Detection: A Survey Liu, Li; Ouyang, Wanli; Wang, Xiaogang International Journal of Computer Vision, Vol. 128, Issue 2 https://doi.org/10.1007/s11263-019-01247-4	journal	October 2019
Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures DeCost, Brian L.; Francis, Toby; Holm, Elizabeth A. Acta Materialia, Vol. 133 https://doi.org/10.1016/j.actamat.2017.05.014	journal	July 2017
Automatic segmentation of inorganic nanoparticles in BF TEM micrographs Groom, D. J.; Yu, K.; Rasouli, S. Ultramicroscopy, Vol. 194 https://doi.org/10.1016/j.ultramic.2018.06.002	journal	November 2018
A computer vision approach for automated analysis and classification of microstructural image data DeCost, Brian L.; Holm, Elizabeth A. Computational Materials Science, Vol. 110 https://doi.org/10.1016/j.commatsci.2015.08.011	journal	December 2015
Deep learning for computational chemistry Goh, Garrett B.; Hodas, Nathan O.; Vishnu, Abhinav Journal of Computational Chemistry, Vol. 38, Issue 16 https://doi.org/10.1002/jcc.24764	journal	March 2017
Perspectives on the Impact of Machine Learning, Deep Learning, and Artificial Intelligence on Materials, Processes, and Structures Engineering Dimiduk, Dennis M.; Holm, Elizabeth A.; Niezgoda, Stephen R. Integrating Materials and Manufacturing Innovation, Vol. 7, Issue 3 https://doi.org/10.1007/s40192-018-0117-8	journal	August 2018
A review of deep learning in the study of materials degradation Nash, Will; Drummond, Tom; Birbilis, Nick npj Materials Degradation, Vol. 2, Issue 1 https://doi.org/10.1038/s41529-018-0058-x	journal	November 2018
Deep materials informatics: Applications of deep learning in materials science Agrawal, Ankit; Choudhary, Alok MRS Communications, Vol. 9, Issue 3 https://doi.org/10.1557/mrc.2019.73	journal	June 2019
Opportunities and Challenges for Machine Learning in Materials Science Morgan, Dane; Jacobs, Ryan Annual Review of Materials Research, Vol. 50, Issue 1 https://doi.org/10.1146/annurev-matsci-070218-010015	journal	July 2020
Deep learning analysis on microscopic imaging in materials science Ge, M.; Su, F.; Zhao, Z. Materials Today Nano, Vol. 11 https://doi.org/10.1016/j.mtnano.2020.100087	journal	August 2020
Automating material image analysis for material discovery Park, Chiwoo; Ding, Yu MRS Communications, Vol. 9, Issue 02 https://doi.org/10.1557/mrc.2019.48	journal	April 2019
Learning-based defect recognition for quasi-periodic HRSTEM images Dennler, Nik; Foncubierta-Rodriguez, Antonio; Neupert, Titus Micron, Vol. 146 https://doi.org/10.1016/j.micron.2021.103069	journal	July 2021
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations Ziatdinov, Maxim; Dyck, Ondrej; Maksov, Artem ACS Nano, Vol. 11, Issue 12 https://doi.org/10.1021/acsnano.7b07504	journal	October 2016
Automatic detection, localization and segmentation of nano-particles with deep learning in microscopy images Oktay, Ayse Betul; Gurses, Anıl Micron, Vol. 120 https://doi.org/10.1016/j.micron.2019.02.009	journal	May 2019
Nanoparticle Recognition on Scanning Probe Microscopy Images Using Computer Vision and Deep Learning Okunev, Alexey G.; Mashukov, Mikhail Yu.; Nartova, Anna V. Nanomaterials, Vol. 10, Issue 7 https://doi.org/10.3390/nano10071285	journal	June 2020
A deep-learning-based approach for fast and robust steel surface defects classification Fu, Guizhong; Sun, Peize; Zhu, Wenbin Optics and Lasers in Engineering, Vol. 121 https://doi.org/10.1016/j.optlaseng.2019.05.005	journal	October 2019
Periodic Surface Defect Detection in Steel Plates Based on Deep Learning Liu, Yang; Xu, Ke; Xu, Jinwu Applied Sciences, Vol. 9, Issue 15 https://doi.org/10.3390/app9153127	journal	August 2019
Automated defect analysis in electron microscopic images Li, Wei; Field, Kevin G.; Morgan, Dane npj Computational Materials, Vol. 4, Issue 1 https://doi.org/10.1038/s41524-018-0093-8	journal	July 2018
Multi defect detection and analysis of electron microscopy images with deep learning Shen, Mingren; Li, Guanzhao; Wu, Dongxia Computational Materials Science, Vol. 199 https://doi.org/10.1016/j.commatsci.2021.110576	journal	November 2021
Automated Detection of Helium Bubbles in Irradiated X-750 Anderson, Chris M.; Klein, Jacob; Rajakumar, Heygaan Ultramicroscopy, Vol. 217 https://doi.org/10.1016/j.ultramic.2020.113068	journal	October 2020
A deep learning based automatic defect analysis framework for In-situ TEM ion irradiations Shen, Mingren; Li, Guanzhao; Wu, Dongxia Computational Materials Science, Vol. 197 https://doi.org/10.1016/j.commatsci.2021.110560	journal	September 2021
Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels Roberts, Graham; Haile, Simon Y.; Sainju, Rajat Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-49105-0	journal	September 2019
Radiation tolerance of neutron-irradiated model Fe–Cr–Al alloys Field, Kevin G.; Hu, Xunxiang; Littrell, Kenneth C. Journal of Nuclear Materials, Vol. 465 https://doi.org/10.1016/j.jnucmat.2015.06.023	journal	October 2015
Can machine learning identify the next high-temperature superconductor? Examining extrapolation performance for materials discovery Meredig, Bryce; Antono, Erin; Church, Carena Molecular Systems Design & Engineering, Vol. 3, Issue 5 https://doi.org/10.1039/C8ME00012C	journal	January 2018
Error assessment and optimal cross-validation approaches in machine learning applied to impurity diffusion Lu, Hai-Jin; Zou, Nan; Jacobs, Ryan Computational Materials Science, Vol. 169 https://doi.org/10.1016/j.commatsci.2019.06.010	journal	November 2019
A machine learning approach for engineering bulk metallic glass alloys Ward, Logan; O'Keeffe, Stephanie C.; Stevick, Joseph Acta Materialia, Vol. 159 https://doi.org/10.1016/j.actamat.2018.08.002	journal	October 2018
Influence of welding and neutron irradiation on dislocation loop formation and α′ precipitation in a FeCrAl alloy Zhang, Dalong; Briggs, Samuel A.; Edmondson, Philip D. Journal of Nuclear Materials, Vol. 527 https://doi.org/10.1016/j.jnucmat.2019.151784	journal	December 2019
Rapid and flexible segmentation of electron microscopy data using few-shot machine learning Akers, Sarah; Kautz, Elizabeth; Trevino-Gavito, Andrea npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00652-z	journal	November 2021
Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships Ma, W.; Kautz, E. J.; Baskaran, A. Journal of Applied Physics, Vol. 128, Issue 13 https://doi.org/10.1063/5.0013720	journal	October 2020
Microstructure Generation via Generative Adversarial Network for Heterogeneous, Topologically Complex 3D Materials Hsu, Tim; Epting, William K.; Kim, Hokon JOM, Vol. 73, Issue 1 https://doi.org/10.1007/s11837-020-04484-y	journal	December 2020
The Materials Data Facility: Data Services to Advance Materials Science Research Blaiszik, B.; Chard, K.; Pruyne, J. JOM, Vol. 68, Issue 8 https://doi.org/10.1007/s11837-016-2001-3	journal	July 2016
Democratising deep learning for microscopy with ZeroCostDL4Mic von Chamier, Lucas; Laine, Romain F.; Jukkala, Johanna Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-22518-0	journal	April 2021
Heterogeneous dislocation loop formation near grain boundaries in a neutron-irradiated commercial FeCrAl alloy Field, Kevin G.; Briggs, Samuel A.; Hu, Xunxiang Journal of Nuclear Materials, Vol. 483 https://doi.org/10.1016/j.jnucmat.2016.10.050	journal	January 2017
Dislocation loop formation in model FeCrAl alloys after neutron irradiation below 1 dpa Field, Kevin G.; Briggs, Samuel A.; Sridharan, Kumar Journal of Nuclear Materials, Vol. 495 https://doi.org/10.1016/j.jnucmat.2017.07.061	journal	November 2017
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Ren, Shaoqing; He, Kaiming; Girshick, Ross IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, Issue 6 https://doi.org/10.1109/TPAMI.2016.2577031	journal	June 2017

Similar Records

Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels

Journal Article · 2019 · Scientific Reports · OSTI ID:1559992

Roberts, Graham; Haile, Simon; Sainju, Rajat; +3 more

A deep learning approach for semantic segmentation of unbalanced data in electron tomography of catalytic materials

Journal Article · 2022 · Scientific Reports · OSTI ID:1895125

Genc, Arda; Kovarik, Libor; Fraser, Hamish L.

ADVANCED-STEM-BASED DEEP LEARNING FOR SEMANTIC SEGMENTATION OF DEFECTS IN STEELS

Book · 2019 · OSTI ID:1569022

Zhu, Yuanyuan; Roberts, Graham W.; Sainju, Rajat; +5 more

Related Subjects

Chemistry
Energy & Fuels
Materials Science
Physics

Performance and limitations of deep learning semantic segmentation of multiple defects in transmission electron micrographs

Citation Formats

References (40)

Similar Records

Related Subjects