Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Torbati-Sarraf, Hamidreza; Niverty, Sridhar; Singh, Rajhans; Barboza, Daniel; De Andrade, Vincent; Turaga, Pavan; Chawla, Nikhilesh

doi:10.1007/s11837-021-04706-x

Title: Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Journal Article · Tue May 11 00:00:00 EDT 2021 · JOM. Journal of the Minerals, Metals & Materials Society

DOI:https://doi.org/10.1007/s11837-021-04706-x· OSTI ID:1834598

Torbati-Sarraf, Hamidreza ^[1]; Niverty, Sridhar ^[1]; Singh, Rajhans ^[2]; Barboza, Daniel ^[2]; De Andrade, Vincent ^[3]; Turaga, Pavan ^[2]; Chawla, Nikhilesh ^[1]

Purdue Univ., West Lafayette, IN (United States)
Arizona State Univ., Tempe, AZ (United States)
Argonne National Lab. (ANL), Argonne, IL (United States). Advanced Photon Source (APS)

Four state-of-the-art Deep Learning-based Convolutional Neural Networks (DCNN) were applied to automate the semantic segmentation of a 3D Transmission x-ray Microscopy (TXM) nanotomography image data. The standard U-Net architecture as baseline along with UNet++, PSPNet, and DeepLab v3+ networks were trained to segment the microstructural features of an AA7075 micropillar. A workflow was established to evaluate and compare the DCNN prediction dataset with the manually segmented features using the Intersection of Union (IoU) scores, time of training, confusion matrix, and visual assessment. Comparing all model segmentation accuracy metrics, it was found that using pre-trained models as a backbone along with appropriate training encoder-decoder architecture of the Unet++ can robustly handle large volumes of x-ray radiographic images in a reasonable amount of time. This opens a new window for handling accurate and efficient image segmentation of in situ time-dependent 4D x-ray microscopy experimental datasets.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: US Department of the Navy, Office of Naval Research (ONR); USDOE Office of Science (SC)

Grant/Contract Number:: AC02-06CH11357; N00014-10-1-0350

OSTI ID:: 1834598

Journal Information:: JOM. Journal of the Minerals, Metals & Materials Society, Vol. 73, Issue 7; ISSN 1047-4838

Publisher:: SpringerCopyright Statement

Country of Publication:: United States

Language:: English

References (45)

Nanoscale Three-Dimensional Microstructural Characterization of an Sn-Rich Solder Alloy Using High-Resolution Transmission X-Ray Microscopy (TXM) Kaira, Chandrashekara S.; Mayer, Carl R.; De Andrade, V. Microscopy and Microanalysis, Vol. 22, Issue 4 https://doi.org/10.1017/S1431927616011429	journal	July 2016
Advanced Deep Learning‐Based 3D Microstructural Characterization of Multiphase Metal Matrix Composites Evsevleev, Sergei; Paciornik, Sidnei; Bruno, Giovanni Advanced Engineering Materials, Vol. 22, Issue 4 https://doi.org/10.1002/adem.201901197	journal	February 2020
U-Net: Convolutional Networks for Biomedical Image Segmentation Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28	book	November 2015
Formation of intermetallic δ phase in Al-10Si-0.3Fe alloy investigated by in-situ 4D X-ray synchrotron tomography Yu, J. M.; Wanderka, N.; Rack, A. Acta Materialia, Vol. 129 https://doi.org/10.1016/j.actamat.2017.02.048	journal	May 2017
Real-time quantitative imaging of failure events in materials under load at temperatures above 1,600 °C Bale, Hrishikesh A.; Haboub, Abdel; MacDowell, Alastair A. Nature Materials, Vol. 12, Issue 1 https://doi.org/10.1038/nmat3497	journal	December 2012
TomoPy: a framework for the analysis of synchrotron tomographic data Gürsoy, Dogˇa; De Carlo, Francesco; Xiao, Xianghui Journal of Synchrotron Radiation, Vol. 21, Issue 5 https://doi.org/10.1107/S1600577514013939	journal	August 2014
Learning From Scanning Transmission Electron Microscopy to Enhance Transmission X-ray Microscopy: How We Can Merge STEM and TXM Datasets? Yang, X.; Gürsoy, D.; Phatak, C. Microscopy and Microanalysis, Vol. 22, Issue S3 https://doi.org/10.1017/S1431927616002051	journal	July 2016
Formation of a Three-Phase Spiral Structure Due to Competitive Growth of a Peritectic Phase with a Metastable Eutectic Wang, Yeqing; Gao, Jianrong; Ren, Yang JOM, Vol. 72, Issue 8 https://doi.org/10.1007/s11837-020-04237-x	journal	June 2020
Following the phase transitions of iron in 3D with X-ray tomography and diffraction under extreme conditions Boulard, Eglantine; Denoual, Christophe; Dewaele, Agnès Acta Materialia, Vol. 192 https://doi.org/10.1016/j.actamat.2020.04.030	journal	June 2020
A method for zinger artifact reduction in high-energy x-ray computed tomography Mertens, J. C. E.; Williams, J. J.; Chawla, Nikhilesh Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, Vol. 800 https://doi.org/10.1016/j.nima.2015.08.012	journal	November 2015
Defect detection in textured materials using optimized filters Kumar, A.; Pang, G. K. H. IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics), Vol. 32, Issue 5 https://doi.org/10.1109/TSMCB.2002.1033176	journal	October 2002
In situ 3D quantification of the evolution of creep cavity size, shape, and spatial orientation using synchrotron X-ray tomography Isaac, A.; Sket, F.; Reimers, W. Materials Science and Engineering: A, Vol. 478, Issue 1-2 https://doi.org/10.1016/j.msea.2007.05.108	journal	April 2008
Understanding Nanoscale 4D Microstructural Evolution in Aluminum Alloys using Transmission X-Ray Microscopy (TXM) Kaira, C. Shashank; De Andrade, V.; Singh, S. S. Microscopy and Microanalysis, Vol. 23, Issue S1 https://doi.org/10.1017/S143192761701176X	journal	July 2017
Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning Stan, Tiberiu; Thompson, Zachary T.; Voorhees, Peter W. Materials Characterization, Vol. 160 https://doi.org/10.1016/j.matchar.2020.110119	journal	February 2020
Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning Shashank Kaira, C.; Yang, Xiaogang; De Andrade, Vincent Materials Characterization, Vol. 142 https://doi.org/10.1016/j.matchar.2018.05.053	journal	August 2018
Multiscale investigation of corrosion damage initiation and propagation in AA7075-T651 alloy using correlative microscopy Niverty, Sridhar; Kale, Chaitanya; Solanki, Kiran N. Corrosion Science, Vol. 185 https://doi.org/10.1016/j.corsci.2021.109429	journal	June 2021
Pitting corrosion of naturally aged AA 7075 aluminum alloys with bimodal grain size Tian, Wenming; Li, Songmei; Wang, Bo Corrosion Science, Vol. 113 https://doi.org/10.1016/j.corsci.2016.09.013	journal	December 2016
In situ and real-time 3-D microtomography investigation of dendritic solidification in an Al–10wt.% Cu alloy Limodin, N.; Salvo, L.; Boller, E. Acta Materialia, Vol. 57, Issue 7 https://doi.org/10.1016/j.actamat.2009.01.035	journal	April 2009
UNet++: A Nested U-Net Architecture for Medical Image Segmentation Zhou, Zongwei; Rahman Siddiquee, Md Mahfuzur; Tajbakhsh, Nima Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support https://doi.org/10.1007/978-3-030-00889-5_1	book	January 2018
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs Chen, Liang-Chieh; Papandreou, George; Kokkinos, Iasonas IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 40, Issue 4 https://doi.org/10.1109/TPAMI.2017.2699184	journal	April 2018
Microstructural evolution and deformation behavior of Al-Cu alloys: A Transmission X-ray Microscopy (TXM) and micropillar compression study Kaira, C. Shashank; Kantzos, Christopher; Williams, Jason J. Acta Materialia, Vol. 144 https://doi.org/10.1016/j.actamat.2017.11.009	journal	February 2018
Microstructure and micropore formation in a centrifugally-cast duplex stainless steel via X-ray microtomography Zhang, Qingdong; Niverty, Sridhar; Singaravelu, Arun Sundar S. Materials Characterization, Vol. 148 https://doi.org/10.1016/j.matchar.2018.12.009	journal	February 2019
An integrated ship segmentation method based on discriminator and extractor Zhang, Wen; He, Xujie; Li, Wanyi Image and Vision Computing, Vol. 93 https://doi.org/10.1016/j.imavis.2019.11.002	journal	January 2020
Probing Novel Microstructural Evolution Mechanisms in Aluminum Alloys Using 4D Nanoscale Characterization Kaira, C. Shashank; De Andrade, V.; Singh, Sudhanshu S. Advanced Materials, Vol. 29, Issue 41 https://doi.org/10.1002/adma.201703482	journal	September 2017
3D Time-Resolved Observations of Fatigue Crack Initiation and Growth from Corrosion Pits in Al 7XXX Alloys Using In Situ Synchrotron X-ray Tomography Singaravelu, Arun Sundar Sundaram; Williams, Jason J.; Goyal, Harsh Dev Metallurgical and Materials Transactions A, Vol. 51, Issue 1 https://doi.org/10.1007/s11661-019-05519-z	journal	November 2019
In Situ X-ray Microtomography of Stress Corrosion Cracking and Corrosion Fatigue in Aluminum Alloys Singh, Sudhanshu S.; Stannard, Tyler J.; Xiao, Xianghui JOM, Vol. 69, Issue 8 https://doi.org/10.1007/s11837-017-2413-8	journal	June 2017
Elemental and Chemical Mapping of High Capacity Intermetallic Li-ion Anodes with Transmission X-ray Microscopy Ausderau, Logan J.; Gonzalez Malabet, Hernando J.; Buckley, Joseph R. JOM, Vol. 69, Issue 9 https://doi.org/10.1007/s11837-017-2416-5	journal	June 2017
Application of machine learning techniques in mineral phase segmentation for X-ray microcomputed tomography (µCT) data Guntoro, Pratama Istiadi; Tiu, Glacialle; Ghorbani, Yousef Minerals Engineering, Vol. 142 https://doi.org/10.1016/j.mineng.2019.105882	journal	October 2019
Recent progress in semantic image segmentation Liu, Xiaolong; Deng, Zhidong; Yang, Yuhan Artificial Intelligence Review, Vol. 52, Issue 2 https://doi.org/10.1007/s10462-018-9641-3	journal	June 2018
Four dimensional (4D) microstructural evolution of Cu6Sn5 intermetallic and voids under electromigration in bi-crystal pure Sn solder joints Kelly, Marion Branch; Niverty, Sridhar; Chawla, Nikhilesh Acta Materialia, Vol. 189 https://doi.org/10.1016/j.actamat.2020.02.052	journal	May 2020
Microstructural design for damage tolerance in high strength steels Samei, Javad; Pelligra, Concetta; Amirmaleki, M. Materials Letters, Vol. 269 https://doi.org/10.1016/j.matlet.2020.127664	journal	June 2020
Effect of temper on the distribution of pits in AA7075 alloys Dey, Swapna; Gunjan, Manoj K.; Chattoraj, Indranil Corrosion Science, Vol. 50, Issue 10 https://doi.org/10.1016/j.corsci.2008.07.015	journal	October 2008
Direct observation of the displacement field and microcracking in a glass by means of X-ray tomography during in situ Vickers indentation experiment Lacondemine, Tanguy; Réthoré, Julien; Maire, Éric Acta Materialia, Vol. 179 https://doi.org/10.1016/j.actamat.2019.07.053	journal	October 2019
Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions Chen, Dali; Guo, Dinghao; Liu, Shixin Symmetry, Vol. 12, Issue 4 https://doi.org/10.3390/sym12040639	journal	April 2020
Damage investigation and modeling of 3D woven ceramic matrix composites from X-ray tomography in-situ tensile tests Mazars, Vincent; Caty, Olivier; Couégnat, Guillaume Acta Materialia, Vol. 140 https://doi.org/10.1016/j.actamat.2017.08.034	journal	November 2017
Synchrotron CT imaging of lattice structures with engineered defects Patterson, Brian M.; Kuettner, Lindsey; Shear, Trevor Journal of Materials Science, Vol. 55, Issue 25 https://doi.org/10.1007/s10853-020-04840-y	journal	May 2020
Mini Review: Deep Learning for Atrial Segmentation From Late Gadolinium-Enhanced MRIs Jamart, Kevin; Xiong, Zhaohan; Maso Talou, Gonzalo D. Frontiers in Cardiovascular Medicine, Vol. 7 https://doi.org/10.3389/fcvm.2020.00086	journal	May 2020
An analytical electron microscopy study of constituent particles in commercial 7075-T6 and 2024-T3 alloys Gao, Ming; Feng, C. R.; Wei, Robert P. Metallurgical and Materials Transactions A, Vol. 29, Issue 4 https://doi.org/10.1007/s11661-998-0240-9	journal	April 1998
Upscaling ice crystal growth dynamics in snow: Rigorous modeling and comparison to 4D X-ray tomography data Krol, Quirine; Löwe, Henning Acta Materialia, Vol. 151 https://doi.org/10.1016/j.actamat.2018.03.010	journal	June 2018
Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images Ma, Boyuan; Ban, Xiaojuan; Huang, Haiyou Symmetry, Vol. 10, Issue 4 https://doi.org/10.3390/sym10040107	journal	April 2018
Electromigration in Bi-crystal pure Sn solder joints: Elucidating the role of grain orientation Kelly, Marion Branch; Niverty, Sridhar; Chawla, Nikhilesh Journal of Alloys and Compounds, Vol. 818 https://doi.org/10.1016/j.jallcom.2019.152918	journal	March 2020
Low-dose x-ray tomography through a deep convolutional neural network Yang, Xiaogang; De Andrade, Vincent; Scullin, William Scientific Reports, Vol. 8, Issue 1 https://doi.org/10.1038/s41598-018-19426-7	journal	February 2018
3D microstructural characterization and mechanical properties of constituent particles in Al 7075 alloys using X-ray synchrotron tomography and nanoindentation Singh, Sudhanshu S.; Schwartzstein, Cary; Williams, Jason J. Journal of Alloys and Compounds, Vol. 602 https://doi.org/10.1016/j.jallcom.2014.03.010	journal	July 2014
Damage evolution in SiC particle reinforced Al alloy matrix composites by X-ray synchrotron tomography Williams, J. J.; Flom, Z.; Amell, A. A. Acta Materialia, Vol. 58, Issue 18 https://doi.org/10.1016/j.actamat.2010.07.039	journal	October 2010
A convolutional neural network approach to calibrating the rotation axis for X-ray computed tomography Yang, Xiaogang; De Carlo, Francesco; Phatak, Charudatta Journal of Synchrotron Radiation, Vol. 24, Issue 2 https://doi.org/10.1107/S1600577516020117	journal	January 2017

Similar Records

Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning

Journal Article · Wed Aug 15 00:00:00 EDT 2018 · Materials Characterization · OSTI ID:1834598

Shashank Kaira, C.; Yang, Xiaogang; De Andrade, Vincent; +4 more

Automated correlative segmentation of large Transmission X-ray Microscopy (TXM) tomograms using deep learning

Journal Article · Mon May 28 00:00:00 EDT 2018 · Materials Characterization · OSTI ID:1834598

Shashank Kaira, C.; Yang, Xiaogang; De Andrade, Vincent; +4 more

Deep-learning-based workflow for boundary and small target segmentation in digital rock images using UNet++ and IK-EBM

Journal Article · Wed May 04 00:00:00 EDT 2022 · Journal of Petroleum Science and Engineering · OSTI ID:1834598

Wang, Hongsheng; Dalton, Laura; Fan, Ming; +4 more

Related Subjects

36 MATERIALS SCIENCE

Title: Machine-Learning-based Algorithms for Automated Image Segmentation Techniques of Transmission X-ray Microscopy (TXM)

Citation Formats

References (45)

Similar Records

Related Subjects