Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

Adoption of image-driven machine learning for microstructure characterization and materials design: A Perspective

Journal Article · · JOM. The Journal of the Minerals, Metals and Materials Society
 [1];  [2];  [3];  [1];  [1];  [1]
  1. Rensselaer Polytechnic Institute
  2. BATTELLE (PACIFIC NW LAB)
  3. General Electric Research Center

Microstructure characterization enables the development of structure-processing-property relationships critical to several research areas within the broad field of materials science, from alloy design to the assessment of corrosion resistance, and failure analysis. Conventional approaches to material characterization have relied on either qualitative inference by the human ex-pert or software applications that can extract high-level features from images, such as boundary segmentation, average grain diameter, etc. Such approaches rely heavily on subject matter expert user intervention and knowledge of what phases or more generally, what microstructural features, are of interest. The recent surge in the adoption of machine learning techniques to address problems in materials engineering has brought with it an increased interest and application of Image Driven Machine Learning (IDML) approaches. In this work, we review the applications of IDML to the field of materials characterization. A canonical hierarchy of stages is defined, which when put sequentially together completes an IDML study: problem definition, dataset building, model selection and training, model evaluation, and integration with existing instrumentation or simulation workflow. The studies reviewed in this work are analyzed from the perspective of each of these stages. Such a review permits agranular assessment of the field, for example the impact of IDML on materials characterization at the nanoscale, the size of a typical dataset required to train a semantic segmentation model on electron microscopy images, ubiquitousness of transfer learning in the domain, etc. Finally, we discuss the importance of interpretability and explainability in the field of IDML for materials characterization, and provide an overview of two emerging techniques in the field: semantic segmentation and generative adversarial networks.

Research Organization:
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE
DOE Contract Number:
AC05-76RL01830
OSTI ID:
1829454
Report Number(s):
PNNL-SA-159271
Journal Information:
JOM. The Journal of the Minerals, Metals and Materials Society, Vol. 73, Issue 11
Country of Publication:
United States
Language:
English

References (95)

Quantitative serial sectioning analysis journal March 2001
Three-dimensional nanoscale characterisation of materials by atom probe tomography journal November 2016
Deciphering atomistic mechanisms of the gas-solid interfacial reaction during alloy oxidation journal April 2020
Element redistributions during early stages of oxidation in a Ni38Cr22Fe20Mn10Co10 multi-principal element alloy journal March 2021
The materials innovation ecosystem: A key enabler for the Materials Genome Initiative journal April 2016
Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science journal April 2016
Materials informatics: From the atomic-level to the continuum journal April 2019
Machine learning in materials informatics: recent applications and prospects journal December 2017
A computer vision approach for automated analysis and classification of microstructural image data journal December 2015
Image driven machine learning methods for microstructure recognition journal October 2016
Grain size measurement by the intercept method journal February 1971
Slice-to-voxel stochastic reconstructions on porous media with hybrid deep generative model journal January 2021
Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. journal May 2018
Using machine learning to identify in-situ melt pool signatures indicative of flaw formation in a laser powder bed fusion additive manufacturing process journal January 2019
Optimizing convolutional neural networks to perform semantic segmentation on large materials imaging datasets: X-ray tomography and serial sectioning journal February 2020
Semantic segmentation of synchrotron tomography of multiphase Al-Si alloys using a convolutional neural network with a pixel-wise weighted loss function journal December 2019
Advanced Deep Learning‐Based 3D Microstructural Characterization of Multiphase Metal Matrix Composites journal February 2020
Toward quantitative fractography using convolutional neural networks journal May 2020
Advanced Steel Microstructural Classification by Deep Learning Methods journal February 2018
New methods for automatic quantification of microstructural features using digital image processing journal March 2018
Quantitative Metallographic Analysis of GCr15 Microstructure Using Mask R-CNN journal May 2020
A deep learning approach for determining the chiral indices of carbon nanotubes from high-resolution transmission electron microscopy images journal November 2020
Deep Learning of Atomically Resolved Scanning Transmission Electron Microscopy Images: Chemical Identification and Tracking Local Transformations journal October 2016
Deep Learning for Semantic Segmentation of Defects in Advanced STEM Images of Steels journal September 2019
Machine Learning to Reveal Nanoparticle Dynamics from Liquid-Phase TEM Videos journal July 2020
3D nanostructural characterisation of grain boundaries in atom probe data utilising machine learning methods journal November 2019
Machine learning enabled autonomous microstructural characterization in 3D samples journal January 2020
Phase Segmentation in Atom-Probe Tomography Using Deep Learning-Based Edge Detection journal December 2019
Adaptive characterization of microstructure dataset using a two stage machine learning approach journal May 2020
Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships journal October 2020
Decoding crystallography from high-resolution electron imaging and diffraction datasets with deep learning journal October 2019
Computational microstructure characterization and reconstruction: Review of the state-of-the-art techniques journal June 2018
Reduced-order structure-property linkages for polycrystalline microstructures based on 2-point statistics journal May 2017
Microstructure of two‐phase random media. I. The n ‐point probability functions journal August 1982
Microstructure informatics using higher-order statistics and efficient data-mining protocols journal April 2011
Hierarchical n-point polytope functions for quantitative representation of complex heterogeneous materials and microstructural evolution journal October 2019
Stochastic nonlinear analysis of unidirectional fiber composites using image-based microstructural uncertainty quantification journal March 2021
Autonomy in materials research: a case study in carbon nanotube growth journal October 2016
Harnessing the Materials Project for machine-learning and accelerated discovery journal September 2018
Data-driven design of inorganic materials with the Automatic Flow Framework for Materials Discovery journal September 2018
NOMAD: The FAIR concept for big data-driven materials science journal September 2018
An efficient and accurate framework for calculating lattice thermal conductivity of solids: AFLOW—AAPL Automatic Anharmonic Phonon Library journal October 2017
Big–deep–smart data in imaging for guiding materials design journal September 2015
A machine learning approach to thermal conductivity modeling: A case study on irradiated uranium-molybdenum nuclear fuels journal April 2019
Machine-learning assisted laser powder bed fusion process optimization for AlSi10Mg: New microstructure description indices and fracture mechanisms journal December 2020
An approach for the aging process optimization of Al–Zn–Mg–Cu series alloys journal August 2009
Deep learning approaches for mining structure-property linkages in high contrast composites from simulation datasets journal August 2018
Progress in nanoinformatics and informational materials science journal September 2018
Energy landscapes for machine learning journal January 2017
Towards exact molecular dynamics simulations with machine-learned force fields journal September 2018
Construction of high-dimensional neural network potentials using environment-dependent atom pairs journal May 2012
Machine learning unifies the modeling of materials and molecules journal December 2017
Materials Synthesis Insights from Scientific Literature via Text Extraction and Machine Learning journal October 2017
Building data-driven models with microstructural images: Generalization and interpretability journal December 2017
An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction journal August 2020
Classification of crystal structure using a convolutional neural network journal June 2017
Materials discovery and design using machine learning journal September 2017
Machine learning for molecular and materials science journal July 2018
Human-level concept learning through probabilistic program induction journal December 2015
Use of neural networks to analyze pulse shape data in low-background detectors journal July 2018
A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy journal July 2017
Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions journal April 2020
High Throughput Quantitative Metallography for Complex Microstructures Using Deep Learning: A Case Study in Ultrahigh Carbon Steel journal February 2019
Machine Learning Techniques for the Segmentation of Tomographic Image Data of Functional Materials journal June 2019
Integrated application of semantic segmentation-assisted deep learning to quantitative multi-phased microstructural analysis in composite materials: Case study of cathode composite materials of solid oxide fuel cells journal September 2020
Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images journal April 2018
Computer Vision and Machine Learning for Autonomous Characterization of AM Powder Feedstocks journal December 2016
Establishing structure-property localization linkages for elastic deformation of three-dimensional high contrast composites using deep learning approaches journal March 2019
Deep Network-Based Feature Extraction and Reconstruction of Complex Material Microstructures
  • Cang, Ruijin; Ren, Max Yi
  • ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 2B: 42nd Design Automation Conference https://doi.org/10.1115/DETC2016-59404
conference December 2016
Trainable Weka Segmentation: a machine learning tool for microscopy pixel classification journal March 2017
Hierarchical level features based trainable segmentation for electron microscopy images journal June 2013
Resolution enhancement in scanning electron microscopy using deep learning journal August 2019
Crystal symmetry determination in electron diffraction using machine learning journal January 2020
A Deep Learning Approach to Identify Local Structures in Atomic-Resolution Transmission Electron Microscopy Images journal July 2018
Microstructural diagram for steel based on crystallography with machine learning journal March 2019
Material image segmentation based on feature similarity and non-spherical clustering journal October 2019
Semantic segmentation of the micro-structure of strain-hardening cement-based composites (SHCC) by applying deep learning on micro-computed tomography scans journal April 2020
High-throughput experiments facilitate materials innovation: A review journal February 2019
Quantitative microstructure analysis for solid-state metal additive manufacturing via deep learning journal June 2020
Inferring low-dimensional microstructure representations using convolutional neural networks journal November 2017
Novel microstructure quantification framework for databasing, visualization, and analysis of microstructure data journal July 2013
Descriptor-based methodology for statistical characterization and 3D reconstruction of microstructural materials journal April 2014
A Machine Learning-Based Design Representation Method for Designing Heterogeneous Microstructures journal May 2015
Rotation equivariant and invariant neural networks for microscopy image analysis journal July 2019
AutoDetect-mNP: An Unsupervised Machine Learning Algorithm for Automated Analysis of Transmission Electron Microscope Images of Metal Nanoparticles journal February 2021
Towards data-driven next-generation transmission electron microscopy journal October 2020
167-PFlops Deep Learning for Electron Microscopy: From Learning Physics to Atomic Manipulation
  • Patton, Robert M.; Johnston, J. Travis; Young, Steven R.
  • SC18: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2018.00053
conference November 2018
Toward Large-Scale Image Segmentation on Summit
  • Seal, Sudip K.; Lim, Seung-Hwan; Wang, Dali
  • ICPP '20: 49th International Conference on Parallel Processing, 49th International Conference on Parallel Processing - ICPP https://doi.org/10.1145/3404397.3404468
conference August 2020
Opportunities and Challenges for Machine Learning in Materials Science journal July 2020
Cost-Effective Active Learning for Deep Image Classification journal December 2017
Active Learning Methods for Remote Sensing Image Classification journal July 2009
Visual interpretability for deep learning: a survey journal January 2018
Elucidating the Behavior of Nanophotonic Structures through Explainable Machine Learning Algorithms journal July 2020
Interpretable deep learning for guided microstructure-property explorations in photovoltaics journal October 2019
Correlated Materials Characterization via Multimodal Chemical and Functional Imaging journal November 2018