Decoding diffraction and spectroscopy data with machine learning: A tutorial

Vizoso, Daniel Enrique; Dingreville, Remi Philippe Michel

doi:10.1063/5.0255593

Title: Decoding diffraction and spectroscopy data with machine learning: A tutorial

Journal Article · 03 April 2025 · Journal of Applied Physics

DOI: https://doi.org/10.1063/5.0255593 · OSTI ID:2549513

^[1];

^[1]

Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

This Tutorial provides a step-by-step guide on how to apply supervised machine-learning techniques to analyze diffraction and spectroscopy data. This Tutorial details four models—a reconstruction-focused model, a regression-focused model, a hybrid reconstruction/regression model, and a multimodal model—that use x-ray diffraction profiles and vibrational density of states spectra to predict various microstructural descriptors. In this Tutorial, we cover data pre-processing steps, constructions of the models via dimensionality reduction and regression, training, and analysis of these models. Comparisons of the model’s performance are provided, highlighting the strength and weakness of the various approaches utilized.

View Journal Article

Cite

Export

Save

Research Organization:: Sandia National Laboratories (SNL-NM), Albuquerque, NM (United States)

Sponsoring Organization:: USDOE; USDOE National Nuclear Security Administration (NNSA)

Grant/Contract Number:: NA0003525

OSTI ID:: 2549513

Report Number(s):: SAND--2025-04172J

Journal Information:: Journal of Applied Physics, Journal Name: Journal of Applied Physics Journal Issue: 13 Vol. 137; ISSN 0021-8979

Publisher:: American Institute of Physics (AIP)Copyright Statement

Country of Publication:: United States

Language:: English

References (60)

Results and interpretation of a fitting challenge for MR spectroscopy set up by the MRS study group of ISMRM Marjańska, Małgorzata; Deelchand, Dinesh K.; Kreis, Roland Magnetic Resonance in Medicine, Vol. 87, Issue 1 https://doi.org/10.1002/mrm.28942	journal	August 2021
Linear Methods for Regression Hastie, Trevor; Tibshirani, Robert; Friedman, Jerome The Elements of Statistical Learning https://doi.org/10.1007/978-0-387-84858-7_3	book	December 2008
The high throughput X-ray spectroscopy mission: XMM Peacock, A.; Ellwood, J. Space Science Reviews, Vol. 48, Issue 3-4 https://doi.org/10.1007/BF00226012	journal	January 1988
Integral wide-field spectroscopy in astronomy: the Imaging FTS solution Maillard, J. P.; Drissen, L.; Grandmont, F. Experimental Astronomy, Vol. 35, Issue 3 https://doi.org/10.1007/s10686-013-9330-9	journal	March 2013
An Open Combinatorial Diffraction Dataset Including Consensus Human and Machine Learning Labels with Quantified Uncertainty for Training New Machine Learning Models Hattrick-Simpers, Jason R.; DeCost, Brian; Kusne, A. Gilad Integrating Materials and Manufacturing Innovation, Vol. 10, Issue 2 https://doi.org/10.1007/s40192-021-00213-8	journal	June 2021
Beyond Combinatorial Materials Science: The 100 Prisoners Problem Fowler, J. Elliott; Kottwitz, Matthew A.; Trask, Nat Integrating Materials and Manufacturing Innovation, Vol. 13, Issue 1 https://doi.org/10.1007/s40192-023-00330-6	journal	January 2024
SMART mineral mapping: Synchrotron-based machine learning approach for 2D characterization with coupled micro XRF-XRD Kim, Julie J.; Ling, Florence T.; Plattenberger, Dan A. Computers & Geosciences, Vol. 156 https://doi.org/10.1016/j.cageo.2021.104898	journal	November 2021
Crystallographic prediction from diffraction and chemistry data for higher throughput classification using machine learning Aguiar, Jeffery A.; Gong, Matthew L.; Tasdizen, Tolga Computational Materials Science, Vol. 173 https://doi.org/10.1016/j.commatsci.2019.109409	journal	February 2020
LAMMPS - a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales Thompson, Aidan P.; Aktulga, H. Metin; Berger, Richard Computer Physics Communications, Vol. 271 https://doi.org/10.1016/j.cpc.2021.108171	journal	February 2022
Dataset of simulated vibrational density of states and X-ray diffraction profiles of mechanically deformed and disordered atomic structures in Gold, Iron, Magnesium, and Silicon Vizoso, Daniel; Dingreville, Rémi Data in Brief, Vol. 55 https://doi.org/10.1016/j.dib.2024.110689	journal	August 2024
Raman spectroscopy as a tool to determine the thermal maturity of organic matter: Application to sedimentary, metamorphic and structural geology Henry, Delano G.; Jarvis, Ian; Gillmore, Gavin Earth-Science Reviews, Vol. 198 https://doi.org/10.1016/j.earscirev.2019.102936	journal	November 2019
Small-angle X-ray scattering: a high-throughput technique for investigating archaeological bone preservation Hiller, J. C.; Collins, M. J.; Chamberlain, A. T. Journal of Archaeological Science, Vol. 31, Issue 10 https://doi.org/10.1016/j.jas.2004.02.013	journal	October 2004
An X-ray Diffraction (XRD) and X-ray Fluorescence (XRF) investigation in human and animal fossil bones from Holocene to Middle Triassic Piga, Giampaolo; Santos-Cubedo, Andrés; Moya Solà, Salvador Journal of Archaeological Science, Vol. 36, Issue 9 https://doi.org/10.1016/j.jas.2009.04.013	journal	September 2009
Application of chord length distributions and principal component analysis for quantification and representation of diverse polycrystalline microstructures Latypov, Marat I.; Kühbach, Markus; Beyerlein, Irene J. Materials Characterization, Vol. 145 https://doi.org/10.1016/j.matchar.2018.09.020	journal	November 2018
Navigating high-dimensional process-structure–property relations in nanocrystalline Pt-Au alloys with machine learning Desai, Saaketh; Jain, Manish; Addamane, Sadhvikas J. Materials & Design, Vol. 248 https://doi.org/10.1016/j.matdes.2024.113494	journal	December 2024
Machine learning for materials science: Barriers to broader adoption Boyce, Brad; Dingreville, Remi; Desai, Saaketh Matter, Vol. 6, Issue 5 https://doi.org/10.1016/j.matt.2023.03.028	journal	May 2023
Reconstruction of high-resolution atomic force microscopy measurements from fast-scan data using a Noise2Noise algorithm Natinsky, Eva; Khan, Ryan M.; Cullinan, Michael Measurement, Vol. 227 https://doi.org/10.1016/j.measurement.2024.114263	journal	March 2024
An extensive experimental survey of regression methods Fernández-Delgado, M.; Sirsat, M. S.; Cernadas, E. Neural Networks, Vol. 111 https://doi.org/10.1016/j.neunet.2018.12.010	journal	March 2019
Use of X-ray powder diffraction for quantitative analysis of carbonate rock reservoir samples Al-Jaroudi, Said S.; Ul-Hamid, Anwar; Mohammed, Abdul-Rashid I. Powder Technology, Vol. 175, Issue 3 https://doi.org/10.1016/j.powtec.2007.01.013	journal	June 2007
Advances in Laser-Induced Breakdown Spectroscopy analysis for geology: A critical review Fabre, Cécile Spectrochimica Acta Part B: Atomic Spectroscopy, Vol. 166 https://doi.org/10.1016/j.sab.2020.105799	journal	April 2020
Quantitative mineralogy for facies definition in the Marcellus Shale (Appalachian Basin, USA) using XRD-XRF integration Hupp, Brittany N.; Donovan, Joseph J. Sedimentary Geology, Vol. 371 https://doi.org/10.1016/j.sedgeo.2018.04.007	journal	September 2018
Vibrational spectroscopy as a tool for studying drug-cell interaction: Could high throughput vibrational spectroscopic screening improve drug development? Jamieson, Lauren E.; Byrne, Hugh J. Vibrational Spectroscopy, Vol. 91 https://doi.org/10.1016/j.vibspec.2016.09.003	journal	July 2017
Connecting Vibrational Spectroscopy to Atomic Structure via Supervised Manifold Learning: Beyond Peak Analysis Vizoso, Daniel; Subhash, Ghatu; Rajan, Krishna Chemistry of Materials, Vol. 35, Issue 3 https://doi.org/10.1021/acs.chemmater.2c03207	journal	January 2023
Applying Machine Learning to Vibrational Spectroscopy Fu, Weiqiang; Hopkins, W. Scott The Journal of Physical Chemistry A, Vol. 122, Issue 1 https://doi.org/10.1021/acs.jpca.7b10303	journal	December 2017
Machine Learning Automated Analysis of Enormous Synchrotron X-ray Diffraction Datasets Zhao, Xiaodong; Luo, YiXuan; Liu, Juejing The Journal of Physical Chemistry C, Vol. 127, Issue 30, p. 14830-14838 https://doi.org/10.1021/acs.jpcc.3c03572	journal	July 2023
High Throughput Single Nanoparticle Spectroscopy Sebba, David S.; Watson, Dakota A.; Nolan, John P. ACS Nano, Vol. 3, Issue 6 https://doi.org/10.1021/nn9003346	journal	May 2009
High-throughput optical imaging and spectroscopy of individual carbon nanotubes in devices Liu, Kaihui; Hong, Xiaoping; Zhou, Qin Nature Nanotechnology, Vol. 8, Issue 12 https://doi.org/10.1038/nnano.2013.227	journal	November 2013
Fast and interpretable classification of small X-ray diffraction datasets using data augmentation and deep neural networks Oviedo, Felipe; Ren, Zekun; Sun, Shijing npj Computational Materials, Vol. 5, Issue 1 https://doi.org/10.1038/s41524-019-0196-x	journal	May 2019
Decoding defect statistics from diffractograms via machine learning Kunka, Cody; Shanker, Apaar; Chen, Elton Y. npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00539-z	journal	May 2021
Three-dimensional coherent X-ray diffraction imaging via deep convolutional neural networks Wu, Longlong; Yoo, Shinjae; Suzana, Ana F. npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00644-z	journal	October 2021
Hydrogenative alkene perdeuteration aided by a transient cooperative ligand Luo, Jie; Lu, Lijun; Montag, Michael Nature Chemistry, Vol. 15, Issue 10 https://doi.org/10.1038/s41557-023-01313-y	journal	September 2023
Symmetry prediction and knowledge discovery from X-ray diffraction patterns using an interpretable machine learning approach Suzuki, Yuta; Hino, Hideitsu; Hawai, Takafumi Scientific Reports, Vol. 10, Issue 1 https://doi.org/10.1038/s41598-020-77474-4	journal	December 2020
Efficient few-shot machine learning for classification of EBSD patterns Kaufmann, Kevin; Lane, Hobson; Liu, Xiao Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-87557-5	journal	April 2021
Principal component analysis Bro, Rasmus; Smilde, Age K. Anal. Methods, Vol. 6, Issue 9 https://doi.org/10.1039/C3AY41907J	journal	January 2014
Methodological evolutions of Raman spectroscopy in art and archaeology Bersani, Danilo; Conti, Claudia; Matousek, Pavel Analytical Methods, Vol. 8, Issue 48 https://doi.org/10.1039/C6AY02327D	journal	January 2016
Revealing inconsistencies in X-ray width methods for nanomaterials Kunka, Cody; Boyce, Brad L.; Foiles, Stephen M. Nanoscale, Vol. 11, Issue 46 https://doi.org/10.1039/C9NR08268A	journal	January 2019
Regression methods in the comparison of accuracy Thompson, Michael The Analyst, Vol. 107, Issue 1279 https://doi.org/10.1039/an9820701169	journal	January 1982
Stress measurements using ultraviolet micro-Raman spectroscopy Dombrowski, K. F.; De Wolf, I.; Dietrich, B. Applied Physics Letters, Vol. 75, Issue 16 https://doi.org/10.1063/1.125044	journal	October 1999
Invited Article: Simultaneous mapping of temperature and stress in microdevices using micro-Raman spectroscopy Beechem, Thomas; Graham, Samuel; Kearney, Sean P. Review of Scientific Instruments, Vol. 78, Issue 6 https://doi.org/10.1063/1.2738946	journal	June 2007
Stress measurements in silicon devices through Raman spectroscopy: Bridging the gap between theory and experiment De Wolf, Ingrid; Maes, H. E.; Jones, Stephen K. Journal of Applied Physics, Vol. 79, Issue 9 https://doi.org/10.1063/1.361485	journal	May 1996
A laboratory spectrometer for high throughput X-ray emission spectroscopy in catalysis research Malzer, Wolfgang; Grötzsch, Daniel; Gnewkow, Richard Review of Scientific Instruments, Vol. 89, Issue 11 https://doi.org/10.1063/1.5035171	journal	November 2018
Powder x-ray diffraction analysis with machine learning for organic-semiconductor crystal-structure determination Niitsu, Naoyuki; Mitani, Masato; Ishii, Hiroyuki Applied Physics Letters, Vol. 125, Issue 1 https://doi.org/10.1063/5.0208919	journal	July 2024
Uncertainty quantification on small angle x-ray scattering measurement using Bayesian deep learning Yang, Hairui; Wu, Zhaolong; Zhang, Kezhong Journal of Applied Physics, Vol. 136, Issue 14 https://doi.org/10.1063/5.0226735	journal	October 2024
Harnessing interpretable and unsupervised machine learning to address big data from modern X-ray diffraction Venderley, Jordan; Mallayya, Krishnanand; Matty, Michael Proceedings of the National Academy of Sciences, Vol. 119, Issue 24 https://doi.org/10.1073/pnas.2109665119	journal	June 2022
Applications of machine learning in spectroscopy Meza Ramirez, Carlos A.; Greenop, Michael; Ashton, Lorna Applied Spectroscopy Reviews, Vol. 56, Issue 8-10 https://doi.org/10.1080/05704928.2020.1859525	journal	December 2020
Automated Data Analysis for Powder X-Ray Diffraction Using Machine Learning Suzuki, Yuta Synchrotron Radiation News, Vol. 35, Issue 4 https://doi.org/10.1080/08940886.2022.2112496	journal	July 2022
LIII. On lines and planes of closest fit to systems of points in space Pearson, Karl The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, Vol. 2, Issue 11 https://doi.org/10.1080/14786440109462720	journal	November 1901
Highly optimized embedded-atom-method potentials for fourteen fcc metals Sheng, H. W.; Kramer, M. J.; Cadien, A. Physical Review B, Vol. 83, Issue 13 https://doi.org/10.1103/PhysRevB.83.134118	journal	April 2011
Stress analysis of polycrystalline thin films and surface regions by X-ray diffraction Welzel, U.; Ligot, J.; Lamparter, P. Journal of Applied Crystallography, Vol. 38, Issue 1 https://doi.org/10.1107/S0021889804029516	journal	January 2005
Phase identification and structure determination from multiphase crystalline powder samples by rotation electron diffraction Yun, Yifeng; Wan, Wei; Rabbani, Faiz Journal of Applied Crystallography, Vol. 47, Issue 6 https://doi.org/10.1107/S1600576714023875	journal	November 2014
High-throughput continuous rotation electron diffraction data acquisition via software automation Cichocka, Magdalena Ola; Ångström, Jonas; Wang, Bin Journal of Applied Crystallography, Vol. 51, Issue 6 https://doi.org/10.1107/S1600576718015145	journal	November 2018
High-throughput synchrotron X-ray diffraction for combinatorial phase mapping Gregoire, J. M.; Van Campen, D. G.; Miller, C. E. Journal of Synchrotron Radiation, Vol. 21, Issue 6 https://doi.org/10.1107/S1600577514016488	journal	October 2014
TEM and XRD determination of crystallite size and lattice strain as a function of illite crystallinity in pelitic rocks Jiang, W. ‐T.; Peacor, D. R.; ÁRkai, P. Journal of Metamorphic Geology, Vol. 15, Issue 2 https://doi.org/10.1111/j.1525-1314.1997.00016.x	journal	February 1997
Guided combinatorial synthesis and automated characterization expedites the discovery of hard, electrically conductive PtxAu1−x films Adams, D. P.; Kothari, R.; Addamane, S. Journal of Vacuum Science & Technology A, Vol. 42, Issue 5 https://doi.org/10.1116/6.0003785	journal	August 2024
Application of machine learning to x-ray diffraction-based classification Zhao, Bi; Greenberg, Joel A.; Wolter, Scott Anomaly Detection and Imaging with X-Rays (ADIX) III https://doi.org/10.1117/12.2304683	conference	May 2018
Application of Vibrational Spectroscopy and Imaging to Point-of-Care Medicine: A Review Pahlow, Susanne; Weber, Karina; Popp, Jürgen Applied Spectroscopy, Vol. 72, Issue 1_suppl https://doi.org/10.1177/0003702818791939	journal	September 2018
Infrared Methods for High Throughput Screening of Metabolites: Food and Medical Applications Cozzolino, Daniel Combinatorial Chemistry & High Throughput Screening, Vol. 14, Issue 2 https://doi.org/10.2174/138620711794474105	journal	February 2011
X-ray Diffraction Data Analysis by Machine Learning Methods—A Review Surdu, Vasile-Adrian; Győrgy, Romuald Applied Sciences, Vol. 13, Issue 17 https://doi.org/10.3390/app13179992	journal	September 2023
A Review of Machine Learning for Near-Infrared Spectroscopy Zhang, Wenwen; Kasun, Liyanaarachchi Chamara; Wang, Qi Jie Sensors, Vol. 22, Issue 24 https://doi.org/10.3390/s22249764	journal	December 2022
Phase Identification Using Series of Selected Area Diffraction Patterns and Energy Dispersive Spectrometry within TEM Lin, Kun-Lin Microscopy Research, Vol. 02, Issue 04 https://doi.org/10.4236/mr.2014.24008	journal	January 2014

Similar Records

Exploring the transferability of machine-learning models for analyzing XRD data of shocked microstructures: from single crystal to polycrystals

Journal Article · 2025 · Digital Discovery · OSTI ID:2564790

Vizoso, Daniel Enrique; Tsurkan, Phillip; Ma, Ke; +2 more

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

Journal Article · 2023 · Mechanical Systems and Signal Processing · OSTI ID:2320359

Nemani, Venkat; Biggio, Luca; Huan, Xun; +6 more

SNM Radiation Signature Classification Using Different Semi-Supervised Machine Learning Models

Journal Article · 2023 · Journal of Nuclear Engineering · OSTI ID:1988203

Stomps, Jordan R.; Wilson, Paul P. H.; Dayman, Kenneth J.; +3 more

Related Subjects

36 MATERIALS SCIENCE
Artificial neural networks
Machine learning
Material characterization methods
Molecular dynamics
Regression analysis
Spectroscopy
Statistical analysis
X-ray diffraction

Title: Decoding diffraction and spectroscopy data with machine learning: A tutorial

Citation Formats

References (60)

Similar Records

Related Subjects