Structure-aware graph neural network based deep transfer learning framework for enhanced predictive analytics on diverse materials datasets

Gupta, Vishu; Choudhary, Kamal; DeCost, Brian; Tavazza, Francesca; Campbell, Carelyn; Liao, Wei-keng; Choudhary, Alok; Agrawal, Ankit

doi:10.1038/s41524-023-01185-3

Title: Structure-aware graph neural network based deep transfer learning framework for enhanced predictive analytics on diverse materials datasets

Journal Article · 01 January 2024 · npj Computational Materials

DOI: https://doi.org/10.1038/s41524-023-01185-3 · OSTI ID:2274845

;

; Liao, Wei-keng; Choudhary, Alok;

Abstract Modern data mining methods have demonstrated effectiveness in comprehending and predicting materials properties. An essential component in the process of materials discovery is to know which material(s) will possess desirable properties. For many materials properties, performing experiments and density functional theory computations are costly and time-consuming. Hence, it is challenging to build accurate predictive models for such properties using conventional data mining methods due to the small amount of available data. Here we present a framework for materials property prediction tasks using structure information that leverages graph neural network-based architecture along with deep-transfer-learning techniques to drastically improve the model’s predictive ability on diverse materials (3D/2D, inorganic/organic, computational/experimental) data. We evaluated the proposed framework in cross-property and cross-materials class scenarios using 115 datasets to find that transfer learning models outperform the models trained from scratch in 104 cases, i.e., ≈90%, with additional benefits in performance for extrapolation problems. We believe the proposed framework can be widely useful in accelerating materials discovery in materials science.

View Journal Article

Cite

Export

Save

Research Organization:: Northwestern Univ., Evanston, IL (United States)

Sponsoring Organization:: USDOE; USDOE Office of Science (SC)

Grant/Contract Number:: SC0021399

OSTI ID:: 2274845

Journal Information:: npj Computational Materials, Journal Name: npj Computational Materials Journal Issue: 1 Vol. 10; ISSN 2057-3960

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (52)

Crystal structure representations for machine learning models of formation energies Faber, Felix; Lindmaa, Alexander; von Lilienfeld, O. Anatole International Journal of Quantum Chemistry, Vol. 115, Issue 16 https://doi.org/10.1002/qua.24917	journal	April 2015
A first-principles approach to modeling alloy phase equilibria Asta, M.; Ozolins, V.; Woodward, C. JOM, Vol. 53, Issue 9 https://doi.org/10.1007/s11837-001-0062-3	journal	September 2001
Expanding Materials Selection Via Transfer Learning for High-Temperature Oxide Selection McClure, Zachary D.; Strachan, Alejandro JOM, Vol. 73, Issue 1 https://doi.org/10.1007/s11837-020-04411-1	journal	November 2020
Accuracy of ab initio methods in predicting the crystal structures of metals: A review of 80 binary alloys Curtarolo, Stefano; Morgan, Dane; Ceder, Gerbrand Calphad, Vol. 29, Issue 3 https://doi.org/10.1016/j.calphad.2005.01.002	journal	September 2005
Matminer: An open source toolkit for materials data mining Ward, Logan; Dunn, Alexander; Faghaninia, Alireza Computational Materials Science, Vol. 152 https://doi.org/10.1016/j.commatsci.2018.05.018	journal	September 2018
Inverse design of composite metal oxide optical materials based on deep transfer learning and global optimization Dong, Rongzhi; Dan, Yabo; Li, Xiang Computational Materials Science, Vol. 188 https://doi.org/10.1016/j.commatsci.2020.110166	journal	February 2021
Transfer learning for materials informatics using crystal graph convolutional neural network Lee, Joohwi; Asahi, Ryoji Computational Materials Science, Vol. 190 https://doi.org/10.1016/j.commatsci.2021.110314	journal	April 2021
Atomistic calculations and materials informatics: A review Ward, Logan; Wolverton, Chris Current Opinion in Solid State and Materials Science, Vol. 21, Issue 3 https://doi.org/10.1016/j.cossms.2016.07.002	journal	June 2017
Computational Data-Driven Materials Discovery Mannodi-Kanakkithodi, Arun; Chan, Maria K. Y. Trends in Chemistry, Vol. 3, Issue 2 https://doi.org/10.1016/j.trechm.2020.12.007	journal	February 2021
Data-Driven Strategies for Accelerated Materials Design Pollice, Robert; dos Passos Gomes, Gabriel; Aldeghi, Matteo Accounts of Chemical Research, Vol. 54, Issue 4 https://doi.org/10.1021/acs.accounts.0c00785	journal	February 2021
Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals Chen, Chi; Ye, Weike; Zuo, Yunxing Chemistry of Materials, Vol. 31, Issue 9 https://doi.org/10.1021/acs.chemmater.9b01294	journal	April 2019
MPpredictor: An Artificial Intelligence-Driven Web Tool for Composition-Based Material Property Prediction Gupta, Vishu; Choudhary, Kamal; Mao, Yuwei Journal of Chemical Information and Modeling, Vol. 63, Issue 7 https://doi.org/10.1021/acs.jcim.3c00307	journal	March 2023
Predicting the Band Gaps of Inorganic Solids by Machine Learning Zhuo, Ya; Mansouri Tehrani, Aria; Brgoch, Jakoah The Journal of Physical Chemistry Letters, Vol. 9, Issue 7 https://doi.org/10.1021/acs.jpclett.8b00124	journal	March 2018
Predicting Materials Properties with Little Data Using Shotgun Transfer Learning Yamada, Hironao; Liu, Chang; Wu, Stephen ACS Central Science, Vol. 5, Issue 10 https://doi.org/10.1021/acscentsci.9b00804	journal	September 2019
Inference for the Generalization Error Nadeau, Claude Machine Learning, Vol. 52, Issue 3, p. 239-281 https://doi.org/10.1023/A:1024068626366	journal	September 2003
Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data Gupta, Vishu; Choudhary, Kamal; Tavazza, Francesca Nature Communications, Vol. 12, Issue 1 https://doi.org/10.1038/s41467-021-26921-5	journal	November 2021
Machine learning in materials informatics: recent applications and prospects Ramprasad, Rampi; Batra, Rohit; Pilania, Ghanshyam npj Computational Materials, Vol. 3, Issue 1 https://doi.org/10.1038/s41524-017-0056-5	journal	December 2017
Machine learning enabled autonomous microstructural characterization in 3D samples Chan, Henry; Cherukara, Mathew; Loeffler, Troy D. npj Computational Materials, Vol. 6, Issue 1 https://doi.org/10.1038/s41524-019-0267-z	journal	January 2020
A general and transferable deep learning framework for predicting phase formation in materials Feng, Shuo; Fu, Huadong; Zhou, Huiyu npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-020-00488-z	journal	January 2021
Atomistic Line Graph Neural Network for improved materials property predictions Choudhary, Kamal; DeCost, Brian npj Computational Materials, Vol. 7, Issue 1 https://doi.org/10.1038/s41524-021-00650-1	journal	November 2021
An AI-driven microstructure optimization framework for elastic properties of titanium beyond cubic crystal systems Mao, Yuwei; Hasan, Mahmudul; Paul, Arindam npj Computational Materials, Vol. 9, Issue 1 https://doi.org/10.1038/s41524-023-01067-8	journal	June 2023
Machine-learned potentials for next-generation matter simulations Friederich, Pascal; Häse, Florian; Proppe, Jonny Nature Materials, Vol. 20, Issue 6 https://doi.org/10.1038/s41563-020-0777-6	journal	May 2021
ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition Jha, Dipendra; Ward, Logan; Paul, Arindam Scientific Reports, Vol. 8, Issue 1 https://doi.org/10.1038/s41598-018-35934-y	journal	December 2018
Using a Novel Transfer Learning Method for Designing Thin Film Solar Cells with Enhanced Quantum Efficiencies Kaya, Mine; Hajimirza, Shima Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-41316-9	journal	March 2019
Plasma Hsp90 levels in patients with systemic sclerosis and relation to lung and skin involvement: a cross-sectional and longitudinal study Štorkánová, Hana; Oreská, Sabína; Špiritović, Maja Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-020-79139-8	journal	January 2021
Enabling deeper learning on big data for materials informatics applications Jha, Dipendra; Gupta, Vishu; Ward, Logan Scientific Reports, Vol. 11, Issue 1 https://doi.org/10.1038/s41598-021-83193-1	journal	February 2021
Moving closer to experimental level materials property prediction using AI Jha, Dipendra; Gupta, Vishu; Liao, Wei-keng Scientific Reports, Vol. 12, Issue 1 https://doi.org/10.1038/s41598-022-15816-0	journal	July 2022
Improving deep learning model performance under parametric constraints for materials informatics applications Gupta, Vishu; Peltekian, Alec; Liao, Wei-keng Scientific Reports, Vol. 13, Issue 1 https://doi.org/10.1038/s41598-023-36336-5	journal	June 2023
Quantum chemistry structures and properties of 134 kilo molecules Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias Scientific Data, Vol. 1, Issue 1 https://doi.org/10.1038/sdata.2014.22	journal	August 2014
A database to enable discovery and design of piezoelectric materials de Jong, Maarten; Chen, Wei; Geerlings, Henry Scientific Data, Vol. 2, Issue 1 https://doi.org/10.1038/sdata.2015.53	journal	September 2015
High-throughput screening of inorganic compounds for the discovery of novel dielectric and optical materials Petousis, Ioannis; Mrdjenovich, David; Ballouz, Eric Scientific Data, Vol. 4, Issue 1 https://doi.org/10.1038/sdata.2016.134	journal	January 2017
The Harvard organic photovoltaic dataset Lopez, Steven A.; Pyzer-Knapp, Edward O.; Simm, Gregor N. Scientific Data, Vol. 3, Issue 1 https://doi.org/10.1038/sdata.2016.86	journal	September 2016
Experimental formation enthalpies for intermetallic phases and other inorganic compounds Kim, George; Meschel, S. V.; Nash, Philip Scientific Data, Vol. 4, Issue 1 https://doi.org/10.1038/sdata.2017.162	journal	October 2017
Holistic computational structure screening of more than 12 000 candidates for solid lithium-ion conductor materials Sendek, Austin D.; Yang, Qian; Cubuk, Ekin D. Energy & Environmental Science, Vol. 10, Issue 1 https://doi.org/10.1039/C6EE02697D	journal	January 2017
A combined DFT and restricted open-shell configuration interaction method including spin-orbit coupling: Application to transition metal L-edge X-ray absorption spectroscopy Roemelt, Michael; Maganas, Dimitrios; DeBeer, Serena The Journal of Chemical Physics, Vol. 138, Issue 20 https://doi.org/10.1063/1.4804607	journal	May 2013
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation Jain, Anubhav; Ong, Shyue Ping; Hautier, Geoffroy APL Materials, Vol. 1, Issue 1 https://doi.org/10.1063/1.4812323	journal	July 2013
Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science Agrawal, Ankit; Choudhary, Alok APL Materials, Vol. 4, Issue 5 https://doi.org/10.1063/1.4946894	journal	April 2016
OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features Qiao, Zhuoran; Welborn, Matthew; Anandkumar, Animashree The Journal of Chemical Physics, Vol. 153, Issue 12 https://doi.org/10.1063/5.0021955	journal	September 2020
Prediction model of band gap for inorganic compounds by combination of density functional theory calculations and machine learning techniques Lee, Joohwi; Seko, Atsuto; Shitara, Kazuki Physical Review B, Vol. 93, Issue 11 https://doi.org/10.1103/PhysRevB.93.115104	journal	March 2016
Big Data of Materials Science: Critical Role of the Descriptor Ghiringhelli, Luca M.; Vybiral, Jan; Levchenko, Sergey V. Physical Review Letters, Vol. 114, Issue 10 https://doi.org/10.1103/PhysRevLett.114.105503	journal	March 2015
Prediction of Low-Thermal-Conductivity Compounds with First-Principles Anharmonic Lattice-Dynamics Calculations and Bayesian Optimization Seko, Atsuto; Togo, Atsushi; Hayashi, Hiroyuki Physical Review Letters, Vol. 115, Issue 20 https://doi.org/10.1103/PhysRevLett.115.205901	journal	November 2015
Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties Xie, Tian; Grossman, Jeffrey C. Physical Review Letters, Vol. 120, Issue 14 https://doi.org/10.1103/PhysRevLett.120.145301	journal	April 2018
Quantifying uncertainty in high-throughput density functional theory: A comparison of AFLOW, Materials Project, and OQMD Hegde, Vinay I.; Borg, Christopher K. H.; del Rosario, Zachary Physical Review Materials, Vol. 7, Issue 5 https://doi.org/10.1103/PhysRevMaterials.7.053805	journal	May 2023
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks Oquab, Maxime; Bottou, Leon; Laptev, Ivan 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2014.222	conference	June 2014
Pre-Activation based Representation Learning to Enhance Predictive Analytics on Small Materials Data Gupta, Vishu; Liao, Wei-keng; Choudhary, Alok 2023 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN54540.2023.10191086	conference	June 2023
AI for Learning Deformation Behavior of a Material: Predicting Stress-Strain Curves 4000x Faster Than Simulations Mao, Yuwei; Keshavarz, Shahriyar; Gupta, Vishu 2023 International Joint Conference on Neural Networks (IJCNN) https://doi.org/10.1109/IJCNN54540.2023.10191138	conference	June 2023
BRNet: Branched Residual Network for Fast and Accurate Predictive Modeling of Materials Properties Gupta, Vishu; Liao, Wei-keng; Choudhary, Alok Proceedings of the 2022 SIAM International Conference on Data Mining (SDM) https://doi.org/10.1137/1.9781611977172.39	book	January 2022
Physics-based Data-Augmented Deep Learning for Enhanced Autogenous Shrinkage Prediction on Experimental Dataset Gupta, Vishu; Lyu, Yuhui; Suarez, Derick Proceedings of the 2023 Fifteenth International Conference on Contemporary Computing https://doi.org/10.1145/3607947.3607980	conference	August 2023
Opportunities and Challenges for Machine Learning in Materials Science Morgan, Dane; Jacobs, Ryan Annual Review of Materials Research, Vol. 50, Issue 1 https://doi.org/10.1146/annurev-matsci-070218-010015	journal	July 2020
Handbook of Parametric and Nonparametric Statistical Procedures Sheskin, David J. https://doi.org/10.1201/9781420036268	book	August 2003
Deep materials informatics: Applications of deep learning in materials science Agrawal, Ankit; Choudhary, Alok MRS Communications, Vol. 9, Issue 3 https://doi.org/10.1557/mrc.2019.73	journal	June 2019
Materials science with large-scale data and informatics: Unlocking new opportunities Hill, Joanne; Mulholland, Gregory; Persson, Kristin MRS Bulletin, Vol. 41, Issue 5 https://doi.org/10.1557/mrs.2016.93	journal	May 2016

Similar Records

Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data

Journal Article · 2021 · Nature Communications · OSTI ID:1830279

Gupta, Vishu; Choudhary, Kamal; Tavazza, Francesca; +4 more

XElemNet: towards explainable AI for deep neural networks in materials science

Journal Article · 2024 · Scientific Reports · OSTI ID:2473509

Wang, Kewei; Gupta, Vishu; Lee, Claire Songhyun; +7 more

AtomSets as a hierarchical transfer learning framework for small and large materials datasets

Journal Article · 2021 · npj Computational Materials · OSTI ID:1827232

Chen, Chi; Ong, Shyue Ping

Related Subjects

Chemistry
Materials Science

Title: Structure-aware graph neural network based deep transfer learning framework for enhanced predictive analytics on diverse materials datasets

Citation Formats

References (52)

Similar Records

Related Subjects