Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals
Abstract
Graph networks are a new machine learning (ML) paradigm that supports both relational reasoning and combinatorial generalization. Here, we develop universal MatErials Graph Network (MEGNet) models for accurate property prediction in both molecules and crystals. We demonstrate that the MEGNet models outperform prior ML models such as the SchNet in 11 out of 13 properties of the QM9 molecule data set. Similarly, we show that MEGNet models trained on $$\sim 60,000$$ crystals in the Materials Project substantially outperform prior ML models in the prediction of the formation energies, band gaps and elastic moduli of crystals, achieving better than DFT accuracy over a much larger data set. Here, we present two new strategies to address data limitations common in materials science and chemistry. First, we demonstrate a physically-intuitive approach to unify four separate molecular MEGNet models for the internal energy at 0 K and room temperature, enthalpy and Gibbs free energy into a single free energy MEGNet model by incorporating the temperature, pressure and entropy as global state inputs. Second, we show that the learned element embeddings in MEGNet models encode periodic chemical trends and can be transfer-learned from a property model trained on a larger data set (formation energies) to improve property models with smaller amounts of data (band gaps and elastic moduli).
- Publication Date:
- Research Org.:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES) (SC-22). Materials Sciences & Engineering Division
- OSTI Identifier:
- 1564028
- Grant/Contract Number:
- AC02-05CH11231
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Chemistry of Materials
- Additional Journal Information:
- Journal Volume: 31; Journal Issue: 9; Journal ID: ISSN 0897-4756
- Publisher:
- American Chemical Society (ACS)
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 36 MATERIALS SCIENCE
Citation Formats
Chen, Chi, Ye, Weike, Zuo, Yunxing, Zheng, Chen, and Ong, Shyue Ping. Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals. United States: N. p., 2019.
Web. doi:10.1021/acs.chemmater.9b01294.
Chen, Chi, Ye, Weike, Zuo, Yunxing, Zheng, Chen, & Ong, Shyue Ping. Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals. United States. https://doi.org/10.1021/acs.chemmater.9b01294
Chen, Chi, Ye, Weike, Zuo, Yunxing, Zheng, Chen, and Ong, Shyue Ping. Wed .
"Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals". United States. https://doi.org/10.1021/acs.chemmater.9b01294. https://www.osti.gov/servlets/purl/1564028.
@article{osti_1564028,
title = {Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals},
author = {Chen, Chi and Ye, Weike and Zuo, Yunxing and Zheng, Chen and Ong, Shyue Ping},
abstractNote = {Graph networks are a new machine learning (ML) paradigm that supports both relational reasoning and combinatorial generalization. Here, we develop universal MatErials Graph Network (MEGNet) models for accurate property prediction in both molecules and crystals. We demonstrate that the MEGNet models outperform prior ML models such as the SchNet in 11 out of 13 properties of the QM9 molecule data set. Similarly, we show that MEGNet models trained on $\sim 60,000$ crystals in the Materials Project substantially outperform prior ML models in the prediction of the formation energies, band gaps and elastic moduli of crystals, achieving better than DFT accuracy over a much larger data set. Here, we present two new strategies to address data limitations common in materials science and chemistry. First, we demonstrate a physically-intuitive approach to unify four separate molecular MEGNet models for the internal energy at 0 K and room temperature, enthalpy and Gibbs free energy into a single free energy MEGNet model by incorporating the temperature, pressure and entropy as global state inputs. Second, we show that the learned element embeddings in MEGNet models encode periodic chemical trends and can be transfer-learned from a property model trained on a larger data set (formation energies) to improve property models with smaller amounts of data (band gaps and elastic moduli).},
doi = {10.1021/acs.chemmater.9b01294},
journal = {Chemistry of Materials},
number = 9,
volume = 31,
place = {United States},
year = {Wed Apr 10 00:00:00 EDT 2019},
month = {Wed Apr 10 00:00:00 EDT 2019}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 403 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
Learning atoms for materials discovery
journal, June 2018
- Zhou, Quan; Tang, Peizhe; Liu, Shenxiu
- Proceedings of the National Academy of Sciences, Vol. 115, Issue 28
Novel mixed polyanions lithium-ion battery cathode materials predicted by high-throughput ab initio computations
journal, January 2011
- Hautier, Geoffroy; Jain, Anubhav; Chen, Hailong
- Journal of Materials Chemistry, Vol. 21, Issue 43
Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials
journal, March 2015
- Thompson, A. P.; Swiler, L. P.; Trott, C. R.
- Journal of Computational Physics, Vol. 285
AFLOW: An automatic framework for high-throughput materials discovery
text, January 2013
- Curtarolo, Stefano; Setyawan, Wahyu; Hart, Gus L. W.
- arXiv
Deep learning
journal, May 2015
- LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey
- Nature, Vol. 521, Issue 7553
Charting the complete elastic properties of inorganic crystalline compounds
journal, March 2015
- de Jong, Maarten; Chen, Wei; Angsten, Thomas
- Scientific Data, Vol. 2, Issue 1
Data Mined Ionic Substitutions for the Discovery of New Compounds
journal, January 2011
- Hautier, Geoffroy; Fischer, Chris; Ehrlacher, Virginie
- Inorganic Chemistry, Vol. 50, Issue 2
Charting the complete elastic properties of inorganic crystalline compounds
journal, March 2015
- de Jong, Maarten; Chen, Wei; Angsten, Thomas
- Scientific Data, Vol. 2, Issue 1
Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error
journal, October 2017
- Faber, Felix A.; Hutchison, Luke; Huang, Bing
- Journal of Chemical Theory and Computation, Vol. 13, Issue 11
Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials
journal, March 2015
- Thompson, A. P.; Swiler, L. P.; Trott, C. R.
- Journal of Computational Physics, Vol. 285
Deep neural networks for accurate predictions of crystal stability
journal, September 2018
- Ye, Weike; Chen, Chi; Wang, Zhenbin
- Nature Communications, Vol. 9, Issue 1
Machine Learning Directed Search for Ultraincompressible, Superhard Materials
journal, July 2018
- Mansouri Tehrani, Aria; Oliynyk, Anton O.; Parry, Marcus
- Journal of the American Chemical Society, Vol. 140, Issue 31
Resolution of the Band Gap Prediction Problem for Materials Design
journal, March 2016
- Crowley, Jason M.; Tahir-Kheli, Jamil; Goddard, William A.
- The Journal of Physical Chemistry Letters, Vol. 7, Issue 7
Machine learning in materials informatics: recent applications and prospects
journal, December 2017
- Ramprasad, Rampi; Batra, Rohit; Pilania, Ghanshyam
- npj Computational Materials, Vol. 3, Issue 1
SchNet – A deep learning architecture for molecules and materials
journal, June 2018
- Schütt, K. T.; Sauceda, H. E.; Kindermans, P. -J.
- The Journal of Chemical Physics, Vol. 148, Issue 24
Low Data Drug Discovery with One-Shot Learning
journal, April 2017
- Altae-Tran, Han; Ramsundar, Bharath; Pappu, Aneesh S.
- ACS Central Science, Vol. 3, Issue 4
Gated Graph Sequence Neural Networks
preprint, January 2015
- Li, Yujia; Tarlow, Daniel; Brockschmidt, Marc
- arXiv
High-Throughput Bubble Screening Method for Combinatorial Discovery of Electrocatalysts for Water Splitting
journal, January 2014
- Xiang, Chengxiang; Suram, Santosh K.; Haber, Joel A.
- ACS Combinatorial Science, Vol. 16, Issue 2
Discovery of Intermetallic Compounds from Traditional to Machine-Learning Approaches
journal, December 2017
- Oliynyk, Anton O.; Mar, Arthur
- Accounts of Chemical Research, Vol. 51, Issue 1
Mining Unexplored Chemistries for Phosphors for High-Color-Quality White-Light-Emitting Diodes
journal, May 2018
- Wang, Zhenbin; Ha, Jungmin; Kim, Yoon Hwa
- Joule, Vol. 2, Issue 5
Molecular graph convolutions: moving beyond fingerprints
journal, August 2016
- Kearnes, Steven; McCloskey, Kevin; Berndl, Marc
- Journal of Computer-Aided Molecular Design, Vol. 30, Issue 8
Molecular graph convolutions: moving beyond fingerprints
journal, August 2016
- Kearnes, Steven; McCloskey, Kevin; Berndl, Marc
- Journal of Computer-Aided Molecular Design, Vol. 30, Issue 8
The Materials Application Programming Interface (API): A simple, flexible and efficient API for materials data based on REpresentational State Transfer (REST) principles
journal, February 2015
- Ong, Shyue Ping; Cholia, Shreyas; Jain, Anubhav
- Computational Materials Science, Vol. 97
Accelerating materials property predictions using machine learning
journal, September 2013
- Pilania, Ghanshyam; Wang, Chenchen; Jiang, Xun
- Scientific Reports, Vol. 3, Issue 1
Convolutional Embedding of Attributed Molecular Graphs for Physical Property Prediction
journal, July 2017
- Coley, Connor W.; Barzilay, Regina; Green, William H.
- Journal of Chemical Information and Modeling, Vol. 57, Issue 8
The Open Quantum Materials Database (OQMD): assessing the accuracy of DFT formation energies
journal, December 2015
- Kirklin, Scott; Saal, James E.; Meredig, Bryce
- npj Computational Materials, Vol. 1, Issue 1
Multilayer feedforward networks are universal approximators
journal, January 1989
- Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert
- Neural Networks, Vol. 2, Issue 5
Machine Learning in Materials Science
book, January 2016
- Mueller, Tim; Kusne, Aaron Gilad; Ramprasad, Rampi
- Reviews in Computational Chemistry, Vol. 29
Data-Driven Learning of Total and Local Energies in Elemental Boron
journal, April 2018
- Deringer, Volker L.; Pickard, Chris J.; Csányi, Gábor
- Physical Review Letters, Vol. 120, Issue 15
Machine learning unifies the modeling of materials and molecules
journal, December 2017
- Bartók, Albert P.; De, Sandip; Poelking, Carl
- Science Advances, Vol. 3, Issue 12
Data-Driven Learning of Total and Local Energies in Elemental Boron
journal, April 2018
- Deringer, Volker L.; Pickard, Chris J.; Csányi, Gábor
- Physical Review Letters, Vol. 120, Issue 15
Quantum chemistry structures and properties of 134 kilo molecules
text, January 2014
- Raghunathan, Ramakrishnan,; O., Dral, Pavlo; Matthias, Rupp,
- Springer Nature
Commentary: The Materials Project: A materials genome approach to accelerating materials innovation
journal, July 2013
- Jain, Anubhav; Ong, Shyue Ping; Hautier, Geoffroy
- APL Materials, Vol. 1, Issue 1
The Materials Application Programming Interface (API): A simple, flexible and efficient API for materials data based on REpresentational State Transfer (REST) principles
journal, February 2015
- Ong, Shyue Ping; Cholia, Shreyas; Jain, Anubhav
- Computational Materials Science, Vol. 97
Materials Design and Discovery with High-Throughput Density Functional Theory: The Open Quantum Materials Database (OQMD)
journal, September 2013
- Saal, James E.; Kirklin, Scott; Aykol, Muratahan
- JOM, Vol. 65, Issue 11
Resolution of the Band Gap Prediction Problem for Materials Design
journal, March 2016
- Crowley, Jason M.; Tahir-Kheli, Jamil; Goddard, William A.
- The Journal of Physical Chemistry Letters, Vol. 7, Issue 7
Quantum-chemical insights from deep tensor neural networks
journal, January 2017
- Schütt, Kristof T.; Arbabzadah, Farhad; Chmiela, Stefan
- Nature Communications, Vol. 8, Issue 1
AFLOW: An automatic framework for high-throughput materials discovery
journal, June 2012
- Curtarolo, Stefano; Setyawan, Wahyu; Hart, Gus L. W.
- Computational Materials Science, Vol. 58
A high-throughput infrastructure for density functional theory calculations
journal, June 2011
- Jain, Anubhav; Hautier, Geoffroy; Moore, Charles J.
- Computational Materials Science, Vol. 50, Issue 8
Hierarchical visualization of materials space with graph convolutional neural networks
journal, November 2018
- Xie, Tian; Grossman, Jeffrey C.
- The Journal of Chemical Physics, Vol. 149, Issue 17
Neural Message Passing with Edge Updates for Predicting Properties of Molecules and Materials
preprint, January 2018
- Jørgensen, Peter Bjørn; Jacobsen, Karsten Wedel; Schmidt, Mikkel N.
- arXiv
Quantum-chemical insights from deep tensor neural networks
journal, January 2017
- Schütt, Kristof T.; Arbabzadah, Farhad; Chmiela, Stefan
- Nature Communications, Vol. 8, Issue 1
Charting the complete elastic properties of inorganic crystalline compounds
dataset, January 2022
- de Jong, Maarten; Chen, Wei; Angsten, Thomas
- Materials Data Facility
A general-purpose machine learning framework for predicting properties of inorganic materials
journal, August 2016
- Ward, Logan; Agrawal, Ankit; Choudhary, Alok
- npj Computational Materials, Vol. 2, Issue 1
Universal fragment descriptors for predicting properties of inorganic crystals
journal, June 2017
- Isayev, Olexandr; Oses, Corey; Toher, Cormac
- Nature Communications, Vol. 8, Issue 1
Quantum chemistry structures and properties of 134 kilo molecules
journal, August 2014
- Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias
- Scientific Data, Vol. 1, Issue 1
Finding Nature’s Missing Ternary Oxide Compounds Using Machine Learning and Density Functional Theory
journal, June 2010
- Hautier, Geoffroy; Fischer, Christopher C.; Jain, Anubhav
- Chemistry of Materials, Vol. 22, Issue 12
Machine learning for molecular and materials science
journal, July 2018
- Butler, Keith T.; Davies, Daniel W.; Cartwright, Hugh
- Nature, Vol. 559, Issue 7715
Structure maps for. Pseudobinary and ternary phases
journal, August 1988
- Pettifor, D. G.
- Materials Science and Technology, Vol. 4, Issue 8
Low Data Drug Discovery with One-Shot Learning
journal, April 2017
- Altae-Tran, Han; Ramsundar, Bharath; Pappu, Aneesh S.
- ACS Central Science, Vol. 3, Issue 4
Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis
journal, February 2013
- Ong, Shyue Ping; Richards, William Davidson; Jain, Anubhav
- Computational Materials Science, Vol. 68
Deep Residual Learning for Image Recognition
conference, June 2016
- He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Works referencing / citing this record:
Machine learning for interatomic potential models
journal, February 2020
- Mueller, Tim; Hernandez, Alberto; Wang, Chuhong
- The Journal of Chemical Physics, Vol. 152, Issue 5
A Critical Review of Machine Learning of Energy Materials
journal, January 2020
- Chen, Chi; Zuo, Yunxing; Ye, Weike
- Advanced Energy Materials, Vol. 10, Issue 8
Machine learning for the modeling of interfaces in energy storage and conversion materials
journal, July 2019
- Artrith, Nongnuch
- Journal of Physics: Energy, Vol. 1, Issue 3
Representations and descriptors unifying the study of molecular and bulk systems
journal, December 2019
- Rossi, Kevin; Cumby, James
- International Journal of Quantum Chemistry, Vol. 120, Issue 8
Recent advances and applications of machine learning in solid-state materials science
journal, August 2019
- Schmidt, Jonathan; Marques, Mário R. G.; Botti, Silvana
- npj Computational Materials, Vol. 5, Issue 1