Hierarchical modeling of molecular energies using a deep neural network

Lubbers, Nicholas Edward; Smith, Justin Steven; Barros, Kipton Marcos

doi:10.1063/1.5011181

Title: Hierarchical modeling of molecular energies using a deep neural network

Journal Article · Mon Mar 19 00:00:00 EDT 2018 · Journal of Chemical Physics

DOI:https://doi.org/10.1063/1.5011181· OSTI ID:1479929

^[1];

^[2];

^[1]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States); Univ. of Florida, Gainesville, FL (United States)

In this work, we introduce the Hierarchically Interacting Particle Neural Network (HIP-NN) to model molecular properties from datasets of quantum calculations. Inspired by a many-body expansion, HIP-NN decomposes properties, such as energy, as a sum over hierarchical terms. These terms are generated from a neural network—a composition of many nonlinear transformations—acting on a representation of the molecule. HIP-NN achieves the state-of-the-art performance on a dataset of 131k ground state organic molecules and predicts energies with 0.26 kcal/mol mean absolute error. Finally, with minimal tuning, our model is also competitive on a dataset of molecular dynamics trajectories. In addition to enabling accurate energy predictions, the hierarchical structure of HIP-NN helps to identify regions of model uncertainty.

View Accepted Manuscript (DOE)

View Accepted Manuscript (Publisher)

Cite

Export

Save

Research Organization:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE Laboratory Directed Research and Development (LDRD) Program

Grant/Contract Number:: AC52-06NA25396

OSTI ID:: 1479929

Alternate ID(s):: OSTI ID: 1426848

Report Number(s):: LA-UR-17-28890

Journal Information:: Journal of Chemical Physics, Vol. 148, Issue 24; ISSN 0021-9606

Publisher:: American Institute of Physics (AIP)Copyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 184 works

Citation information provided by
Web of Science

References (47)

Molecular graph convolutions: moving beyond fingerprints Kearnes, Steven; McCloskey, Kevin; Berndl, Marc Journal of Computer-Aided Molecular Design, Vol. 30, Issue 8 https://doi.org/10.1007/s10822-016-9938-8	journal	August 2016
Deep learning LeCun, Yann; Bengio, Yoshua; Hinton, Geoffrey Nature, Vol. 521, Issue 7553 https://doi.org/10.1038/nature14539	journal	May 2015
Learning molecular energies using localized graph kernels Ferré, Grégoire; Haut, Terry; Barros, Kipton The Journal of Chemical Physics, Vol. 146, Issue 11 https://doi.org/10.1063/1.4978623	journal	March 2017
Gaussian approximation potentials: A brief tutorial introduction Bartók, Albert P.; Csányi, Gábor International Journal of Quantum Chemistry, Vol. 115, Issue 16 https://doi.org/10.1002/qua.24927	journal	April 2015
The ReaxFF reactive force-field: development, applications and future directions Senftle, Thomas P.; Hong, Sungwook; Islam, Md Mahbubul npj Computational Materials, Vol. 2, Issue 1 https://doi.org/10.1038/npjcompumats.2015.11	journal	March 2016
Efficient and accurate machine-learning interpolation of atomic energies in compositions with many species Artrith, Nongnuch; Urban, Alexander; Ceder, Gerbrand Physical Review B, Vol. 96, Issue 1 https://doi.org/10.1103/physrevb.96.014112	journal	July 2017
Machine learning scheme for fast extraction of chemically interpretable interatomic potentials Dolgirev, Pavel E.; Kruglov, Ivan A.; Oganov, Artem R. AIP Advances, Vol. 6, Issue 8 https://doi.org/10.1063/1.4961886	journal	August 2016
Efficient Parallel Linear Scaling Construction of the Density Matrix for Born–Oppenheimer Molecular Dynamics Mniszewski, S. M.; Cawkwell, M. J.; Wall, M. E. Journal of Chemical Theory and Computation, Vol. 11, Issue 10 https://doi.org/10.1021/acs.jctc.5b00552	journal	September 2015
Density Functional Theory Engel, Eberhard; Dreizler, Reiner M. Theoretical and Mathematical Physics https://doi.org/10.1007/978-3-642-14090-7	book	January 2011
Structural Ensembles of Intrinsically Disordered Proteins Depend Strongly on Force Field: A Comparison to Experiment Rauscher, Sarah; Gapsys, Vytautas; Gajda, Michal J. Journal of Chemical Theory and Computation, Vol. 11, Issue 11 https://doi.org/10.1021/acs.jctc.5b00736	journal	October 2015
Machine Learning Predictions of Molecular Properties: Accurate Many-Body Potentials and Nonlocality in Chemical Space Hansen, Katja; Biegler, Franziska; Ramakrishnan, Raghunathan The Journal of Physical Chemistry Letters, Vol. 6, Issue 12 https://doi.org/10.1021/acs.jpclett.5b00831	journal	June 2015
Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error Faber, Felix A.; Hutchison, Luke; Huang, Bing Journal of Chemical Theory and Computation, Vol. 13, Issue 11 https://doi.org/10.1021/acs.jctc.7b00577	journal	October 2017
The many-body expansion combined with neural networks Yao, Kun; Herr, John E.; Parkhill, John The Journal of Chemical Physics, Vol. 146, Issue 1 https://doi.org/10.1063/1.4973380	journal	January 2017
ReaxFF: A Reactive Force Field for Hydrocarbons van Duin, Adri C. T.; Dasgupta, Siddharth; Lorant, Francois The Journal of Physical Chemistry A, Vol. 105, Issue 41 https://doi.org/10.1021/jp004368u	journal	October 2001
Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17 Ruddigkeit, Lars; van Deursen, Ruud; Blum, Lorenz C. Journal of Chemical Information and Modeling, Vol. 52, Issue 11 https://doi.org/10.1021/ci300415d	journal	November 2012
ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost Smith, J. S.; Isayev, O.; Roitberg, A. E. Chemical Science, Vol. 8, Issue 4 https://doi.org/10.1039/c6sc05720a	journal	January 2017
Constructing high-dimensional neural network potentials: A tutorial review Behler, Jörg International Journal of Quantum Chemistry, Vol. 115, Issue 16 https://doi.org/10.1002/qua.24890	journal	March 2015
Density functional tight binding Elstner, Marcus; Seifert, Gotthard Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 372, Issue 2011 https://doi.org/10.1098/rsta.2012.0483	journal	March 2014
Quantum-chemical insights from deep tensor neural networks Schütt, Kristof T.; Arbabzadah, Farhad; Chmiela, Stefan Nature Communications, Vol. 8, Issue 1 https://doi.org/10.1038/ncomms13890	journal	January 2017
Fourier series of atomic radial distribution functions: A molecular fingerprint for machine learning models of quantum chemical properties von Lilienfeld, O. Anatole; Ramakrishnan, Raghunathan; Rupp, Matthias International Journal of Quantum Chemistry, Vol. 115, Issue 16 https://doi.org/10.1002/qua.24912	journal	April 2015
Many-Body Effects in Intermolecular Forces Elrod, Matthew J.; Saykally, Richard J. Chemical Reviews, Vol. 94, Issue 7 https://doi.org/10.1021/cr00031a010	journal	November 1994
On the Correlation Problem in Atomic and Molecular Systems. Calculation of Wavefunction Components in Ursell‐Type Expansion Using Quantum‐Field Theoretical Methods Čížek, Jiří The Journal of Chemical Physics, Vol. 45, Issue 11 https://doi.org/10.1063/1.1727484	journal	December 1966
Intrinsic Bond Energies from a Bonds-in-Molecules Neural Network Yao, Kun; Herr, John E.; Brown, Seth N. The Journal of Physical Chemistry Letters, Vol. 8, Issue 12 https://doi.org/10.1021/acs.jpclett.7b01072	journal	June 2017
Machine learning of accurate energy-conserving molecular force fields Chmiela, Stefan; Tkatchenko, Alexandre; Sauceda, Huziel E. Science Advances, Vol. 3, Issue 5 https://doi.org/10.1126/sciadv.1603015	journal	May 2017
Quantum chemistry structures and properties of 134 kilo molecules Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias Scientific Data, Vol. 1, Issue 1 https://doi.org/10.1038/sdata.2014.22	journal	August 2014
Active learning of linearly parametrized interatomic potentials Podryabinkin, Evgeny V.; Shapeev, Alexander V. Computational Materials Science, Vol. 140 https://doi.org/10.1016/j.commatsci.2017.08.031	journal	December 2017
Publisher’s Note: On representing chemical environments [Phys. Rev. B 87 , 184115 (2013)] Bartók, Albert P.; Kondor, Risi; Csányi, Gábor Physical Review B, Vol. 87, Issue 21 https://doi.org/10.1103/physrevb.87.219902	journal	June 2013
Machine Learning for Quantum Mechanical Properties of Atoms in Molecules Rupp, Matthias; Ramakrishnan, Raghunathan; von Lilienfeld, O. Anatole The Journal of Physical Chemistry Letters, Vol. 6, Issue 16 https://doi.org/10.1021/acs.jpclett.5b01456	journal	July 2015
Machine learning of accurate energy-conserving molecular force fields Chmiela, Stefan; Tkatchenko, Alexandre; Sauceda, Huziel E. Technische Universität Berlin https://doi.org/10.14279/depositonce-6849	text	January 2017
ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost The University of North Carolina at Chapel Hill University Libraries https://doi.org/10.17615/bhbf-9r93	text	January 2017
Machine Learning Predictions of Molecular Properties: Accurate Many-Body Potentials and Nonlocality in Chemical Space Hansen, Katja; Biegler, Franziska; Ramakrishnan, Raghunathan American Chemical Society https://doi.org/10.5451/unibas-ep43351	text	January 2015
Machine Learning for Quantum Mechanical Properties of Atoms in Molecules Matthias, Rupp,; Raghunathan, Ramakrishnan,; Anatole, von Lilienfeld, O. American Chemical Society https://doi.org/10.5451/unibas-ep53266	text	January 2015
Fourier series of atomic radial distribution functions: A molecular fingerprint for machine learning models of quantum chemical properties Von Lilienfeld, O. Anatole; Ramakrishnan, Raghunathan; Rupp, Matthias Wiley https://doi.org/10.5451/unibas-ep53267	text	January 2015
Prediction Errors of Molecular Machine Learning Models Lower than Hybrid DFT Error Faber, Felix A.; Hutchison, Luke; Huang, Bing American Chemical Society https://doi.org/10.5451/unibas-ep63149	text	January 2017
Machine learning based interatomic potential for amorphous carbon Deringer, Volker; Csanyi, Gabor Apollo - University of Cambridge Repository https://doi.org/10.17863/cam.7577	text	January 2017
Deep Learning Ganatra, Sanket Zenodo https://doi.org/10.5281/zenodo.1189359	text	January 2018
Fast and accurate modeling of molecular atomization energies with machine learning Rupp, Matthias; Tkatchenko, Alexandre; Müller, Klaus-Robert American Physical Society https://doi.org/10.5451/unibas-ep43360	text	January 2012
Quantum chemistry structures and properties of 134 kilo molecules Raghunathan, Ramakrishnan,; O., Dral, Pavlo; Matthias, Rupp, Springer Nature https://doi.org/10.5451/unibas-ep43357	text	January 2014
Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning Rupp, Matthias; Tkatchenko, Alexandre; Müller, Klaus-Robert arXiv https://doi.org/10.48550/arxiv.1109.2618	text	January 2011
Fourier series of atomic radial distribution functions: A molecular fingerprint for machine learning models of quantum chemical properties von Lilienfeld, O. Anatole; Ramakrishnan, Raghunathan; Rupp, Matthias arXiv https://doi.org/10.48550/arxiv.1307.2918	preprint	January 2013
Gaussian Approximation Potentials: a brief tutorial introduction Bartók, Albert P.; Csányi, Gábor arXiv https://doi.org/10.48550/arxiv.1502.01366	preprint	January 2015
Comparing molecules and solids across structural and alchemical space De, Sandip; Bartók, Albert P.; Csányi, Gábor arXiv https://doi.org/10.48550/arxiv.1601.04077	text	January 2016
Quantum-Chemical Insights from Deep Tensor Neural Networks Schütt, Kristof T.; Arbabzadah, Farhad; Chmiela, Stefan arXiv https://doi.org/10.48550/arxiv.1609.08259	text	January 2016
Machine Learning of Accurate Energy-Conserving Molecular Force Fields Chmiela, Stefan; Tkatchenko, Alexandre; Sauceda, Huziel E. arXiv https://doi.org/10.48550/arxiv.1611.04678	text	January 2016
Learning molecular energies using localized graph kernels Ferré, G.; Haut, T.; Barros, K. arXiv https://doi.org/10.48550/arxiv.1612.00193	text	January 2016
IDH mutations may not preclude distant, trans-tentorial spread in gliomas: a case report and review of the literature Hong, Christopher S.; Hsieh, Jason K.; Edwards, Nancy A. World Journal of Surgical Oncology, Vol. 14, Issue 1 https://doi.org/10.1186/s12957-016-0814-8	journal	February 2016
On Combining Different Acceleration Techniques at the Iterative Solution of PDEs by the Method of Collocations and Least Residuals Shapeev, Vasily P.; Vorozhtsov, Evgenii V. Modeling and Analysis of Information Systems, Vol. 24, Issue 1 https://doi.org/10.18255/1818-1015-2017-1-39-63	journal	January 2017

Cited By (23)

Enhanced Deep‐Learning Prediction of Molecular Properties via Augmentation of Bond Topology Cho, Hyeoncheol; Choi, Insung S. ChemMedChem, Vol. 14, Issue 17 https://doi.org/10.1002/cmdc.201900458	journal	August 2019
Band Gap Prediction for Large Organic Crystal Structures with Machine Learning Olsthoorn, Bart; Geilhufe, R. Matthias; Borysov, Stanislav S. Advanced Quantum Technologies, Vol. 2, Issue 7-8 https://doi.org/10.1002/qute.201900023	journal	July 2019
Data-enabled structure–property mappings for lanthanide-activated inorganic scintillators Pilania, G.; Liu, Xiang-Yang; Wang, Zhehui Journal of Materials Science, Vol. 54, Issue 11 https://doi.org/10.1007/s10853-019-03434-7	journal	February 2019
Towards exact molecular dynamics simulations with machine-learned force fields Chmiela, Stefan; Sauceda, Huziel E.; Müller, Klaus-Robert Nature Communications, Vol. 9, Issue 1 https://doi.org/10.1038/s41467-018-06169-2	journal	September 2018
Machine-learned multi-system surrogate models for materials prediction Nyshadham, Chandramouli; Rupp, Matthias; Bekker, Brayden npj Computational Materials, Vol. 5, Issue 1 https://doi.org/10.1038/s41524-019-0189-9	journal	April 2019
Enumeration of de novo inorganic complexes for chemical discovery and machine learning Gugler, Stefan; Janet, Jon Paul; Kulik, Heather J. Molecular Systems Design & Engineering, Vol. 5, Issue 1 https://doi.org/10.1039/c9me00069k	journal	January 2020
Machine learning of molecular properties: Locality and active learning Gubaev, Konstantin; Podryabinkin, Evgeny V.; Shapeev, Alexander V. The Journal of Chemical Physics, Vol. 148, Issue 24 https://doi.org/10.1063/1.5005095	journal	June 2018
Guest Editorial: Special Topic on Data-Enabled Theoretical Chemistry Rupp, Matthias; von Lilienfeld, O. Anatole; Burke, Kieron The Journal of Chemical Physics, Vol. 148, Issue 24 https://doi.org/10.1063/1.5043213	journal	June 2018
Unsupervised machine learning for detection of phase transitions in off-lattice systems. II. Applications Jadrich, R. B.; Lindquist, B. A.; Piñeros, W. D. The Journal of Chemical Physics, Vol. 149, Issue 19 https://doi.org/10.1063/1.5049850	journal	November 2018
Molecular force fields with gradient-domain machine learning: Construction and application to dynamics of small molecules with coupled cluster forces Sauceda, Huziel E.; Chmiela, Stefan; Poltavsky, Igor The Journal of Chemical Physics, Vol. 150, Issue 11 https://doi.org/10.1063/1.5078687	journal	March 2019
Chemical diversity in molecular orbital energy predictions with kernel ridge regression Stuke, Annika; Todorović, Milica; Rupp, Matthias The Journal of Chemical Physics, Vol. 150, Issue 20 https://doi.org/10.1063/1.5086105	journal	May 2019
A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules Cheng, Lixue; Welborn, Matthew; Christensen, Anders S. The Journal of Chemical Physics, Vol. 150, Issue 13 https://doi.org/10.1063/1.5088393	journal	April 2019
Machine-learned electron correlation model based on correlation energy density at complete basis set limit Nudejima, Takuro; Ikabata, Yasuhiro; Seino, Junji The Journal of Chemical Physics, Vol. 151, Issue 2 https://doi.org/10.1063/1.5100165	journal	July 2019
Data-driven material models for atomistic simulation Wood, M. A.; Cusentino, M. A.; Wirth, B. D. Physical Review B, Vol. 99, Issue 18 https://doi.org/10.1103/physrevb.99.184305	journal	May 2019
Machine Learning a General-Purpose Interatomic Potential for Silicon Bartók, Albert P.; Kermode, James; Bernstein, Noam Physical Review X, Vol. 8, Issue 4 https://doi.org/10.1103/physrevx.8.041048	journal	December 2018
Dataset’s chemical diversity limits the generalizability of machine learning predictions Glavatskikh, Marta; Leguy, Jules; Hunault, Gilles Journal of Cheminformatics, Vol. 11, Issue 1 https://doi.org/10.1186/s13321-019-0391-2	journal	November 2019
Machine Learning a General-Purpose Interatomic Potential for Silicon Bartók, Ap; Kermode, J.; Bernstein, N. Apollo - University of Cambridge Repository https://doi.org/10.17863/cam.34909	text	January 2018
Machine-learned multi-system surrogate models for materials prediction Nyshadham, C.; Rupp, M.; Bekker, B. Apollo - University of Cambridge Repository https://doi.org/10.17863/cam.39878	text	January 2019
Machine-learned multi-system surrogate models for materials prediction Nyshadham, Chandramouli; Rupp, Matthias; Bekker, Brayden arXiv https://doi.org/10.48550/arxiv.1809.09203	text	January 2018
Molecular Force Fields with Gradient-Domain Machine Learning: Construction and Application to Dynamics of Small Molecules with Coupled Cluster Forces Sauceda, Huziel E.; Chmiela, Stefan; Poltavsky, Igor arXiv https://doi.org/10.48550/arxiv.1901.06594	text	January 2019
Data-driven Material Models for Atomistic Simulation Wood, Mitchell A.; Cusentino, Mary Alice; Wirth, Brian D. arXiv https://doi.org/10.48550/arxiv.1902.09395	text	January 2019
Fast and Accurate Artificial Neural Network Potential Model for MAPbI₃ Perovskite Materials Chen, Hsin-An; Pao, Chun-Wei ACS Omega, Vol. 4, Issue 6 https://doi.org/10.1021/acsomega.9b00378	journal	June 2019
ANI-1: A data set of 20M off-equilibrium DFT calculations for organic molecules Smith, Justin S.; Isayev, Olexandr; Roitberg, Adrian E. arXiv https://doi.org/10.48550/arxiv.1708.04987	text	January 2017

Similar Records

Transferable Dynamic Molecular Charge Assignment Using Deep Neural Networks

Journal Article · Tue Jul 31 00:00:00 EDT 2018 · Journal of Chemical Theory and Computation · OSTI ID:1479929

Nebgen, Benjamin Tyler; Lubbers, Nicholas Edward; Smith, Justin Steven; +6 more

Bond order predictions using deep neural networks

Journal Article · Sun Feb 14 00:00:00 EST 2021 · Journal of Applied Physics · OSTI ID:1479929

Magedov, Sergey; Koh, Christopher; Malone, Walter; +2 more

Accurate and transferable multitask prediction of chemical properties with an atoms-in-molecules neural network

Journal Article · Fri Aug 09 00:00:00 EDT 2019 · Science Advances · OSTI ID:1479929

Zubatyuk, Roman; Smith, Justin Steven; Leszczynski, Jerzy; +1 more

Related Subjects

97 MATHEMATICS AND COMPUTING
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY

Title: Hierarchical modeling of molecular energies using a deep neural network

Citation Formats

References (47)

Cited By (23)

Similar Records

Related Subjects