Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics
Abstract
Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that aremore »
- Authors:
-
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center of Nonlinear Studies, Los Alamos National Laboratory, Los Alamos, NM 87545
- Computer, Computational and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, NM 87545
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center of Nonlinear Studies, Los Alamos National Laboratory, Los Alamos, NM 87545, Center for Integrated Nanotechnologies, Los Alamos National Laboratory, Los Alamos, NM 87545
- Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center for Integrated Nanotechnologies, Los Alamos National Laboratory, Los Alamos, NM 87545
- Publication Date:
- Research Org.:
- Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
- Sponsoring Org.:
- USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)
- OSTI Identifier:
- 1874630
- Alternate Identifier(s):
- OSTI ID: 1875159; OSTI ID: 1903543
- Report Number(s):
- LA-UR-22-31577
Journal ID: ISSN 0027-8424; e2120333119
- Grant/Contract Number:
- LDRD; 89233218CNA000001; FWP: LANLE3F2
- Resource Type:
- Published Article
- Journal Name:
- Proceedings of the National Academy of Sciences of the United States of America
- Additional Journal Information:
- Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 119 Journal Issue: 27; Journal ID: ISSN 0027-8424
- Publisher:
- Proceedings of the National Academy of Sciences
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; machine learning; quantum chemistry; Hamiltonian; semiempirical quantum chemistry; model transferability
Citation Formats
Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, and Nebgen, Benjamin. Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics. United States: N. p., 2022.
Web. doi:10.1073/pnas.2120333119.
Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, & Nebgen, Benjamin. Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics. United States. https://doi.org/10.1073/pnas.2120333119
Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, and Nebgen, Benjamin. Tue .
"Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics". United States. https://doi.org/10.1073/pnas.2120333119.
@article{osti_1874630,
title = {Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics},
author = {Zhou, Guoqing and Lubbers, Nicholas and Barros, Kipton and Tretiak, Sergei and Nebgen, Benjamin},
abstractNote = {Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.},
doi = {10.1073/pnas.2120333119},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 27,
volume = 119,
place = {United States},
year = {Tue Jul 05 00:00:00 EDT 2022},
month = {Tue Jul 05 00:00:00 EDT 2022}
}
https://doi.org/10.1073/pnas.2120333119
Works referenced in this record:
970 Million Druglike Small Molecules for Virtual Screening in the Chemical Universe Database GDB-13
journal, July 2009
- Blum, Lorenz C.; Reymond, Jean-Louis
- Journal of the American Chemical Society, Vol. 131, Issue 25
Predicting phosphorescence energies and inferring wavefunction localization with machine learning
journal, January 2021
- Sifain, Andrew E.; Lystrom, Levi; Messerly, Richard A.
- Chemical Science, Vol. 12, Issue 30
The ORCA program system: The ORCA program system
journal, June 2011
- Neese, Frank
- Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 2, Issue 1
Semiempirical Quantum-Chemical Orthogonalization-Corrected Methods: Theory, Implementation, and Parameters
journal, January 2016
- Dral, Pavlo O.; Wu, Xin; Spörkel, Lasse
- Journal of Chemical Theory and Computation, Vol. 12, Issue 3
A full coupled‐cluster singles and doubles model: The inclusion of disconnected triples
journal, February 1982
- Purvis, George D.; Bartlett, Rodney J.
- The Journal of Chemical Physics, Vol. 76, Issue 4
Machine Learning for Electronically Excited States of Molecules
journal, November 2020
- Westermayr, Julia; Marquetand, Philipp
- Chemical Reviews
Virtual Exploration of the Small-Molecule Chemical Universe below 160 Daltons
journal, February 2005
- Fink, Tobias; Bruggesser, Heinz; Reymond, Jean-Louis
- Angewandte Chemie International Edition, Vol. 44, Issue 10
Advanced Corrections of Hydrogen Bonding and Dispersion for Semiempirical Quantum Mechanical Methods
journal, December 2011
- Řezáč, Jan; Hobza, Pavel
- Journal of Chemical Theory and Computation, Vol. 8, Issue 1
Less is more: Sampling chemical space with active learning
journal, June 2018
- Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas
- The Journal of Chemical Physics, Vol. 148, Issue 24
Machine learning of molecular electronic properties in chemical compound space
journal, September 2013
- Montavon, Grégoire; Rupp, Matthias; Gobre, Vivekanand
- New Journal of Physics, Vol. 15, Issue 9
TorchANI: A Free and Open Source PyTorch-Based Deep Learning Implementation of the ANI Neural Network Potentials
journal, June 2020
- Gao, Xiang; Ramezanghorbani, Farhad; Isayev, Olexandr
- Journal of Chemical Information and Modeling, Vol. 60, Issue 7
A Density Functional Tight Binding Layer for Deep Learning of Chemical Hamiltonians
journal, October 2018
- Li, Haichen; Collins, Christopher; Tanha, Matteus
- Journal of Chemical Theory and Computation, Vol. 14, Issue 11
SchNet – A deep learning architecture for molecules and materials
journal, June 2018
- Schütt, K. T.; Sauceda, H. E.; Kindermans, P. -J.
- The Journal of Chemical Physics, Vol. 148, Issue 24
Nonadiabatic Excited-State Dynamics with Machine Learning
journal, September 2018
- Dral, Pavlo O.; Barbatti, Mario; Thiel, Walter
- The Journal of Physical Chemistry Letters, Vol. 9, Issue 19
Virtual Exploration of the Chemical Universe up to 11 Atoms of C, N, O, F: Assembly of 26.4 Million Structures (110.9 Million Stereoisomers) and Analysis for New Ring Systems, Stereochemistry, Physicochemical Properties, Compound Classes, and Drug Discovery
journal, January 2007
- Fink, Tobias; Reymond, Jean-Louis
- Journal of Chemical Information and Modeling, Vol. 47, Issue 2
Inclusion of Machine Learning Kernel Ridge Regression Potential Energy Surfaces in On-the-Fly Nonadiabatic Molecular Dynamics Simulation
journal, May 2018
- Hu, Deping; Xie, Yu; Li, Xusong
- The Journal of Physical Chemistry Letters, Vol. 9, Issue 11
OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features
journal, September 2020
- Qiao, Zhuoran; Welborn, Matthew; Anandkumar, Animashree
- The Journal of Chemical Physics, Vol. 153, Issue 12
SchNetPack: A Deep Learning Toolbox For Atomistic Systems
journal, November 2018
- Schütt, K. T.; Kessel, P.; Gastegger, M.
- Journal of Chemical Theory and Computation, Vol. 15, Issue 1
Development and use of quantum mechanical molecular models. 76. AM1: a new general purpose quantum mechanical molecular model
journal, June 1985
- Dewar, Michael J. S.; Zoebisch, Eve G.; Healy, Eamonn F.
- Journal of the American Chemical Society, Vol. 107, Issue 13
Nobel Lecture: Electronic structure of matter—wave functions and density functionals
journal, October 1999
- Kohn, W.
- Reviews of Modern Physics, Vol. 71, Issue 5
PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges
journal, April 2019
- Unke, Oliver T.; Meuwly, Markus
- Journal of Chemical Theory and Computation, Vol. 15, Issue 6
Coupled-cluster theory in quantum chemistry
journal, February 2007
- Bartlett, Rodney J.; Musiał, Monika
- Reviews of Modern Physics, Vol. 79, Issue 1
Machine Learning of Partial Charges Derived from High-Quality Quantum-Mechanical Calculations
journal, February 2018
- Bleiziffer, Patrick; Schaller, Kay; Riniker, Sereina
- Journal of Chemical Information and Modeling, Vol. 58, Issue 3
Hierarchical modeling of molecular energies using a deep neural network
journal, June 2018
- Lubbers, Nicholas; Smith, Justin S.; Barros, Kipton
- The Journal of Chemical Physics, Vol. 148, Issue 24
A semiempirical model for the two-center repulsion integrals in the NDDO approximation
journal, January 1977
- Dewar, Michael J. S.; Thiel, Walter
- Theoretica Chimica Acta, Vol. 46, Issue 2
Self-Consistent Equations Including Exchange and Correlation Effects
journal, November 1965
- Kohn, W.; Sham, L. J.
- Physical Review, Vol. 140, Issue 4A, p. A1133-A1138
Discovering a Transferable Charge Assignment Model Using Machine Learning
journal, July 2018
- Sifain, Andrew E.; Lubbers, Nicholas; Nebgen, Benjamin T.
- The Journal of Physical Chemistry Letters, Vol. 9, Issue 16
SciPy 1.0: fundamental algorithms for scientific computing in Python
journal, February 2020
- Virtanen, Pauli; Gommers, Ralf; Oliphant, Travis E.
- Nature Methods
GFN2-xTB—An Accurate and Broadly Parametrized Self-Consistent Tight-Binding Quantum Chemical Method with Multipole Electrostatics and Density-Dependent Dispersion Contributions
journal, January 2019
- Bannwarth, Christoph; Ehlert, Sebastian; Grimme, Stefan
- Journal of Chemical Theory and Computation, Vol. 15, Issue 3
Optimization of parameters for semiempirical methods II. Applications
journal, March 1989
- Stewart, James J. P.
- Journal of Computational Chemistry, Vol. 10, Issue 2
Optimization of parameters for semiempirical methods I. Method
journal, March 1989
- Stewart, James J. P.
- Journal of Computational Chemistry, Vol. 10, Issue 2
Automated discovery of a robust interatomic potential for aluminum
journal, February 2021
- Smith, Justin S.; Nebgen, Benjamin; Mathew, Nithin
- Nature Communications, Vol. 12, Issue 1
S66: A Well-balanced Database of Benchmark Interaction Energies Relevant to Biomolecular Structures
journal, July 2011
- Řezáč, Jan; Riley, Kevin E.; Hobza, Pavel
- Journal of Chemical Theory and Computation, Vol. 7, Issue 8
Learning excited states from ground states by using an artificial neural network
journal, June 2020
- Kiyohara, Shin; Tsubaki, Masashi; Mizoguchi, Teruyasu
- npj Computational Materials, Vol. 6, Issue 1
Optimization of parameters for semiempirical methods VI: more modifications to the NDDO approximations and re-optimization of parameters
journal, November 2012
- Stewart, James J. P.
- Journal of Molecular Modeling, Vol. 19, Issue 1
Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning
journal, July 2019
- Smith, Justin S.; Nebgen, Benjamin T.; Zubatyuk, Roman
- Nature Communications, Vol. 10, Issue 1
Transferable Dynamic Molecular Charge Assignment Using Deep Neural Networks
journal, July 2018
- Nebgen, Benjamin; Lubbers, Nicholas; Smith, Justin S.
- Journal of Chemical Theory and Computation, Vol. 14, Issue 9
Big Data Meets Quantum Chemistry Approximations: The Δ-Machine Learning Approach
journal, April 2015
- Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias
- Journal of Chemical Theory and Computation, Vol. 11, Issue 5
Machine-Learning-Assisted Accurate Band Gap Predictions of Functionalized MXene
journal, May 2018
- Rajan, Arunkumar Chitteth; Mishra, Avanish; Satsangi, Swanti
- Chemistry of Materials, Vol. 30, Issue 12
Reparametrisation of Force Constants in MOPAC 6.0/7.0 for Better Description of the Activation Barrier of Peptide Bond Rotations
journal, September 1996
- Ludwig, Olaf; Schinke, Heiko; Brandt, Wolfgang
- Journal of Molecular Modeling, Vol. 2, Issue 9
The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules
journal, May 2020
- Smith, Justin S.; Zubatyuk, Roman; Nebgen, Benjamin
- Scientific Data, Vol. 7, Issue 1
Multi-fidelity machine learning models for accurate bandgap predictions of solids
journal, March 2017
- Pilania, G.; Gubernatis, J. E.; Lookman, T.
- Computational Materials Science, Vol. 129
Machine learned Hückel theory: Interfacing physics and deep neural networks
journal, June 2021
- Zubatiuk, Tetiana; Nebgen, Benjamin; Lubbers, Nicholas
- The Journal of Chemical Physics, Vol. 154, Issue 24
Quantum Chemical Models (Nobel Lecture)
journal, July 1999
- Pople, John A.
- Angewandte Chemie International Edition, Vol. 38, Issue 13-14
Some difficulties encountered with AM1 and PM3 calculations
journal, October 1998
- Anh, Nguyên Trong; Frison, Gilles; Solladié-Cavallo, Arlette
- Tetrahedron, Vol. 54, Issue 42
Semiempirical Quantum-Chemical Methods with Orthogonalization and Dispersion Corrections
journal, January 2019
- Dral, Pavlo O.; Wu, Xin; Thiel, Walter
- Journal of Chemical Theory and Computation, Vol. 15, Issue 3
Electronic spectra from TDDFT and machine learning in chemical space
journal, August 2015
- Ramakrishnan, Raghunathan; Hartmann, Mia; Tapavicza, Enrico
- The Journal of Chemical Physics, Vol. 143, Issue 8
Optimization of parameters for semiempirical methods V: Modification of NDDO approximations and application to 70 elements
journal, September 2007
- Stewart, James J. P.
- Journal of Molecular Modeling, Vol. 13, Issue 12
A Survey on Transfer Learning
journal, October 2010
- Pan, Sinno Jialin; Yang, Qiang
- IEEE Transactions on Knowledge and Data Engineering, Vol. 22, Issue 10
Ground states of molecules. 38. The MNDO method. Approximations and parameters
journal, June 1977
- Dewar, Michael J. S.; Thiel, Walter
- Journal of the American Chemical Society, Vol. 99, Issue 15
DrugBank 4.0: shedding new light on drug metabolism
journal, November 2013
- Law, Vivian; Knox, Craig; Djoumbou, Yannick
- Nucleic Acids Research, Vol. 42, Issue D1
NEXMD Software Package for Nonadiabatic Excited State Molecular Dynamics Simulations
journal, July 2020
- Malone, Walter; Nebgen, Benjamin; White, Alexander
- Journal of Chemical Theory and Computation, Vol. 16, Issue 9
Graphics Processing Unit-Accelerated Semiempirical Born Oppenheimer Molecular Dynamics Using PyTorch
journal, July 2020
- Zhou, Guoqing; Nebgen, Ben; Lubbers, Nicholas
- Journal of Chemical Theory and Computation, Vol. 16, Issue 8