DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics

Abstract

Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that aremore » simultaneously accurate, transferable, and interpretable.« less

Authors:
ORCiD logo [1];  [2];  [3]; ORCiD logo [4];  [5]
  1. Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center of Nonlinear Studies, Los Alamos National Laboratory, Los Alamos, NM 87545
  2. Computer, Computational and Statistical Sciences Division, Los Alamos National Laboratory, Los Alamos, NM 87545
  3. Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545
  4. Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center of Nonlinear Studies, Los Alamos National Laboratory, Los Alamos, NM 87545, Center for Integrated Nanotechnologies, Los Alamos National Laboratory, Los Alamos, NM 87545
  5. Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, Center for Integrated Nanotechnologies, Los Alamos National Laboratory, Los Alamos, NM 87545
Publication Date:
Research Org.:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE National Nuclear Security Administration (NNSA)
OSTI Identifier:
1874630
Alternate Identifier(s):
OSTI ID: 1875159; OSTI ID: 1903543
Report Number(s):
LA-UR-22-31577
Journal ID: ISSN 0027-8424; e2120333119
Grant/Contract Number:  
LDRD; 89233218CNA000001; FWP: LANLE3F2
Resource Type:
Published Article
Journal Name:
Proceedings of the National Academy of Sciences of the United States of America
Additional Journal Information:
Journal Name: Proceedings of the National Academy of Sciences of the United States of America Journal Volume: 119 Journal Issue: 27; Journal ID: ISSN 0027-8424
Publisher:
Proceedings of the National Academy of Sciences
Country of Publication:
United States
Language:
English
Subject:
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; machine learning; quantum chemistry; Hamiltonian; semiempirical quantum chemistry; model transferability

Citation Formats

Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, and Nebgen, Benjamin. Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics. United States: N. p., 2022. Web. doi:10.1073/pnas.2120333119.
Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, & Nebgen, Benjamin. Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics. United States. https://doi.org/10.1073/pnas.2120333119
Zhou, Guoqing, Lubbers, Nicholas, Barros, Kipton, Tretiak, Sergei, and Nebgen, Benjamin. Tue . "Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics". United States. https://doi.org/10.1073/pnas.2120333119.
@article{osti_1874630,
title = {Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics},
author = {Zhou, Guoqing and Lubbers, Nicholas and Barros, Kipton and Tretiak, Sergei and Nebgen, Benjamin},
abstractNote = {Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.},
doi = {10.1073/pnas.2120333119},
journal = {Proceedings of the National Academy of Sciences of the United States of America},
number = 27,
volume = 119,
place = {United States},
year = {Tue Jul 05 00:00:00 EDT 2022},
month = {Tue Jul 05 00:00:00 EDT 2022}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
https://doi.org/10.1073/pnas.2120333119

Save / Share:

Works referenced in this record:

970 Million Druglike Small Molecules for Virtual Screening in the Chemical Universe Database GDB-13
journal, July 2009

  • Blum, Lorenz C.; Reymond, Jean-Louis
  • Journal of the American Chemical Society, Vol. 131, Issue 25
  • DOI: 10.1021/ja902302h

Predicting phosphorescence energies and inferring wavefunction localization with machine learning
journal, January 2021

  • Sifain, Andrew E.; Lystrom, Levi; Messerly, Richard A.
  • Chemical Science, Vol. 12, Issue 30
  • DOI: 10.1039/D1SC02136B

The ORCA program system: The ORCA program system
journal, June 2011

  • Neese, Frank
  • Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 2, Issue 1
  • DOI: 10.1002/wcms.81

Electronic Structure
book, September 2020


Semiempirical Quantum-Chemical Orthogonalization-Corrected Methods: Theory, Implementation, and Parameters
journal, January 2016

  • Dral, Pavlo O.; Wu, Xin; Spörkel, Lasse
  • Journal of Chemical Theory and Computation, Vol. 12, Issue 3
  • DOI: 10.1021/acs.jctc.5b01046

A full coupled‐cluster singles and doubles model: The inclusion of disconnected triples
journal, February 1982

  • Purvis, George D.; Bartlett, Rodney J.
  • The Journal of Chemical Physics, Vol. 76, Issue 4
  • DOI: 10.1063/1.443164

Machine Learning for Electronically Excited States of Molecules
journal, November 2020


Virtual Exploration of the Small-Molecule Chemical Universe below 160 Daltons
journal, February 2005

  • Fink, Tobias; Bruggesser, Heinz; Reymond, Jean-Louis
  • Angewandte Chemie International Edition, Vol. 44, Issue 10
  • DOI: 10.1002/anie.200462457

Advanced Corrections of Hydrogen Bonding and Dispersion for Semiempirical Quantum Mechanical Methods
journal, December 2011

  • Řezáč, Jan; Hobza, Pavel
  • Journal of Chemical Theory and Computation, Vol. 8, Issue 1
  • DOI: 10.1021/ct200751e

Less is more: Sampling chemical space with active learning
journal, June 2018

  • Smith, Justin S.; Nebgen, Ben; Lubbers, Nicholas
  • The Journal of Chemical Physics, Vol. 148, Issue 24
  • DOI: 10.1063/1.5023802

Machine learning of molecular electronic properties in chemical compound space
journal, September 2013


TorchANI: A Free and Open Source PyTorch-Based Deep Learning Implementation of the ANI Neural Network Potentials
journal, June 2020

  • Gao, Xiang; Ramezanghorbani, Farhad; Isayev, Olexandr
  • Journal of Chemical Information and Modeling, Vol. 60, Issue 7
  • DOI: 10.1021/acs.jcim.0c00451

A Density Functional Tight Binding Layer for Deep Learning of Chemical Hamiltonians
journal, October 2018

  • Li, Haichen; Collins, Christopher; Tanha, Matteus
  • Journal of Chemical Theory and Computation, Vol. 14, Issue 11
  • DOI: 10.1021/acs.jctc.8b00873

SchNet – A deep learning architecture for molecules and materials
journal, June 2018

  • Schütt, K. T.; Sauceda, H. E.; Kindermans, P. -J.
  • The Journal of Chemical Physics, Vol. 148, Issue 24
  • DOI: 10.1063/1.5019779

Nonadiabatic Excited-State Dynamics with Machine Learning
journal, September 2018

  • Dral, Pavlo O.; Barbatti, Mario; Thiel, Walter
  • The Journal of Physical Chemistry Letters, Vol. 9, Issue 19
  • DOI: 10.1021/acs.jpclett.8b02469

Inclusion of Machine Learning Kernel Ridge Regression Potential Energy Surfaces in On-the-Fly Nonadiabatic Molecular Dynamics Simulation
journal, May 2018


OrbNet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features
journal, September 2020

  • Qiao, Zhuoran; Welborn, Matthew; Anandkumar, Animashree
  • The Journal of Chemical Physics, Vol. 153, Issue 12
  • DOI: 10.1063/5.0021955

SchNetPack: A Deep Learning Toolbox For Atomistic Systems
journal, November 2018

  • Schütt, K. T.; Kessel, P.; Gastegger, M.
  • Journal of Chemical Theory and Computation, Vol. 15, Issue 1
  • DOI: 10.1021/acs.jctc.8b00908

Development and use of quantum mechanical molecular models. 76. AM1: a new general purpose quantum mechanical molecular model
journal, June 1985

  • Dewar, Michael J. S.; Zoebisch, Eve G.; Healy, Eamonn F.
  • Journal of the American Chemical Society, Vol. 107, Issue 13
  • DOI: 10.1021/ja00299a024

Nobel Lecture: Electronic structure of matter—wave functions and density functionals
journal, October 1999


PhysNet: A Neural Network for Predicting Energies, Forces, Dipole Moments, and Partial Charges
journal, April 2019

  • Unke, Oliver T.; Meuwly, Markus
  • Journal of Chemical Theory and Computation, Vol. 15, Issue 6
  • DOI: 10.1021/acs.jctc.9b00181

Coupled-cluster theory in quantum chemistry
journal, February 2007


Machine Learning of Partial Charges Derived from High-Quality Quantum-Mechanical Calculations
journal, February 2018

  • Bleiziffer, Patrick; Schaller, Kay; Riniker, Sereina
  • Journal of Chemical Information and Modeling, Vol. 58, Issue 3
  • DOI: 10.1021/acs.jcim.7b00663

Hierarchical modeling of molecular energies using a deep neural network
journal, June 2018

  • Lubbers, Nicholas; Smith, Justin S.; Barros, Kipton
  • The Journal of Chemical Physics, Vol. 148, Issue 24
  • DOI: 10.1063/1.5011181

A semiempirical model for the two-center repulsion integrals in the NDDO approximation
journal, January 1977

  • Dewar, Michael J. S.; Thiel, Walter
  • Theoretica Chimica Acta, Vol. 46, Issue 2
  • DOI: 10.1007/BF00548085

Self-Consistent Equations Including Exchange and Correlation Effects
journal, November 1965


Discovering a Transferable Charge Assignment Model Using Machine Learning
journal, July 2018

  • Sifain, Andrew E.; Lubbers, Nicholas; Nebgen, Benjamin T.
  • The Journal of Physical Chemistry Letters, Vol. 9, Issue 16
  • DOI: 10.1021/acs.jpclett.8b01939

SciPy 1.0: fundamental algorithms for scientific computing in Python
journal, February 2020


GFN2-xTB—An Accurate and Broadly Parametrized Self-Consistent Tight-Binding Quantum Chemical Method with Multipole Electrostatics and Density-Dependent Dispersion Contributions
journal, January 2019

  • Bannwarth, Christoph; Ehlert, Sebastian; Grimme, Stefan
  • Journal of Chemical Theory and Computation, Vol. 15, Issue 3
  • DOI: 10.1021/acs.jctc.8b01176

Optimization of parameters for semiempirical methods II. Applications
journal, March 1989


Optimization of parameters for semiempirical methods I. Method
journal, March 1989


Automated discovery of a robust interatomic potential for aluminum
journal, February 2021


S66: A Well-balanced Database of Benchmark Interaction Energies Relevant to Biomolecular Structures
journal, July 2011

  • Řezáč, Jan; Riley, Kevin E.; Hobza, Pavel
  • Journal of Chemical Theory and Computation, Vol. 7, Issue 8
  • DOI: 10.1021/ct2002946

Learning excited states from ground states by using an artificial neural network
journal, June 2020

  • Kiyohara, Shin; Tsubaki, Masashi; Mizoguchi, Teruyasu
  • npj Computational Materials, Vol. 6, Issue 1
  • DOI: 10.1038/s41524-020-0336-3

Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning
journal, July 2019


Transferable Dynamic Molecular Charge Assignment Using Deep Neural Networks
journal, July 2018

  • Nebgen, Benjamin; Lubbers, Nicholas; Smith, Justin S.
  • Journal of Chemical Theory and Computation, Vol. 14, Issue 9
  • DOI: 10.1021/acs.jctc.8b00524

Big Data Meets Quantum Chemistry Approximations: The Δ-Machine Learning Approach
journal, April 2015

  • Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias
  • Journal of Chemical Theory and Computation, Vol. 11, Issue 5
  • DOI: 10.1021/acs.jctc.5b00099

Machine-Learning-Assisted Accurate Band Gap Predictions of Functionalized MXene
journal, May 2018


Reparametrisation of Force Constants in MOPAC 6.0/7.0 for Better Description of the Activation Barrier of Peptide Bond Rotations
journal, September 1996

  • Ludwig, Olaf; Schinke, Heiko; Brandt, Wolfgang
  • Journal of Molecular Modeling, Vol. 2, Issue 9
  • DOI: 10.1007/s0089460020341

The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules
journal, May 2020


Multi-fidelity machine learning models for accurate bandgap predictions of solids
journal, March 2017


Machine learned Hückel theory: Interfacing physics and deep neural networks
journal, June 2021

  • Zubatiuk, Tetiana; Nebgen, Benjamin; Lubbers, Nicholas
  • The Journal of Chemical Physics, Vol. 154, Issue 24
  • DOI: 10.1063/5.0052857

Quantum Chemical Models (Nobel Lecture)
journal, July 1999


Some difficulties encountered with AM1 and PM3 calculations
journal, October 1998


Semiempirical Quantum-Chemical Methods with Orthogonalization and Dispersion Corrections
journal, January 2019

  • Dral, Pavlo O.; Wu, Xin; Thiel, Walter
  • Journal of Chemical Theory and Computation, Vol. 15, Issue 3
  • DOI: 10.1021/acs.jctc.8b01265

Electronic spectra from TDDFT and machine learning in chemical space
journal, August 2015

  • Ramakrishnan, Raghunathan; Hartmann, Mia; Tapavicza, Enrico
  • The Journal of Chemical Physics, Vol. 143, Issue 8
  • DOI: 10.1063/1.4928757

Optimization of parameters for semiempirical methods V: Modification of NDDO approximations and application to 70 elements
journal, September 2007


A Survey on Transfer Learning
journal, October 2010

  • Pan, Sinno Jialin; Yang, Qiang
  • IEEE Transactions on Knowledge and Data Engineering, Vol. 22, Issue 10
  • DOI: 10.1109/TKDE.2009.191

Ground states of molecules. 38. The MNDO method. Approximations and parameters
journal, June 1977

  • Dewar, Michael J. S.; Thiel, Walter
  • Journal of the American Chemical Society, Vol. 99, Issue 15
  • DOI: 10.1021/ja00457a004

DrugBank 4.0: shedding new light on drug metabolism
journal, November 2013

  • Law, Vivian; Knox, Craig; Djoumbou, Yannick
  • Nucleic Acids Research, Vol. 42, Issue D1
  • DOI: 10.1093/nar/gkt1068

NEXMD Software Package for Nonadiabatic Excited State Molecular Dynamics Simulations
journal, July 2020

  • Malone, Walter; Nebgen, Benjamin; White, Alexander
  • Journal of Chemical Theory and Computation, Vol. 16, Issue 9
  • DOI: 10.1021/acs.jctc.0c00248

Graphics Processing Unit-Accelerated Semiempirical Born Oppenheimer Molecular Dynamics Using PyTorch
journal, July 2020

  • Zhou, Guoqing; Nebgen, Ben; Lubbers, Nicholas
  • Journal of Chemical Theory and Computation, Vol. 16, Issue 8
  • DOI: 10.1021/acs.jctc.0c00243