Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning

Journal Article · · npj Computational Materials
 [1];  [1];  [1];  [2];  [1];  [1];  [1]
  1. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
  2. NVIDIA Corporation, Santa Clara, CA (United States)
+ Show Author Affiliations
The development of machine learning models has led to an abundance of datasets containing quantum mechanical (QM) calculations for molecular and material systems. However, traditional training methods for machine learning models are unable to leverage the plethora of data available as they require that each dataset be generated using the same QM method. Taking machine learning interatomic potentials (MLIPs) as an example, we show that meta-learning techniques, a recent advancement from the machine learning community, can be used to fit multiple levels of QM theory in the same training process. Meta-learning changes the training procedure to learn a representation that can be easily re-trained to new tasks with small amounts of data. We then demonstrate that meta-learning enables simultaneously training to multiple large organic molecule datasets. As a proof of concept, we examine the performance of a MLIP refit to a small drug-like molecule and show that pre-training potentials to multiple levels of theory with meta-learning improves performance. This difference in performance can be seen both in the reduced error and in the improved smoothness of the potential energy surface produced. We therefore show that meta-learning can utilize existing datasets with inconsistent QM levels of theory to produce models that are better at specializing to new datasets. This opens new routes for creating pre-trained, foundation models for interatomic potentials.
Research Organization:
Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)
Sponsoring Organization:
USDOE; USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Basic Energy Sciences (BES). Chemical Sciences, Geosciences & Biosciences Division (CSGB)
Grant/Contract Number:
89233218CNA000001
OSTI ID:
2403610
Alternate ID(s):
OSTI ID: 2570749
Report Number(s):
LA-UR--23-27568; 10.1038/s41524-024-01339-x; 2057-3960
Journal Information:
npj Computational Materials, Journal Name: npj Computational Materials Journal Issue: 1 Vol. 10; ISSN 2057-3960
Publisher:
Nature Publishing GroupCopyright Statement
Country of Publication:
United States
Language:
English

References (46)

Toward Interpretable Machine Learning Models for Materials Discovery journal October 2019
A survey of deep meta-learning journal April 2021
Pre-trained models: Past, present and future journal January 2021
Machine learning in materials science: From explainable predictions to autonomous design journal June 2021
Machine Learning Force Fields journal March 2021
Combining Machine Learning and Computational Chemistry for Predictive Insights Into Chemical Systems journal July 2021
Computational Discovery of Transition-metal Complexes: From High-throughput Screening to Machine Learning journal July 2021
PubChemQC PM6: Data Sets of 221 Million Molecules with Optimized Molecular Geometries and Electronic Properties journal October 2020
Meta Learning for Low-Resource Molecular Optimization journal March 2021
Linear Atomic Cluster Expansion Force Fields for Organic Molecules: Beyond RMSE journal November 2021
Data-Efficient Machine Learning Potentials from Transfer Learning of Periodic Correlated Electronic Structure Methods: Liquid Water at AFQMC, CCSD, and CCSD(T) Accuracy journal February 2023
The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts journal February 2023
The OPLS [optimized potentials for liquid simulations] potential functions for proteins, energy minimizations for crystals of cyclic peptides and crambin journal March 1988
A new force field for molecular mechanical simulation of nucleic acids and proteins journal February 1984
Quantum-chemical insights from deep tensor neural networks journal January 2017
Universal fragment descriptors for predicting properties of inorganic crystals journal June 2017
Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning journal July 2019
Prediction of organic homolytic bond dissociation enthalpies at near chemical accuracy with sub-second computational cost journal May 2020
E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials journal May 2022
The ANI-1ccx and ANI-1x data sets, coupled-cluster and density functional theory properties for molecules journal May 2020
QM7-X, a comprehensive dataset of quantum-mechanical properties spanning the chemical space of small organic molecules journal February 2021
GEOM, energy-annotated molecular conformations for property prediction and molecular generation journal April 2022
QMugs, quantum mechanical properties of drug-like molecules journal June 2022
Transition1x - a dataset for building generalizable reactive machine learning potentials journal December 2022
MultiXC-QM9: Large dataset of molecular and reaction energies from multi-level quantum chemical methods journal November 2023
ElemNet: Deep Learning the Chemistry of Materials From Only Elemental Composition journal December 2018
Quantum chemistry structures and properties of 134 kilo molecules journal August 2014
ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost journal January 2017
The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics journal January 2018
A machine learning based intramolecular potential for a flexible organic molecule journal January 2020
Transfer learning for chemically accurate interatomic neural network potentials journal January 2023
Atom-centered symmetry functions for constructing high-dimensional neural network potentials journal February 2011
Hierarchical modeling of molecular energies using a deep neural network journal June 2018
Less is more: Sampling chemical space with active learning journal June 2018
Machine learning for interatomic potential models journal February 2020
The ORCA quantum chemistry program package journal June 2020
Training algorithm matters for the performance of neural network potential: A case study of Adam and the Kalman filter optimizers journal November 2021
Gaussian Approximation Potentials: The Accuracy of Quantum Mechanics, without the Electrons journal April 2010
Machine Learning Energies of 2 Million Elpasolite ( A B C 2 D 6 ) Crystals journal September 2016
Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties journal April 2018
SISSO: A compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates journal August 2018
Coupled-cluster theory in quantum chemistry journal February 2007
A Survey on Transfer Learning journal October 2010
Fingerprinting diverse nanoporous materials for optimal hydrogen storage conditions using meta-learning journal July 2021
Machine Learning for Molecular Simulation journal April 2020
Meta-learning prediction of physical and chemical properties of magnetized water and fertilizer based on LSTM journal November 2021

Similar Records

Multi-fidelity learning for interatomic potentials: low-level forces and high-level energies are all you need
Journal Article · Sun Sep 28 20:00:00 EDT 2025 · Machine Learning: Science and Technology · OSTI ID:3002460
Systematic softening in universal machine learning interatomic potentials
Journal Article · Thu Jan 09 19:00:00 EST 2025 · npj Computational Materials · OSTI ID:2528028

Related Subjects

71 CLASSICAL AND QUANTUM MECHANICS
GENERAL PHYSICS