DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method

Abstract

The fragment molecular orbital method in GAMESS is parallelized in a multithreaded OpenMP implementation combined with the MPI version of the two-level generalized distributed data interface. The energy and analytic gradient in gas phase and the polarizable continuum model of solvation are parallelized in this hybrid three-level scheme, achieving a large memory footprint reduction and a high parallel efficiency on Intel Xeon Phi processors. The parallel efficiency is demonstrated on the Stampede2 and Theta supercomputers using up to 2048 nodes (262 144 threads).

Authors:
 [1];  [2];  [3]
  1. Lomonosov Moscow State Univ. (Russian Federation)
  2. Argonne National Lab. (ANL), Argonne, IL (United States)
  3. National Inst. of Advanced Industrial Science and Technology (AIST), Tsukuba (Japan)
Publication Date:
Research Org.:
Argonne National Laboratory (ANL), Argonne, IL (United States)
Sponsoring Org.:
USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC)
OSTI Identifier:
1529983
Grant/Contract Number:  
AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
International Journal of Quantum Chemistry
Additional Journal Information:
Journal Volume: 119; Journal Issue: 12; Journal ID: ISSN 0020-7608
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; DDI; FMO; GAMESS; MPI; OpenMP; parallelization

Citation Formats

Mironov, Vladimir, Alexeev, Yuri, and Fedorov, Dmitri G. Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method. United States: N. p., 2019. Web. doi:10.1002/qua.25937.
Mironov, Vladimir, Alexeev, Yuri, & Fedorov, Dmitri G. Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method. United States. https://doi.org/10.1002/qua.25937
Mironov, Vladimir, Alexeev, Yuri, and Fedorov, Dmitri G. Fri . "Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method". United States. https://doi.org/10.1002/qua.25937. https://www.osti.gov/servlets/purl/1529983.
@article{osti_1529983,
title = {Multithreaded parallelization of the energy and analytic gradient in the fragment molecular orbital method},
author = {Mironov, Vladimir and Alexeev, Yuri and Fedorov, Dmitri G.},
abstractNote = {The fragment molecular orbital method in GAMESS is parallelized in a multithreaded OpenMP implementation combined with the MPI version of the two-level generalized distributed data interface. The energy and analytic gradient in gas phase and the polarizable continuum model of solvation are parallelized in this hybrid three-level scheme, achieving a large memory footprint reduction and a high parallel efficiency on Intel Xeon Phi processors. The parallel efficiency is demonstrated on the Stampede2 and Theta supercomputers using up to 2048 nodes (262 144 threads).},
doi = {10.1002/qua.25937},
journal = {International Journal of Quantum Chemistry},
number = 12,
volume = 119,
place = {United States},
year = {Fri Apr 26 00:00:00 EDT 2019},
month = {Fri Apr 26 00:00:00 EDT 2019}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 10 works
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Large-Scale Computations in Chemistry: A Bird’s Eye View of a Vibrant Field
journal, April 2015

  • Akimov, Alexey V.; Prezhdo, Oleg V.
  • Chemical Reviews, Vol. 115, Issue 12
  • DOI: 10.1021/cr500524c

Linear scaling electronic structure methods
journal, July 1999


Linear Scaling Density Functional Calculations with Gaussian Orbitals
journal, June 1999

  • Scuseria, Gustavo E.
  • The Journal of Physical Chemistry A, Vol. 103, Issue 25
  • DOI: 10.1021/jp990629s

Linear-Scaling Techniques in Computational Chemistry and Physics: Methods and Applications
book, January 2011

  • Zalesny, Robert; Papadopoulos, Manthos G.; Mezey, Paul G.
  • Challenges and Advances in Computational Chemistry and Physics
  • DOI: 10.1007/978-90-481-2853-2

Density matrix search using direct inversion in the iterative subspace as a linear scaling alternative to diagonalization in electronic structure calculations
journal, October 2003

  • Li, Xiaosong; Millam, John M.; Scuseria, Gustavo E.
  • The Journal of Chemical Physics, Vol. 119, Issue 15
  • DOI: 10.1063/1.1607961

Efficient Calculation of Electronic Structure Using O(N) Density Functional Theory
journal, August 2017

  • Nakata, Ayako; Futamura, Yasunori; Sakurai, Tetsuya
  • Journal of Chemical Theory and Computation, Vol. 13, Issue 9
  • DOI: 10.1021/acs.jctc.7b00385

Hybrid MPI-OpenMP Parallelism in the ONETEP Linear-Scaling Electronic Structure Code: Application to the Delamination of Cellulose Nanofibrils
journal, October 2014

  • Wilkinson, Karl A.; Hine, Nicholas D. M.; Skylaris, Chris-Kriton
  • Journal of Chemical Theory and Computation, Vol. 10, Issue 11
  • DOI: 10.1021/ct500686r

Fragmentation Methods: A Route to Accurate Calculations on Large Systems
journal, August 2011

  • Gordon, Mark S.; Fedorov, Dmitri G.; Pruitt, Spencer R.
  • Chemical Reviews, Vol. 112, Issue 1
  • DOI: 10.1021/cr200093j

Investigation of the interaction between molecules at medium distances
journal, April 1975


Toward a Molecular Orbital Derived Empirical Potential for Liquid Simulations
journal, January 1997

  • Gao, Jiali
  • The Journal of Physical Chemistry B, Vol. 101, Issue 4
  • DOI: 10.1021/jp962833a

Divide and Conquer Hartree−Fock Calculations on Proteins
journal, January 2010

  • He, Xiao; Merz, Kenneth M.
  • Journal of Chemical Theory and Computation, Vol. 6, Issue 2
  • DOI: 10.1021/ct9006635

Calculation of protein–ligand binding affinities based on a fragment quantum mechanical method
journal, January 2015

  • Liu, Jinfeng; Wang, Xianwei; Zhang, John Z. H.
  • RSC Advances, Vol. 5, Issue 129
  • DOI: 10.1039/C5RA20185C

Coupled-Cluster Interaction Energies for 200-Atom Host-Guest Systems
journal, September 2014


Extension of the Effective Fragment Potential Method to Macromolecules
journal, June 2016

  • Gurunathan, Pradeep Kumar; Acharya, Atanu; Ghosh, Debashree
  • The Journal of Physical Chemistry B, Vol. 120, Issue 27
  • DOI: 10.1021/acs.jpcb.6b04166

Ab initio molecular dynamics with intramolecular noncovalent interactions for unsolvated polypeptides
journal, January 2016


Systematic fragmentation of large molecules by annihilation
journal, January 2012

  • Collins, Michael A.
  • Physical Chemistry Chemical Physics, Vol. 14, Issue 21
  • DOI: 10.1039/c2cp23832b

Vibrational infrared and Raman spectra of polypeptides: Fragments-in-fragments within molecular tailoring approach
journal, March 2016

  • Sahu, Nityananda; Gadre, Shridhar R.
  • The Journal of Chemical Physics, Vol. 144, Issue 11
  • DOI: 10.1063/1.4943966

N -body:Many-body QM:QM vibrational frequencies: Application to small hydrogen-bonded clusters
journal, November 2013

  • Howard, J. Coleman; Tschumper, Gregory S.
  • The Journal of Chemical Physics, Vol. 139, Issue 18
  • DOI: 10.1063/1.4829463

Theoretical Study of Protein–Ligand Interactions Using the Molecules-in-Molecules Fragmentation-Based Method
journal, August 2018

  • Thapa, Bishnu; Beckett, Daniel; Erickson, Jon
  • Journal of Chemical Theory and Computation, Vol. 14, Issue 10
  • DOI: 10.1021/acs.jctc.8b00531

Automated error control in divide-and-conquer self-consistent field calculations: Automated Error Control
journal, February 2018

  • Kobayashi, Masato; Fujimori, Toshikazu; Taketsugu, Tetsuya
  • Journal of Computational Chemistry, Vol. 39, Issue 15
  • DOI: 10.1002/jcc.25174

Fragment molecular orbital method: an approximate computational method for large molecules
journal, November 1999


Extending the Power of Quantum Chemistry to Large Systems with the Fragment Molecular Orbital Method
journal, August 2007

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • The Journal of Physical Chemistry A, Vol. 111, Issue 30
  • DOI: 10.1021/jp0716740

Exploring chemistry with the fragment molecular orbital method
journal, January 2012

  • Fedorov, Dmitri G.; Nagata, Takeshi; Kitaura, Kazuo
  • Physical Chemistry Chemical Physics, Vol. 14, Issue 21
  • DOI: 10.1039/c2cp23784a

Electron-correlated fragment-molecular-orbital calculations for biomolecular and nano systems
journal, January 2014

  • Tanaka, Shigenori; Mochizuki, Yuji; Komeiji, Yuto
  • Phys. Chem. Chem. Phys., Vol. 16, Issue 22
  • DOI: 10.1039/C4CP00316K

The fragment molecular orbital method: theoretical development, implementation in GAMESS, and applications: Fragment molecular orbital method in GAMESS
journal, June 2017

  • Fedorov, Dmitri G.
  • Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 7, Issue 6
  • DOI: 10.1002/wcms.1322

The Use of Many-Body Expansions and Geometry Optimizations in Fragment-Based Methods
journal, July 2014

  • Fedorov, Dmitri G.; Asada, Naoya; Nakanishi, Isao
  • Accounts of Chemical Research, Vol. 47, Issue 9
  • DOI: 10.1021/ar500224r

Importance of Three-Body Interactions in Molecular Dynamics Simulations of Water Demonstrated with the Fragment Molecular Orbital Method
journal, March 2016

  • Pruitt, Spencer R.; Nakata, Hiroya; Nagata, Takeshi
  • Journal of Chemical Theory and Computation, Vol. 12, Issue 4
  • DOI: 10.1021/acs.jctc.5b01208

Energy Decomposition Analysis in Solution Based on the Fragment Molecular Orbital Method
journal, December 2011

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • The Journal of Physical Chemistry A, Vol. 116, Issue 1
  • DOI: 10.1021/jp209579w

Subsystem Analysis for the Fragment Molecular Orbital Method and Its Application to Protein–Ligand Binding in Solution
journal, March 2016

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • The Journal of Physical Chemistry A, Vol. 120, Issue 14
  • DOI: 10.1021/acs.jpca.6b00163

Towards good correlation between fragment molecular orbital interaction energies and experimental IC50 for ligand binding: A case study of p38 MAP kinase
journal, January 2018

  • Sheng, Yinglei; Watanabe, Hirofumi; Maruyama, Keiya
  • Computational and Structural Biotechnology Journal, Vol. 16
  • DOI: 10.1016/j.csbj.2018.10.003

Simulations of Chemical Reactions with the Frozen Domain Formulation of the Fragment Molecular Orbital Method
journal, June 2015

  • Nakata, Hiroya; Fedorov, Dmitri G.; Nagata, Takeshi
  • Journal of Chemical Theory and Computation, Vol. 11, Issue 7
  • DOI: 10.1021/acs.jctc.5b00277

The fragment molecular orbital method combined with density-functional tight-binding and the polarizable continuum model
journal, January 2016

  • Nishimoto, Yoshio; Fedorov, Dmitri G.
  • Physical Chemistry Chemical Physics, Vol. 18, Issue 32
  • DOI: 10.1039/C6CP02186G

Development of the Fragment Molecular Orbital Method for Calculating Nonlocal Excitations in Large Molecular Systems
journal, March 2018

  • Fujita, Takatoshi; Mochizuki, Yuji
  • The Journal of Physical Chemistry A, Vol. 122, Issue 15
  • DOI: 10.1021/acs.jpca.8b00446

Heuristic static load-balancing algorithm applied to the fragment molecular orbital method
conference, November 2012

  • Alexeev, Yuri; Mahajan, Ashutosh; Leyffer, Sven
  • 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2012.62

General atomic and molecular electronic structure system
journal, November 1993

  • Schmidt, Michael W.; Baldridge, Kim K.; Boatz, Jerry A.
  • Journal of Computational Chemistry, Vol. 14, Issue 11, p. 1347-1363
  • DOI: 10.1002/jcc.540141112

Advances in electronic structure theory
book, January 2005


A new hierarchical parallelization scheme: Generalized distributed data interface (GDDI), and an application to the fragment molecular orbital method (FMO)
journal, January 2004

  • Fedorov, Dmitri G.; Olson, Ryan M.; Kitaura, Kazuo
  • Journal of Computational Chemistry, Vol. 25, Issue 6
  • DOI: 10.1002/jcc.20018

The importance of three-body terms in the fragment molecular orbital method
journal, April 2004

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • The Journal of Chemical Physics, Vol. 120, Issue 15
  • DOI: 10.1063/1.1687334

Theoretical study of the prion protein based on the fragment molecular orbital method
journal, December 2009

  • Ishikawa, Takeshi; Ishikura, Takakazu; Kuwata, Kazuo
  • Journal of Computational Chemistry, Vol. 30, Issue 16
  • DOI: 10.1002/jcc.21265

Development of the four-body corrected fragment molecular orbital (FMO4) method
journal, January 2012


Large-Scale MP2 Calculations on the Blue Gene Architecture Using the Fragment Molecular Orbital Method
journal, December 2011

  • Fletcher, Graham D.; Fedorov, Dmitri G.; Pruitt, Spencer R.
  • Journal of Chemical Theory and Computation, Vol. 8, Issue 1
  • DOI: 10.1021/ct200548v

The parallel implementation of a full configuration interaction program
journal, July 2003

  • Gan, Zhengting; Alexeev, Yuri; Gordon, Mark S.
  • The Journal of Chemical Physics, Vol. 119, Issue 1
  • DOI: 10.1063/1.1575193

A parallel distributed data CPHF algorithm for analytic Hessians
journal, January 2007

  • Alexeev, Yuri; Schmidt, Michael W.; Windus, Theresa L.
  • Journal of Computational Chemistry, Vol. 28, Issue 10
  • DOI: 10.1002/jcc.20633

An efficient MPI/OpenMP parallelization of the Hartree–Fock–Roothaan method for the first generation of Intel® Xeon Phi™ processor architecture
journal, February 2017

  • Mironov, Vladimir; Moskovsky, Alexander; D’Mello, Michael
  • The International Journal of High Performance Computing Applications, Vol. 33, Issue 1
  • DOI: 10.1177/1094342017732628

MPI/OpenMP Hybrid Parallel Algorithm for Hartree−Fock Calculations
journal, March 2010

  • Ishimura, Kazuya; Kuramoto, Kei; Ikuta, Yasuhiro
  • Journal of Chemical Theory and Computation, Vol. 6, Issue 4
  • DOI: 10.1021/ct100083w

Fully analytic energy gradient in the fragment molecular orbital method
journal, March 2011

  • Brorsen, Kurt; Fedorov, Dmitri G.
  • The Journal of Chemical Physics, Vol. 134, Issue 12
  • DOI: 10.1063/1.3568010

The three-body fragment molecular orbital method for accurate calculations of large systems
journal, December 2006


Fragment molecular orbital method: use of approximate electrostatic potential
journal, January 2002


On the accuracy of the 3-body fragment molecular orbital method (FMO) applied to density functional theory
journal, May 2004


Second order Møller-Plesset perturbation theory based upon the fragment molecular orbital method
journal, January 2004

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • The Journal of Chemical Physics, Vol. 121, Issue 6
  • DOI: 10.1063/1.1769362

Open-Shell Formulation of the Fragment Molecular Orbital Method
journal, December 2009

  • Pruitt, Spencer R.; Fedorov, Dmitri G.; Kitaura, Kazuo
  • Journal of Chemical Theory and Computation, Vol. 6, Issue 1
  • DOI: 10.1021/ct900442b

Unrestricted Hartree-Fock based on the fragment molecular orbital method: Energy and its analytic gradient
journal, July 2012

  • Nakata, Hiroya; Fedorov, Dmitri G.; Nagata, Takeshi
  • The Journal of Chemical Physics, Vol. 137, Issue 4
  • DOI: 10.1063/1.4737860

Pair interaction energy decomposition analysis
journal, January 2006

  • Fedorov, Dmitri G.; Kitaura, Kazuo
  • Journal of Computational Chemistry, Vol. 28, Issue 1
  • DOI: 10.1002/jcc.20496

Power Measurements of Hartree-Fock Algorithms Using Different Storage Devices
conference, May 2017

  • Mironov, Vladimir; Moskovsky, Alexander; Alexeev, Yuri
  • 2017 IEEE International Parallel and Distributed Processing Symposium: Workshops (IPDPSW), 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
  • DOI: 10.1109/IPDPSW.2017.109

PaSTRI: Error-Bounded Lossy Compression for Two-Electron Integrals in Quantum Chemistry
conference, September 2018

  • Gok, Ali Murat; Di, Sheng; Alexeev, Yuri
  • 2018 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2018.00013

Reducing the scaling of the fragment molecular orbital method using the multipole method
journal, August 2012


The polarizable continuum model (PCM) interfaced with the fragment molecular orbital method (FMO)
journal, January 2006

  • Fedorov, Dmitri G.; Kitaura, Kazuo; Li, Hui
  • Journal of Computational Chemistry, Vol. 27, Issue 8
  • DOI: 10.1002/jcc.20406

Quantum Mechanical Continuum Solvation Models
journal, August 2005

  • Tomasi, Jacopo; Mennucci, Benedetta; Cammi, Roberto
  • Chemical Reviews, Vol. 105, Issue 8
  • DOI: 10.1021/cr9904009

Analytic gradient for second order Møller-Plesset perturbation theory with the polarizable continuum model based on the fragment molecular orbital method
journal, May 2012

  • Nagata, Takeshi; Fedorov, Dmitri G.; Li, Hui
  • The Journal of Chemical Physics, Vol. 136, Issue 20
  • DOI: 10.1063/1.4714601

Continuum solvation of large molecules described by QM/MM: a semi-iterative implementation of the PCM/EFP interface
journal, March 2003

  • Li, Hui; Pomelli, Christian S.; Jensen, Jan H.
  • Theoretical Chemistry Accounts: Theory, Computation, and Modeling (Theoretica Chimica Acta), Vol. 109, Issue 2
  • DOI: 10.1007/s00214-002-0427-x

GAMESS As a Free Quantum-Mechanical Platform for Drug Research
journal, September 2012

  • Alexeev, Yuri; P. Mazanetz, Michael; Ichihara, Osamu
  • Current Topics in Medicinal Chemistry, Vol. 12, Issue 18
  • DOI: 10.2174/156802612804910269

The Fragment Molecular Orbital Method Reveals New Insight into the Chemical Nature of GPCR–Ligand Interactions
journal, December 2015

  • Heifetz, Alexander; Chudyk, Ewa I.; Gleave, Laura
  • Journal of Chemical Information and Modeling, Vol. 56, Issue 1
  • DOI: 10.1021/acs.jcim.5b00644

Rapid and accurate assessment of GPCR-ligand interactions Using the fragment molecular orbital-based density-functional tight-binding method
journal, July 2017

  • Morao, Inaki; Fedorov, Dmitri G.; Robinson, Roger
  • Journal of Computational Chemistry, Vol. 38, Issue 23
  • DOI: 10.1002/jcc.24850

Geometry Optimization of the Active Site of a Large System with the Fragment Molecular Orbital Method
journal, January 2011

  • Fedorov, Dmitri G.; Alexeev, Yuri; Kitaura, Kazuo
  • The Journal of Physical Chemistry Letters, Vol. 2, Issue 4
  • DOI: 10.1021/jz1016894

Density-Functional Tight-Binding Combined with the Fragment Molecular Orbital Method
journal, October 2014

  • Nishimoto, Yoshio; Fedorov, Dmitri G.; Irle, Stephan
  • Journal of Chemical Theory and Computation, Vol. 10, Issue 11
  • DOI: 10.1021/ct500489d

Effect of the damping function in dispersion corrected density functional theory
journal, March 2011

  • Grimme, Stefan; Ehrlich, Stephan; Goerigk, Lars
  • Journal of Computational Chemistry, Vol. 32, Issue 7
  • DOI: 10.1002/jcc.21759

Empirical corrections and pair interaction energies in the fragment molecular orbital method
journal, August 2018


The Fragment Molecular Orbital Method for Geometry Optimizations of Polypeptides and Proteins
journal, April 2007

  • Fedorov, Dmitri G.; Ishida, Toyokazu; Uebayasi, Masami
  • The Journal of Physical Chemistry A, Vol. 111, Issue 14
  • DOI: 10.1021/jp0671042

Fragment molecular orbital method: application to molecular dynamics simulation, ‘ab initio FMO-MD’
journal, April 2003


Fragment Molecular Orbital method-based Molecular Dynamics (FMO-MD) as a simulator for chemical reactions in explicit solvation
journal, January 2009

  • Komeiji, Yuto; Ishikawa, Takeshi; Mochizuki, Yuji
  • Journal of Computational Chemistry, Vol. 30, Issue 1
  • DOI: 10.1002/jcc.21025

Ab initio Path Integral Molecular Dynamics Based on Fragment Molecular Orbital Method
journal, October 2009

  • Fujita, Takatoshi; Watanabe, Hirofumi; Tanaka, Shigenori
  • Journal of the Physical Society of Japan, Vol. 78, Issue 10
  • DOI: 10.1143/JPSJ.78.104723

Quantum Mechanical Continuum Solvation Models
journal, October 2005

  • Tomasi, Jacopo; Menucci, Benedetta; Cammi, Roberto
  • ChemInform, Vol. 36, Issue 42
  • DOI: 10.1002/chin.200542292

Investigation of the interaction between molecules at medium distances
journal, January 1977


GAMESS As a Free Quantum-Mechanical Platform for Drug Research
journal, January 2013

  • Alexeev, Yuri; Mazanetz, Michael P.; Ichihara, Osamu
  • Current Topics in Medicinal Chemistry, Vol. 12, Issue 18
  • DOI: 10.2174/1568026611212180008

Efficient calculation of electronic structure using O(N) density functional theory
preprint, January 2017


Empirical corrections and pair interaction energies in the fragment molecular orbital method
text, January 2018