skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Large-Scale First-Principles Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code

Abstract

First-Principles Molecular Dynamics (FPMD) is an accurate, atomistic simulation approach that is routinely applied to a variety of areas including solid-state physics, chemistry, biochemistry and nanotechnology. FPMD enables one to perform predictive materials simulations, as no empirical or adjustable parameters are used to describe a given system. Instead, a quantum mechanical description of electrons is obtained by solving the Kohn-Sham equations within a pseudopotential plane-wave formalism. This rigorous first-principles treatment of electronic structure is computationally expensive and limits the size of tractable systems to a few hundred atoms on most currently available parallel computers. Developed specifically for large parallel systems at LLNL's Center for Applied Scientific Computing, the Qbox implementation of the FPMD method shows unprecedented performance and scaling on BlueGene/L.

Authors:
; ; ; ; ; ; ; ; ;
Publication Date:
Research Org.:
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
Sponsoring Org.:
USDOE
OSTI Identifier:
883590
Report Number(s):
UCRL-TR-217981
TRN: US200615%%157
DOE Contract Number:
W-7405-ENG-48
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
36 MATERIALS SCIENCE; 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; 75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND SUPERFLUIDITY; ATOMS; BIOCHEMISTRY; CHEMISTRY; COMPUTERS; ELECTRONIC STRUCTURE; ELECTRONS; IMPLEMENTATION; PERFORMANCE; PHYSICS; SIMULATION

Citation Formats

Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, and Sexton, J. Large-Scale First-Principles Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code. United States: N. p., 2006. Web. doi:10.2172/883590.
Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, & Sexton, J. Large-Scale First-Principles Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code. United States. doi:10.2172/883590.
Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, and Sexton, J. Wed . "Large-Scale First-Principles Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code". United States. doi:10.2172/883590. https://www.osti.gov/servlets/purl/883590.
@article{osti_883590,
title = {Large-Scale First-Principles Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code},
author = {Gygi, F and Draeger, E and de Supinski, B and Yates, R K and Franchetti, F and Kral, S and Lorenz, J and Ueberhueber, C and Gunnels, J A and Sexton, J},
abstractNote = {First-Principles Molecular Dynamics (FPMD) is an accurate, atomistic simulation approach that is routinely applied to a variety of areas including solid-state physics, chemistry, biochemistry and nanotechnology. FPMD enables one to perform predictive materials simulations, as no empirical or adjustable parameters are used to describe a given system. Instead, a quantum mechanical description of electrons is obtained by solving the Kohn-Sham equations within a pseudopotential plane-wave formalism. This rigorous first-principles treatment of electronic structure is computationally expensive and limits the size of tractable systems to a few hundred atoms on most currently available parallel computers. Developed specifically for large parallel systems at LLNL's Center for Applied Scientific Computing, the Qbox implementation of the FPMD method shows unprecedented performance and scaling on BlueGene/L.},
doi = {10.2172/883590},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Jan 04 00:00:00 EST 2006},
month = {Wed Jan 04 00:00:00 EST 2006}
}

Technical Report:

Save / Share:
  • We demonstrate that the Qbox code supports unprecedented large-scale First-Principles Molecular Dynamics (FPMD) applications on the BlueGene/L supercomputer. Qbox is an FPMD implementation specifically designed for large-scale parallel platforms such as BlueGene/L. Strong scaling tests for a Materials Science application show an 86% scaling efficiency between 1024 and 32,768 CPUs. Measurements of performance by means of hardware counters show that 37% of the peak FPU performance can be attained.
  • Traditional algorithms for first-principles molecular dynamics (FPMD) simulations only gain a modest capability increase from current petascale computers, due to their O(N 3) complexity and their heavy use of global communications. To address this issue, we are developing a truly scalable O(N) complexity FPMD algorithm, based on density functional theory (DFT), which avoids global communications. The computational model uses a general nonorthogonal orbital formulation for the DFT energy functional, which requires knowledge of selected elements of the inverse of the associated overlap matrix. We present a scalable algorithm for approximately computing selected entries of the inverse of the overlap matrix,more » based on an approximate inverse technique, by inverting local blocks corresponding to principal submatrices of the global overlap matrix. The new FPMD algorithm exploits sparsity and uses nearest neighbor communication to provide a computational scheme capable of extreme scalability. Accuracy is controlled by the mesh spacing of the finite difference discretization, the size of the localization regions in which the electronic orbitals are confined, and a cutoff beyond which the entries of the overlap matrix can be omitted when computing selected entries of its inverse. We demonstrate the algorithm's excellent parallel scaling for up to O(100K) atoms on O(100K) processors, with a wall-clock time of O(1) minute per molecular dynamics time step.« less
  • Enzymes are complicated solvated systems that typically require many atoms to simulate their function with any degree of accuracy. We have recently developed numerical techniques for large scale First-Principles molecular dynamics simulations and applied them to study the enzymatic reaction catalyzed by acetylcholinesterase. We carried out Density functional theory calculations for a quantum mechanical (QM) sub- system consisting of 612 atoms with an O(N) complexity finite-difference approach. The QM sub-system is embedded inside an external potential field representing the electrostatic effect due to the environment. We obtained finite temperature sampling by First-Principles molecular dynamics for the acylation reaction of acetylcholinemore » catalyzed by acetylcholinesterase. Our calculations shows two energies barriers along the reaction coordinate for the enzyme catalyzed acylation of acetylcholine. In conclusion, the second barrier (8.5 kcal/mole) is rate-limiting for the acylation reaction and in good agreement with experiment.« less
  • We present experimental results for a coordinated scheduling implementation of the Linux operating system. Results were collected on an IBM Blue Gene/L machine at scales up to 16K nodes. Our results indicate coordinated scheduling was able to provide a dramatic improvement in scaling performance for two applications characterized as bulk synchronous parallel programs.