LargeScale FirstPrinciples Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code
Abstract
FirstPrinciples Molecular Dynamics (FPMD) is an accurate, atomistic simulation approach that is routinely applied to a variety of areas including solidstate physics, chemistry, biochemistry and nanotechnology. FPMD enables one to perform predictive materials simulations, as no empirical or adjustable parameters are used to describe a given system. Instead, a quantum mechanical description of electrons is obtained by solving the KohnSham equations within a pseudopotential planewave formalism. This rigorous firstprinciples treatment of electronic structure is computationally expensive and limits the size of tractable systems to a few hundred atoms on most currently available parallel computers. Developed specifically for large parallel systems at LLNL's Center for Applied Scientific Computing, the Qbox implementation of the FPMD method shows unprecedented performance and scaling on BlueGene/L.
 Authors:
 Publication Date:
 Research Org.:
 Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
 Sponsoring Org.:
 USDOE
 OSTI Identifier:
 883590
 Report Number(s):
 UCRLTR217981
TRN: US200615%%157
 DOE Contract Number:
 W7405ENG48
 Resource Type:
 Technical Report
 Country of Publication:
 United States
 Language:
 English
 Subject:
 36 MATERIALS SCIENCE; 99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; 75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND SUPERFLUIDITY; ATOMS; BIOCHEMISTRY; CHEMISTRY; COMPUTERS; ELECTRONIC STRUCTURE; ELECTRONS; IMPLEMENTATION; PERFORMANCE; PHYSICS; SIMULATION
Citation Formats
Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, and Sexton, J. LargeScale FirstPrinciples Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code. United States: N. p., 2006.
Web. doi:10.2172/883590.
Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, & Sexton, J. LargeScale FirstPrinciples Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code. United States. doi:10.2172/883590.
Gygi, F, Draeger, E, de Supinski, B, Yates, R K, Franchetti, F, Kral, S, Lorenz, J, Ueberhueber, C, Gunnels, J A, and Sexton, J. Wed .
"LargeScale FirstPrinciples Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code". United States.
doi:10.2172/883590. https://www.osti.gov/servlets/purl/883590.
@article{osti_883590,
title = {LargeScale FirstPrinciples Molecular Dynamics Simulations on the BlueGene/L Platform using the Qbox Code},
author = {Gygi, F and Draeger, E and de Supinski, B and Yates, R K and Franchetti, F and Kral, S and Lorenz, J and Ueberhueber, C and Gunnels, J A and Sexton, J},
abstractNote = {FirstPrinciples Molecular Dynamics (FPMD) is an accurate, atomistic simulation approach that is routinely applied to a variety of areas including solidstate physics, chemistry, biochemistry and nanotechnology. FPMD enables one to perform predictive materials simulations, as no empirical or adjustable parameters are used to describe a given system. Instead, a quantum mechanical description of electrons is obtained by solving the KohnSham equations within a pseudopotential planewave formalism. This rigorous firstprinciples treatment of electronic structure is computationally expensive and limits the size of tractable systems to a few hundred atoms on most currently available parallel computers. Developed specifically for large parallel systems at LLNL's Center for Applied Scientific Computing, the Qbox implementation of the FPMD method shows unprecedented performance and scaling on BlueGene/L.},
doi = {10.2172/883590},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Jan 04 00:00:00 EST 2006},
month = {Wed Jan 04 00:00:00 EST 2006}
}

We demonstrate that the Qbox code supports unprecedented largescale FirstPrinciples Molecular Dynamics (FPMD) applications on the BlueGene/L supercomputer. Qbox is an FPMD implementation specifically designed for largescale parallel platforms such as BlueGene/L. Strong scaling tests for a Materials Science application show an 86% scaling efficiency between 1024 and 32,768 CPUs. Measurements of performance by means of hardware counters show that 37% of the peak FPU performance can be attained.

A Scalable O(N) Algorithm for LargeScale Parallel FirstPrinciples Molecular Dynamics Simulations
Traditional algorithms for firstprinciples molecular dynamics (FPMD) simulations only gain a modest capability increase from current petascale computers, due to their O(N ^{3}) complexity and their heavy use of global communications. To address this issue, we are developing a truly scalable O(N) complexity FPMD algorithm, based on density functional theory (DFT), which avoids global communications. The computational model uses a general nonorthogonal orbital formulation for the DFT energy functional, which requires knowledge of selected elements of the inverse of the associated overlap matrix. We present a scalable algorithm for approximately computing selected entries of the inverse of the overlap matrix,more » 
LargeScale FirstPrinciples Molecular Dynamics Simulations with Electrostatic Embedding: Application to Acetylcholinesterase Catalysis
Enzymes are complicated solvated systems that typically require many atoms to simulate their function with any degree of accuracy. We have recently developed numerical techniques for large scale FirstPrinciples molecular dynamics simulations and applied them to study the enzymatic reaction catalyzed by acetylcholinesterase. We carried out Density functional theory calculations for a quantum mechanical (QM) sub system consisting of 612 atoms with an O(N) complexity finitedifference approach. The QM subsystem is embedded inside an external potential field representing the electrostatic effect due to the environment. We obtained finite temperature sampling by FirstPrinciples molecular dynamics for the acylation reaction of acetylcholinemore »Cited by 3 
Linux OS Jitter Measurements at Large Node Counts using a BlueGene/L
We present experimental results for a coordinated scheduling implementation of the Linux operating system. Results were collected on an IBM Blue Gene/L machine at scales up to 16K nodes. Our results indicate coordinated scheduling was able to provide a dramatic improvement in scaling performance for two applications characterized as bulk synchronous parallel programs.