TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations
Abstract
The encapsulation and commoditization of electronic structure arise naturally as interoperability, and the use of nontraditional compute resources (e.g., new hardware accelerators, cloud computing) remains important for the computational chemistry community. Here, we present TERACHEM CLOUD, a high-performance computing service (HPCS) that offers on-demand electronic structure calculations on both traditional HPC clusters and cloud-based hardware. The framework is designed using off-the-shelf web technologies and containerization to be extremely scalable and portable. Within the HPCS model, users can quickly develop new methods and algorithms in an interactive environment on their laptop while allowing TERACHEM CLOUD to distribute ab initio calculations across all available resources. This approach greatly increases the accessibility of hardware accelerators such as graphics processing units (GPUs) and flexibility for the development of new methods as additional electronic structure packages are integrated into the framework as alternative backends. Cost-performance analysis indicates that traditional nodes are the most cost-effective long-term solution, but commercial cloud providers offer cutting-edge hardware with competitive rates for short-term large-scale calculations. We demonstrate the power of the TERACHEM CLOUD framework by carrying out several showcase calculations, including the generation of 300,000 density functional theory energy and gradient evaluations on medium-sized organic molecules and reproducing 300 fsmore »
- Authors:
-
[1];
[1];
[1]
- Stanford Univ., CA (United States); SLAC National Accelerator Lab., Menlo Park, CA (United States). Photon Ultrafast Laser Science and Engineering Inst. (PULSE)
- Publication Date:
- Research Org.:
- SLAC National Accelerator Lab., Menlo Park, CA (United States)
- Sponsoring Org.:
- USDOE; National Science Foundation (NSF); US Department of the Navy, Office of Naval Research (ONR)
- OSTI Identifier:
- 1633326
- Grant/Contract Number:
- AC02-76SF00515; N00014-16-1-2557; ACI-1450179
- Resource Type:
- Accepted Manuscript
- Journal Name:
- Journal of Chemical Information and Modeling
- Additional Journal Information:
- Journal Volume: 60; Journal Issue: 4; Journal ID: ISSN 1549-9596
- Publisher:
- American Chemical Society
- Country of Publication:
- United States
- Language:
- English
- Subject:
- 37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CHEMISTRY; Ab initio calculations; Chemical calculations; Electronic structure; Cloud computing; Cluster chemistry
Citation Formats
Seritan, Stefan, Thompson, Keiran, and Martínez, Todd J. TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations. United States: N. p., 2020.
Web. doi:10.1021/acs.jcim.9b01152.
Seritan, Stefan, Thompson, Keiran, & Martínez, Todd J. TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations. United States. https://doi.org/10.1021/acs.jcim.9b01152
Seritan, Stefan, Thompson, Keiran, and Martínez, Todd J. Wed .
"TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations". United States. https://doi.org/10.1021/acs.jcim.9b01152. https://www.osti.gov/servlets/purl/1633326.
@article{osti_1633326,
title = {TeraChem Cloud: A High-Performance Computing Service for Scalable Distributed GPU-Accelerated Electronic Structure Calculations},
author = {Seritan, Stefan and Thompson, Keiran and Martínez, Todd J.},
abstractNote = {The encapsulation and commoditization of electronic structure arise naturally as interoperability, and the use of nontraditional compute resources (e.g., new hardware accelerators, cloud computing) remains important for the computational chemistry community. Here, we present TERACHEM CLOUD, a high-performance computing service (HPCS) that offers on-demand electronic structure calculations on both traditional HPC clusters and cloud-based hardware. The framework is designed using off-the-shelf web technologies and containerization to be extremely scalable and portable. Within the HPCS model, users can quickly develop new methods and algorithms in an interactive environment on their laptop while allowing TERACHEM CLOUD to distribute ab initio calculations across all available resources. This approach greatly increases the accessibility of hardware accelerators such as graphics processing units (GPUs) and flexibility for the development of new methods as additional electronic structure packages are integrated into the framework as alternative backends. Cost-performance analysis indicates that traditional nodes are the most cost-effective long-term solution, but commercial cloud providers offer cutting-edge hardware with competitive rates for short-term large-scale calculations. We demonstrate the power of the TERACHEM CLOUD framework by carrying out several showcase calculations, including the generation of 300,000 density functional theory energy and gradient evaluations on medium-sized organic molecules and reproducing 300 fs of nonadiabatic dynamics on the B800-B850 antenna complex in LH2, with the latter demonstration using over 50 Tesla V100 GPUs in a commercial cloud environment in 8 h for approximately $1250.},
doi = {10.1021/acs.jcim.9b01152},
journal = {Journal of Chemical Information and Modeling},
number = 4,
volume = 60,
place = {United States},
year = {2020},
month = {4}
}
Free Publicly Available Full Text
Publisher's Version of Record
Other availability
Search WorldCat to find libraries that may hold this journal
Cited by: 15 works
Citation information provided by
Web of Science
Web of Science
Save to My Library
You must Sign In or Create an Account in order to save documents to your library.
Works referenced in this record:
An Ab Initio Exciton Model Including Charge-Transfer Excited States
journal, July 2017
- Li, Xin; Parrish, Robert M.; Liu, Fang
- Journal of Chemical Theory and Computation, Vol. 13, Issue 8
970 Million Druglike Small Molecules for Virtual Screening in the Chemical Universe Database GDB-13
journal, July 2009
- Blum, Lorenz C.; Reymond, Jean-Louis
- Journal of the American Chemical Society, Vol. 131, Issue 25
The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2018 update
journal, May 2018
- Afgan, Enis; Baker, Dannon; Batut, Bérénice
- Nucleic Acids Research, Vol. 46, Issue W1
Ab Initio Nonadiabatic Quantum Molecular Dynamics
journal, February 2018
- Curchod, Basile F. E.; Martínez, Todd J.
- Chemical Reviews, Vol. 118, Issue 7
Molpro: a general-purpose quantum chemistry program package: Molpro
journal, July 2011
- Werner, Hans-Joachim; Knowles, Peter J.; Knizia, Gerald
- Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 2, Issue 2
Quantum Chemistry on Graphical Processing Units. 1. Strategies for Two-Electron Integral Evaluation
journal, January 2008
- Ufimtsev, Ivan S.; Martínez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 4, Issue 2
Scaling Predictive Modeling in Drug Development with Cloud Computing
journal, December 2014
- Moghadam, Behrooz Torabi; Alvarsson, Jonathan; Holm, Marcus
- Journal of Chemical Information and Modeling, Vol. 55, Issue 1
Automated Discovery and Refinement of Reactive Molecular Dynamics Pathways
journal, January 2016
- Wang, Lee-Ping; McGibbon, Robert T.; Pande, Vijay S.
- Journal of Chemical Theory and Computation, Vol. 12, Issue 2
Singularity: Scientific containers for mobility of compute
journal, May 2017
- Kurtzer, Gregory M.; Sochat, Vanessa; Bauer, Michael W.
- PLOS ONE, Vol. 12, Issue 5
Cloud Parallel Processing of Tandem Mass Spectrometry Based Proteomics Data
journal, September 2012
- Mohammed, Yassene; Mostovenko, Ekaterina; Henneman, Alex A.
- Journal of Proteome Research, Vol. 11, Issue 10
Fast Parallel Algorithms for Short-Range Molecular Dynamics
journal, March 1995
- Plimpton, Steve
- Journal of Computational Physics, Vol. 117, Issue 1
Quantum and thermal effects in dissociative adsorption: Evaluation of free energy barriers in multidimensional quantum systems
journal, February 1994
- Mills, Greg; Jónsson, Hannes
- Physical Review Letters, Vol. 72, Issue 7
GROMACS: High performance molecular simulations through multi-level parallelism from laptops to supercomputers
journal, September 2015
- Abraham, Mark James; Murtola, Teemu; Schulz, Roland
- SoftwareX, Vol. 1-2
Automating NEURON Simulation Deployment in Cloud Resources
journal, September 2016
- Stockton, David B.; Santamaria, Fidel
- Neuroinformatics, Vol. 15, Issue 1
Charge Transfer and Polarization in Solvated Proteins from Ab Initio Molecular Dynamics
journal, June 2011
- Ufimtsev, Ivan S.; Luehr, Nathan; Martinez, Todd J.
- The Journal of Physical Chemistry Letters, Vol. 2, Issue 14
Ab Initio Quantum Chemistry for Protein Structures
journal, October 2012
- Kulik, Heather J.; Luehr, Nathan; Ufimtsev, Ivan S.
- The Journal of Physical Chemistry B, Vol. 116, Issue 41
Reversible multiple time scale molecular dynamics
journal, August 1992
- Tuckerman, M.; Berne, B. J.; Martyna, G. J.
- The Journal of Chemical Physics, Vol. 97, Issue 3
Cost-Effective Cloud Computing: A Case Study Using the Comparative Genomics Tool, Roundup
journal, January 2010
- Kudtarkar, Parul; DeLuca, Todd F.; Fusaro, Vincent A.
- Evolutionary Bioinformatics, Vol. 6
Optimal cloud use of quartic force fields: The first purely commercial cloud computing based study for rovibrational analysis of SiCH −
journal, August 2015
- Fortenberry, Ryan C.; Thackston, Russell
- International Journal of Quantum Chemistry, Vol. 115, Issue 23
Nuclear Data for Astrophysics Research: A New Online Paradigm
journal, August 2011
- Smith, Michael S.
- Journal of the Korean Physical Society, Vol. 59, Issue 2(3)
A climbing image nudged elastic band method for finding saddle points and minimum energy paths
journal, December 2000
- Henkelman, Graeme; Uberuaga, Blas P.; Jónsson, Hannes
- The Journal of Chemical Physics, Vol. 113, Issue 22, p. 9901-9904
Ab Initio Nonadiabatic Dynamics of Multichromophore Complexes: A Scalable Graphical-Processing-Unit-Accelerated Exciton Framework
journal, July 2014
- Sisto, Aaron; Glowacki, David R.; Martinez, Todd J.
- Accounts of Chemical Research, Vol. 47, Issue 9
Accurate sampling using Langevin dynamics
journal, May 2007
- Bussi, Giovanni; Parrinello, Michele
- Physical Review E, Vol. 75, Issue 5
Discovering chemistry with an ab initio nanoreactor
journal, November 2014
- Wang, Lee-Ping; Titov, Alexey; McGibbon, Robert
- Nature Chemistry, Vol. 6, Issue 12
Borg, Omega, and Kubernetes: Lessons learned from three container-management systems over a decade
journal, January 2016
- Burns, Brendan; Grant, Brian; Oppenheimer, David
- Queue, Vol. 14, Issue 1
Understanding the Performance and Potential of Cloud Computing for Scientific Applications
journal, April 2017
- Sadooghi, Iman; Martin, Jesus Hernandez; Li, Tonglin
- IEEE Transactions on Cloud Computing, Vol. 5, Issue 2
BAGEL: Brilliantly Advanced General Electronic-structure Library: BAGEL
journal, August 2017
- Shiozaki, Toru
- Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 8, Issue 1
Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility
journal, June 2009
- Buyya, Rajkumar; Yeo, Chee Shin; Venugopal, Srikumar
- Future Generation Computer Systems, Vol. 25, Issue 6
Multiple time step integrators in ab initio molecular dynamics
journal, February 2014
- Luehr, Nathan; Markland, Thomas E.; Martínez, Todd J.
- The Journal of Chemical Physics, Vol. 140, Issue 8
Fast and Accurate Modeling of Molecular Atomization Energies with Machine Learning
journal, January 2012
- Rupp, Matthias; Tkatchenko, Alexandre; Müller, Klaus-Robert
- Physical Review Letters, Vol. 108, Issue 5
Crystal structure of an integral membrane light-harvesting complex from photosynthetic bacteria
journal, April 1995
- McDermott, G.; Prince, S. M.; Freer, A. A.
- Nature, Vol. 374, Issue 6522
Studying chemical reactivity in a virtual environment
journal, January 2014
- Haag, Moritz P.; Reiher, Markus
- Faraday Discuss., Vol. 169
Interactive Chemical Reactivity Exploration
journal, September 2014
- Haag, Moritz P.; Vaucher, Alain C.; Bosson, Maël
- ChemPhysChem, Vol. 15, Issue 15
Quantum Chemistry on Graphical Processing Units. 3. Analytical Energy Gradients, Geometry Optimization, and First Principles Molecular Dynamics
journal, August 2009
- Ufimtsev, Ivan S.; Martinez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 5, Issue 10
NeuroManager: a workflow analysis based simulation management engine for computational neuroscience
journal, October 2015
- Stockton, David B.; Santamaria, Fidel
- Frontiers in Neuroinformatics, Vol. 9
Ab Initio Interactive Molecular Dynamics on Graphical Processing Units (GPUs)
journal, September 2015
- Luehr, Nathan; Jin, Alex G. B.; Martínez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 11, Issue 10
Enumeration of 166 Billion Organic Small Molecules in the Chemical Universe Database GDB-17
journal, November 2012
- Ruddigkeit, Lars; van Deursen, Ruud; Blum, Lorenz C.
- Journal of Chemical Information and Modeling, Vol. 52, Issue 11
Multiple-Timestep ab Initio Molecular Dynamics Using an Atomic Basis Set Partitioning
journal, September 2015
- Steele, Ryan P.
- The Journal of Physical Chemistry A, Vol. 119, Issue 50
Ab Initio Molecular Dynamics and Time-Resolved Photoelectron Spectroscopy of Electronically Excited Uracil and Thymine
journal, August 2007
- Hudock, Hanneli R.; Levine, Benjamin G.; Thompson, Alexis L.
- The Journal of Physical Chemistry A, Vol. 111, Issue 34
Cloud computing for comparative genomics
journal, May 2010
- Wall, Dennis P.; Kudtarkar, Parul; Fusaro, Vincent A.
- BMC Bioinformatics, Vol. 11, Issue 1
Biomedical Cloud Computing With Amazon Web Services
journal, August 2011
- Fusaro, Vincent A.; Patil, Prasad; Gafni, Erik
- PLoS Computational Biology, Vol. 7, Issue 8
The performance of low-cost commercial cloud computing as an alternative in computational chemistry
journal, March 2015
- Thackston, Russell; Fortenberry, Ryan C.
- Journal of Computational Chemistry, Vol. 36, Issue 12
Molecular Rift: Virtual Reality for Drug Designers
journal, November 2015
- Norrby, Magnus; Grebner, Christoph; Eriksson, Joakim
- Journal of Chemical Information and Modeling, Vol. 55, Issue 11
Quantum-chemical insights from deep tensor neural networks
journal, January 2017
- Schütt, Kristof T.; Arbabzadah, Farhad; Chmiela, Stefan
- Nature Communications, Vol. 8, Issue 1
Sampling molecular conformations and dynamics in a multiuser virtual reality framework
journal, June 2018
- O’Connor, Michael; Deeks, Helen M.; Dawn, Edward
- Science Advances, Vol. 4, Issue 6
Harnessing cloud computing with Galaxy Cloud
journal, November 2011
- Afgan, Enis; Baker, Dannon; Coraor, Nate
- Nature Biotechnology, Vol. 29, Issue 11
Quantum Chemistry on Graphical Processing Units. 2. Direct Self-Consistent-Field Implementation
journal, March 2009
- Ufimtsev, Ivan S.; Martinez, Todd J.
- Journal of Chemical Theory and Computation, Vol. 5, Issue 4
Improved tangent estimate in the nudged elastic band method for finding minimum energy paths and saddle points
journal, December 2000
- Henkelman, Graeme; Jónsson, Hannes
- The Journal of Chemical Physics, Vol. 113, Issue 22
Low Cost, Scalable Proteomics Data Analysis Using Amazon’s Cloud Computing Services and Open Source Search Algorithms
journal, June 2009
- Halligan, Brian D.; Geiger, Joey F.; Vallejos, Andrew K.
- Journal of Proteome Research, Vol. 8, Issue 6
Quantum chemistry structures and properties of 134 kilo molecules
journal, August 2014
- Ramakrishnan, Raghunathan; Dral, Pavlo O.; Rupp, Matthias
- Scientific Data, Vol. 1, Issue 1
A unified framework for the deployment, exposure and access of HPC applications as services in clouds
journal, August 2013
- Wong, Adam K. L.; Goscinski, Andrzej M.
- Future Generation Computer Systems, Vol. 29, Issue 6
Enabling High-Performance Computing as a Service
journal, October 2012
- AbdelBaky, Moustafa; Parashar, Manish; Kim, Hyunjoo
- Computer, Vol. 45, Issue 10
MOLCAS-a software for multiconfigurational quantum chemistry calculations: MOLCAS
journal, September 2012
- Aquilante, Francesco; Pedersen, Thomas Bondo; Veryazov, Valera
- Wiley Interdisciplinary Reviews: Computational Molecular Science, Vol. 3, Issue 2
The Design, Deployment, and Evaluation of the CORAL Pre-Exascale Systems
conference, November 2018
- Vazhkudai, Sudharshan S.; de Supinski, Bronis R.; Bland, Arthur S.
- SC18: International Conference for High Performance Computing, Networking, Storage and Analysis
LHC Cloud Computing with CernVM
conference, February 2011
- Segal, Ben; Buncic, Predrag; Aguado Sanchez, Carlos
- Proceedings of 13th International Workshop on Advanced Computing and Analysis Techniques in Physics Research — PoS(ACAT2010)
The Evolution of Cloud Computing in ATLAS
journal, December 2015
- Taylor, Ryan P.; Berghaus, Frank; Brasolin, Franco
- Journal of Physics: Conference Series, Vol. 664, Issue 2
Evaluating GPU Passthrough in Xen for High Performance Cloud Computing
conference, May 2014
- Younge, Andrew J.; Walters, John Paul; Crago, Stephen
- 2014 IEEE International Parallel & Distributed Processing Symposium Workshops (IPDPSW)
A comparative study of high-performance computing on the cloud
conference, October 2018
- Marathe, Aniruddha; Harris, Rachel; Lowenthal, David K.
- HPDC'13: The 22nd International Symposium on High-Performance Parallel and Distributed Computing, Proceedings of the 22nd international symposium on High-performance parallel and distributed computing
A system for interactive molecular dynamics simulation
conference, January 2001
- Stone, John E.; Gullingsrud, Justin; Schulten, Klaus
- Proceedings of the 2001 symposium on Interactive 3D graphics - SI3D '01