Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends

Ibrahim, Khaled Z.; Epifanovsky, Evgeny; Williams, Samuel W.; Krylov, Anna I.

doi:10.2172/1274416

Title: Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends

Full Record
Other Related Research

Abstract

Coupled-cluster methods provide highly accurate models of molecular structure by explicit numerical calculation of tensors representing the correlation between electrons. These calculations are dominated by a sequence of tensor contractions, motivating the development of numerical libraries for such operations. While based on matrix-matrix multiplication, these libraries are specialized to exploit symmetries in the molecular structure and in electronic interactions, and thus reduce the size of the tensor representation and the complexity of contractions. The resulting algorithms are irregular and their parallelization has been previously achieved via the use of dynamic scheduling or specialized data decompositions. We introduce our efforts to extend the Libtensor framework to work in the distributed memory environment in a scalable and energy efficient manner. We achieve up to 240 speedup compared with the best optimized shared memory implementation. We attain scalability to hundreds of thousands of compute cores on three distributed-memory architectures, (Cray XC30&XC40, BlueGene/Q), and on a heterogeneous GPU-CPU system (Cray XK7). As the bottlenecks shift from being compute-bound DGEMM's to communication-bound collectives as the size of the molecular system scales, we adopt two radically different parallelization approaches for handling load-imbalance. Nevertheless, we preserve a uni ed interface to both programming models to maintain themore »« less

Authors:

Ibrahim, Khaled Z. ^[1]; Epifanovsky, Evgeny ^[2]; Williams, Samuel W. ^[1]; Krylov, Anna I. ^[3]

Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
Q-Chem, Inc., Pleasanton, CA (United States)
Univ. of Southern California, Los Angeles, CA (United States). Dept. of Chemistry

Publication Date:: 2016-07-26

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1274416

Report Number(s):: LBNL-1005853
ir:1005853

DOE Contract Number:: AC02-05CH11231

Resource Type:: Technical Report

Country of Publication:: United States

Language:: English

Subject:: 74 ATOMIC AND MOLECULAR PHYSICS; 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Ibrahim, Khaled Z., Epifanovsky, Evgeny, Williams, Samuel W., and Krylov, Anna I.. Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends.  United States: N. p., 2016. 
        Web.  doi:10.2172/1274416.

Copy to clipboard


                    Ibrahim, Khaled Z., Epifanovsky, Evgeny, Williams, Samuel W., & Krylov, Anna I.. Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends.  United States.  https://doi.org/10.2172/1274416

Copy to clipboard


                    Ibrahim, Khaled Z., Epifanovsky, Evgeny, Williams, Samuel W., and Krylov, Anna I.. 2016.  
        "Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends".  United States.  https://doi.org/10.2172/1274416.  https://www.osti.gov/servlets/purl/1274416.

Copy to clipboard


                    
@article{osti_1274416,

  title        = {Cross-scale Efficient Tensor Contractions for Coupled Cluster Computations Through Multiple Programming Model Backends},

  author       = {Ibrahim, Khaled Z. and Epifanovsky, Evgeny and Williams, Samuel W. and Krylov, Anna I.},

  abstractNote = {Coupled-cluster methods provide highly accurate models of molecular structure by explicit numerical calculation of tensors representing the correlation between electrons. These calculations are dominated by a sequence of tensor contractions, motivating the development of numerical libraries for such operations. While based on matrix-matrix multiplication, these libraries are specialized to exploit symmetries in the molecular structure and in electronic interactions, and thus reduce the size of the tensor representation and the complexity of contractions. The resulting algorithms are irregular and their parallelization has been previously achieved via the use of dynamic scheduling or specialized data decompositions. We introduce our efforts to extend the Libtensor framework to work in the distributed memory environment in a scalable and energy efficient manner. We achieve up to 240 speedup compared with the best optimized shared memory implementation. We attain scalability to hundreds of thousands of compute cores on three distributed-memory architectures, (Cray XC30&XC40, BlueGene/Q), and on a heterogeneous GPU-CPU system (Cray XK7). As the bottlenecks shift from being compute-bound DGEMM's to communication-bound collectives as the size of the molecular system scales, we adopt two radically different parallelization approaches for handling load-imbalance. Nevertheless, we preserve a uni ed interface to both programming models to maintain the productivity of computational quantum chemists.},

  doi          = {10.2172/1274416},

  url          = {https://www.osti.gov/biblio/1274416},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2016},

  month        = {7}

}

Copy to clipboard

Technical Report:

View Technical Report (0.43 MB)

https://doi.org/10.2172/1274416

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Cross-scale efficient tensor contractions for coupled cluster computations through multiple programming model backends

Journal Article Ibrahim, Khaled; Epifanovsky, Evgeny; Williams, Samuel; ... - Journal of Parallel and Distributed Computing

Coupled-cluster methods provide highly accurate models of molecular structure through explicit numerical calculation of tensors representing the correlation between electrons. These calculations are dominated by a sequence of tensor contractions, motivating the development of numerical libraries for such operations. While based on matrix–matrix multiplication, these libraries are specialized to exploit symmetries in the molecular structure and in electronic interactions, and thus reduce the size of the tensor representation and the complexity of contractions. The resulting algorithms are irregular and their parallelization has been previously achieved via the use of dynamic scheduling or specialized data decompositions. We introduce our efforts tomore »« less
Cited by 6
https://doi.org/10.1016/j.jpdc.2017.02.010

Full Text Available
A communication-avoiding 3D algorithm for sparse LU factorization on heterogeneous systems

Journal Article Sao, Piyush; Li, Xiaoye; Vuduc, Richard - Journal of Parallel and Distributed Computing

We propose a new algorithm to improve the strong scalability of right-looking sparse LU factorization on distributed memory systems. Our 3D algorithm for sparse LU uses a three-dimensional MPI process grid, exploits elimination tree parallelism, and trades off increased memory for reduced per-process communication. We also analyze the asymptotic improvements for planar graphs (e.g., those arising from 2D grid or mesh discretizations) and certain non-planar graphs (specifically for 3D grids and meshes). For a planar graph with $$n$$ vertices, our algorithm reduces communication volume asymptotically in $$n$$ by a factor of $$\mathscr{O}$$ $$\Big(\sqrt{log \ n}\Big)$$ and latency by a factormore »« less
Cited by 2
https://doi.org/10.1016/j.jpdc.2019.03.004

Full Text Available
GPU acceleration of the Locally Selfconsistent Multiple Scattering code for first principles calculation of the ground state and statistical physics of materials

Journal Article Eisenbach, Markus; Larkin, Jeff; Lutjens, Justin; ... - Computer Physics Communications

The Locally Self-consistent Multiple Scattering (LSMS) code solves the first principles Density Functional theory Kohn–Sham equation for a wide range of materials with a special focus on metals, alloys and metallic nano-structures. It has traditionally exhibited near perfect scalability on massively parallel high performance computer architectures. In this paper, we present our efforts to exploit GPUs to accelerate the LSMS code to enable first principles calculations of O(100,000) atoms and statistical physics sampling of finite temperature properties. We reimplement the scattering matrix calculation for GPUs with a block matrix inversion algorithm that only uses accelerator memory. Finally, using the Craymore »« less
Cited by 12
https://doi.org/10.1016/j.cpc.2016.07.013

Full Text Available
A brief summary on formalizing parallel tensor distributions redistributions and algorithm derivations.

Technical Report Schatz, Martin; Kolda, Tamara; van de Geijn, Robert

Large-scale datasets in computational chemistry typically require distributed-memory parallel methods to perform a special operation known as tensor contraction. Tensors are multidimensional arrays, and a tensor contraction is akin to matrix multiplication with special types of permutations. Creating an efficient algorithm and optimized im- plementation in this domain is complex, tedious, and error-prone. To address this, we develop a notation to express data distributions so that we can apply use automated methods to find optimized implementations for tensor contractions. We consider the spin-adapted coupled cluster singles and doubles method from computational chemistry and use our methodology to produce an efficientmore »« less
https://doi.org/10.2172/1222973

Full Text Available
GPU acceleration of a petascale application for turbulent mixing at high Schmidt number using OpenMP 4.5

Journal Article Clay, M.; Buaria, D.; Yeung, P.; ... - Computer Physics Communications

This paper reports on the successful implementation of a massively parallel GPU-accelerated algorithm for the direct numerical simulation of turbulent mixing at high Schmidt number. The work stems from a recent development (Comput. Phys. Commun., vol. 219, 2017, 313–328), in which a low-communication algorithm was shown to attain high degrees of scalability on the Cray XE6 architecture when overlapping communication and computation via dedicated communication threads. An even higher level of performance has now been achieved using OpenMP 4.5 on the Cray XK7 architecture, where on each node the 16 integer cores of an AMD Interlagos processor share a singlemore »« less
https://doi.org/10.1016/j.cpc.2018.02.020

Similar Records