A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations

Aktulga, Hasan Metin; Afibuzzaman, Md.; Williams, Samuel; Buluc, Aydin; Shao, Meiyue; Yang, Chao; Ng, Esmond G.; Maris, Pieter; Vary, James P.

doi:10.1109/TPDS.2016.2630699

Title: A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations

Full Record
Other Related Research

Abstract

As on-node parallelism increases and the performance gap between the processor and the memory system widens, achieving high performance in large-scale scientific applications requires an architecture-aware design of algorithms and solvers. We focus on the eigenvalue problem arising in nuclear Configuration Interaction (CI) calculations, where a few extreme eigenpairs of a sparse symmetric matrix are needed. Here, we consider a block iterative eigensolver whose main computational kernels are the multiplication of a sparse matrix with multiple vectors (SpMM), and tall-skinny matrix operations. We then present techniques to significantly improve the SpMM and the transpose operation SpMM T by using the compressed sparse blocks (CSB) format. We achieve 3-4× speedup on the requisite operations over good implementations with the commonly used compressed sparse row (CSR) format. We develop a performance model that allows us to correctly estimate the performance of our SpMM kernel implementations, and we identify cache bandwidth as a potential performance bottleneck beyond DRAM. We also analyze and optimize the performance of LOBPCG kernels (inner product and linear combinations on multiple vectors) and show up to 15× speedup over using high performance BLAS libraries for these operations. The resulting high performance LOBPCG solver achieves 1.4× to 1.8× speedup overmore »« less

Authors:

Aktulga, Hasan Metin ^[1]; Afibuzzaman, Md. ^[1]; Williams, Samuel ^[2]; Buluc, Aydin ^[2]; Shao, Meiyue ^[2]; Yang, Chao ^[2]; Ng, Esmond G. ^[2]; Maris, Pieter ^[3]; Vary, James P. ^[3]

Michigan State Univ., East Lansing, MI (United States)
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
Iowa State Univ., Ames, IA (United States). Dept. of Physics and Astronomy

Publication Date:: 2017-06-01

Research Org.:: Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)

Sponsoring Org.:: USDOE Office of Science (SC), Nuclear Physics (NP); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

OSTI Identifier:: 1379875

Grant/Contract Number:: AC02-05CH11231; SC0008485; FG02-87ER40371

Resource Type:: Accepted Manuscript

Journal Name:: IEEE Transactions on Parallel and Distributed Systems

Additional Journal Information:: Journal Volume: 28; Journal Issue: 6; Journal ID: ISSN 1045-9219

Publisher:: IEEE

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; Sparse matrix multiplication; block eigensolver; configuration interaction; extended roofline model; tall-skinny matrices

Citation Formats


                    Aktulga, Hasan Metin, Afibuzzaman, Md., Williams, Samuel, Buluc, Aydin, Shao, Meiyue, Yang, Chao, Ng, Esmond G., Maris, Pieter, and Vary, James P. A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations.  United States: N. p., 2017. 
Web.  https://doi.org/10.1109/TPDS.2016.2630699.

Copy to clipboard


                    Aktulga, Hasan Metin, Afibuzzaman, Md., Williams, Samuel, Buluc, Aydin, Shao, Meiyue, Yang, Chao, Ng, Esmond G., Maris, Pieter, & Vary, James P. A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations.  United States.  https://doi.org/10.1109/TPDS.2016.2630699

Copy to clipboard


                    Aktulga, Hasan Metin, Afibuzzaman, Md., Williams, Samuel, Buluc, Aydin, Shao, Meiyue, Yang, Chao, Ng, Esmond G., Maris, Pieter, and Vary, James P. Thu .  
"A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations".  United States.  https://doi.org/10.1109/TPDS.2016.2630699.  https://www.osti.gov/servlets/purl/1379875.

Copy to clipboard


                    
@article{osti_1379875,

  title        = {A High Performance Block Eigensolver for Nuclear Configuration Interaction Calculations},

  author       = {Aktulga, Hasan Metin and Afibuzzaman, Md. and Williams, Samuel and Buluc, Aydin and Shao, Meiyue and Yang, Chao and Ng, Esmond G. and Maris, Pieter and Vary, James P.},

  abstractNote = {As on-node parallelism increases and the performance gap between the processor and the memory system widens, achieving high performance in large-scale scientific applications requires an architecture-aware design of algorithms and solvers. We focus on the eigenvalue problem arising in nuclear Configuration Interaction (CI) calculations, where a few extreme eigenpairs of a sparse symmetric matrix are needed. Here, we consider a block iterative eigensolver whose main computational kernels are the multiplication of a sparse matrix with multiple vectors (SpMM), and tall-skinny matrix operations. We then present techniques to significantly improve the SpMM and the transpose operation SpMM T by using the compressed sparse blocks (CSB) format. We achieve 3-4× speedup on the requisite operations over good implementations with the commonly used compressed sparse row (CSR) format. We develop a performance model that allows us to correctly estimate the performance of our SpMM kernel implementations, and we identify cache bandwidth as a potential performance bottleneck beyond DRAM. We also analyze and optimize the performance of LOBPCG kernels (inner product and linear combinations on multiple vectors) and show up to 15× speedup over using high performance BLAS libraries for these operations. The resulting high performance LOBPCG solver achieves 1.4× to 1.8× speedup over the existing Lanczos solver on a series of CI computations on high-end multicore architectures (Intel Xeons). We also analyze the performance of our techniques on an Intel Xeon Phi Knights Corner (KNC) processor.},

  doi          = {10.1109/TPDS.2016.2630699},

  journal      = {IEEE Transactions on Parallel and Distributed Systems},

  number       = 6,

  volume       = 28,

  place        = {United States},

  year         = {2017},

  month        = {6}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1109/TPDS.2016.2630699

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 3 works

Citation information provided by
Web of Science

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Optimizing Sparse Matrix-Multiple Vectors Multiplication for Nuclear Configuration Interaction Calculations

Conference Aktulga, Hasan Metin ; Buluc, Aydin ; Williams, Samuel ; ...

Obtaining highly accurate predictions on the properties of light atomic nuclei using the configuration interaction (CI) approach requires computing a few extremal Eigen pairs of the many-body nuclear Hamiltonian matrix. In the Many-body Fermion Dynamics for nuclei (MFDn) code, a block Eigen solver is used for this purpose. Due to the large size of the sparse matrices involved, a significant fraction of the time spent on the Eigen value computations is associated with the multiplication of a sparse matrix (and the transpose of that matrix) with multiple vectors (SpMM and SpMM-T). Existing implementations of SpMM and SpMM-T significantly underperform expectations.more »« less
https://doi.org/10.1109/IPDPS.2014.125

Full Text Available
On the performance and energy efficiency of sparse linear algebra on GPUs

Journal Article Anzt, Hartwig ; Tomov, Stanimire ; Dongarra, Jack - International Journal of High Performance Computing Applications

In this paper we unveil some performance and energy efficiency frontiers for sparse computations on GPU-based supercomputers. We compare the resource efficiency of different sparse matrix–vector products (SpMV) taken from libraries such as cuSPARSE and MAGMA for GPU and Intel’s MKL for multicore CPUs, and develop a GPU sparse matrix–matrix product (SpMM) implementation that handles the simultaneous multiplication of a sparse matrix with a set of vectors in block-wise fashion. While a typical sparse computation such as the SpMV reaches only a fraction of the peak of current GPUs, we show that the SpMM succeeds in exceeding the memory-bound limitationsmore »« less
Cited by 3
https://doi.org/10.1177/1094342016672081
Roofline Analysis in the Intel® Advisor to Deliver Optimized Performance for applications on Intel® Xeon Phi™ Processor

Conference Koskela, Tuomas S. ; Lobet, Mathieu ; Deslippe, Jack ; ...

In this session we show, in two case studies, how the roofline feature of Intel Advisor has been utilized to optimize the performance of kernels of the XGC1 and PICSAR codes in preparation for Intel Knights Landing architecture. The impact of the implemented optimizations and the benefits of using the automatic roofline feature of Intel Advisor to study performance of large applications will be presented. This demonstrates an effective optimization strategy that has enabled these science applications to achieve up to 4.6 times speed-up and prepare for future exascale architectures. # Goal/Relevance of Session The roofline model [1,2] is amore »« less
Full Text Available
Basker: Parallel sparse LU factorization utilizing hierarchical parallelism and data layouts

Journal Article Booth, Joshua Dennis ; Ellingwood, Nathan David ; Thornquist, Heidi K. ; ... - Parallel Computing

Transient simulation in circuit simulation tools, such as SPICE and Xyce, depend on scalable and robust sparse LU factorizations for efficient numerical simulation of circuits and power grids. As the need for simulations of very large circuits grow, the prevalence of multicore architectures enable us to use shared memory parallel algorithms for such simulations. A parallel factorization is a critical component of such shared memory parallel simulations. We develop a parallel sparse factorization algorithm that can solve problems from circuit simulations efficiently, and map well to architectural features. This new factorization algorithm exposes hierarchical parallelism to accommodate irregular structure thatmore »« less
Cited by 2
https://doi.org/10.1016/j.parco.2017.06.003

Full Text Available
SU-G-TeP1-15: Toward a Novel GPU Accelerated Deterministic Solution to the Linear Boltzmann Transport Equation

Journal Article Yang, R ; Fallone, B ; Cross Cancer Institute, Edmonton, AB ; ... - Medical Physics

Purpose: To develop a Graphic Processor Unit (GPU) accelerated deterministic solution to the Linear Boltzmann Transport Equation (LBTE) for accurate dose calculations in radiotherapy (RT). A deterministic solution yields the potential for major speed improvements due to the sparse matrix-vector and vector-vector multiplications and would thus be of benefit to RT. Methods: In order to leverage the massively parallel architecture of GPUs, the first order LBTE was reformulated as a second order self-adjoint equation using the Least Squares Finite Element Method (LSFEM). This produces a symmetric positive-definite matrix which is efficiently solved using a parallelized conjugate gradient (CG) solver. Themore »« less
https://doi.org/10.1118/1.4957005

Similar Records