Kernel fusion in atomistic spin dynamics simulations on Nvidia GPUs using tensor core

Chen, Hongwei; Chen, Shiyang; Turner, Joshua J.; Feiguin, Adrian

doi:10.1016/j.jocs.2024.102357

Kernel fusion in atomistic spin dynamics simulations on Nvidia GPUs using tensor core

Journal Article · Tue Jun 11 00:00:00 EDT 2024 · Journal of Computational Science

DOI:https://doi.org/10.1016/j.jocs.2024.102357· OSTI ID:2446864

^[1]; Chen, Shiyang ^[2]; Turner, Joshua J. ^[3]; Feiguin, Adrian ^[4]

Northeastern Univ., Boston, MA (United States); Stanford Univ., CA (United States). Stanford Institute for Materials and Energy Sciences (SIMES); SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States). Linac Coherent Light Source (LCLS); SLAC
Rutgers Univ., Piscataway, NJ (United States)
Stanford Univ., CA (United States). Stanford Institute for Materials and Energy Sciences (SIMES); SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States). Linac Coherent Light Source (LCLS)
Northeastern Univ., Boston, MA (United States)

In atomistic spin dynamics simulations, the time cost of constructing the space- and time-displaced pair correlation function in real space increases quadratically as the number of spins N, leading to significant computational effort. The GEMM subroutine can be adopted to accelerate the calculation of the dynamical spin-spin correlation function, but the computational cost of simulating large spin systems (>40000 spins) on CPUs remains expensive. In this work, we perform the simulation on the graphics processing unit (GPU), a hardware solution widely used as an accelerator for scientific computing and deep learning. Here we show that GPUs can accelerate the simulation up to 25-fold compared to multi-core CPUs when using the GEMM subroutine on both. To hide memory latency, we fuse the element-wise operation into the GEMM kernel using CUTLASS that can improve the performance by 26% ~ 33% compared to implementation based on cuBLAS. Furthermore, we perform the on-the-fly calculation in the epilogue of the GEMM subroutine to avoid saving intermediate results on global memory, which makes the large-scale atomistic spin dynamics simulation feasible and affordable.

View Accepted Manuscript (DOE)

Research Organization:: SLAC National Accelerator Laboratory (SLAC), Menlo Park, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Basic Energy Sciences (BES)

Grant/Contract Number:: AC02-76SF00515; SC0022216; AC02-05CH11231

OSTI ID:: 2446864

Journal Information:: Journal of Computational Science, Journal Name: Journal of Computational Science Vol. 81; ISSN 1877-7503

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (34)

X-ray Photon Correlation Spectroscopy Studies of Surfaces and Thin Films Sinha, Sunil K.; Jiang, Zhang; Lurio, Laurence B. Advanced Materials, Vol. 26, Issue 46 https://doi.org/10.1002/adma.201401094	journal	September 2014
Parallelization and implementation of multi-spin Monte Carlo simulation of 2D square Ising model using MPI and C++ Hassani, Dariush; Rafibakhsh, Shahnoosh Journal of Theoretical and Applied Physics, Vol. 12, Issue 3 https://doi.org/10.1007/s40094-018-0301-4	journal	August 2018
Improved CUDA programs for GPU computing of Swendsen–Wang multi-cluster spin flip algorithm: 2D and 3D Ising, Potts, and XY models Komura, Yukihiro; Okabe, Yutaka Computer Physics Communications, Vol. 200 https://doi.org/10.1016/j.cpc.2015.10.003	journal	March 2016
SPILADY: A parallel CPU and GPU code for spin–lattice magnetic molecular dynamics simulations Ma, Pui-Wai; Dudarev, S. L.; Woo, C. H. Computer Physics Communications, Vol. 207 https://doi.org/10.1016/j.cpc.2016.05.017	journal	October 2016
A high-performance implementation of atomistic spin dynamics simulations on x86 CPUs Chen, Hongwei; Zhai, Yujia; Turner, Joshua J. Computer Physics Communications, Vol. 291 https://doi.org/10.1016/j.cpc.2023.108851	journal	October 2023
A GPU-based large-scale Monte Carlo simulation method for systems with long-range interactions Liang, Yihao; Xing, Xiangjun; Li, Yaohang Journal of Computational Physics, Vol. 338 https://doi.org/10.1016/j.jcp.2017.02.069	journal	June 2017
Massively parallel symplectic algorithm for coupled magnetic spin dynamics and molecular dynamics Tranchida, J.; Plimpton, S. J.; Thibaudeau, P. Journal of Computational Physics, Vol. 372 https://doi.org/10.1016/j.jcp.2018.06.042	journal	November 2018
Kokkos: Enabling manycore performance portability through polymorphic memory access patterns Carter Edwards, H.; Trott, Christian R.; Sunderland, Daniel Journal of Parallel and Distributed Computing, Vol. 74, Issue 12 https://doi.org/10.1016/j.jpdc.2014.07.003	journal	December 2014
GPU accelerated Monte Carlo simulations of lattice spin models Weigel, M.; Yavorskii, T. Physics Procedia, Vol. 15 https://doi.org/10.1016/j.phpro.2011.06.006	journal	January 2011
Signatures of a liquid-crystal transition in spin-wave excitations of skyrmions Mohanta, Narayan; Christianson, Andrew D.; Okamoto, Satoshi Communications Physics, Vol. 3, Issue 1 https://doi.org/10.1038/s42005-020-00489-w	journal	December 2020
Monte Carlo Simulation in Statistical Physics Binder, K.; Heermann, D.; Roelofs, Lyle Computers in Physics, Vol. 7, Issue 2 https://doi.org/10.1063/1.4823159	journal	January 1993
The Landau–Lifshitz equation in atomistic models Ellis, M. O. A.; Evans, R. F. L.; Ostler, T. A. Low Temperature Physics, Vol. 41, Issue 9 https://doi.org/10.1063/1.4930971	journal	September 2015
Skyrmion fluctuations at a first-order phase transition boundary Esposito, V.; Zheng, X. Y.; Seaberg, M. H. Applied Physics Letters, Vol. 116, Issue 18 https://doi.org/10.1063/5.0004879	journal	May 2020
The Mpemba effect in spin glasses is a persistent memory effect Baity-Jesi, Marco; Calore, Enrico; Cruz, Andres Proceedings of the National Academy of Sciences, Vol. 116, Issue 31 https://doi.org/10.1073/pnas.1819803116	journal	July 2019
Spin dynamics of the antiferromagnetic Heisenberg model on a kagome bilayer Saha, Preetha; Zhang, Depei; Lee, Seung-Hun Physical Review B, Vol. 103, Issue 22 https://doi.org/10.1103/PhysRevB.103.224402	journal	June 2021
Spin-dynamics study of the dynamic critical behavior of the three-dimensional classical Heisenberg ferromagnet Chen, Kun; Landau, D. P. Physical Review B, Vol. 49, Issue 5 https://doi.org/10.1103/PhysRevB.49.3266	journal	February 1994
Monte Carlo Calculation of the Scaling Equation of State for the Classical Heisenberg Ferromagnet Binder, K.; Müller-Krumbhaar, H. Physical Review B, Vol. 7, Issue 7 https://doi.org/10.1103/PhysRevB.7.3297	journal	April 1973
Spatial correlation functions in three-dimensional Ising spin glasses De Dominicis, Cirano; Giardina, Irene; Marinari, Enzo Physical Review B, Vol. 72, Issue 1 https://doi.org/10.1103/PhysRevB.72.014443	journal	July 2005
Comprehensive study of the dynamics of a classical Kitaev spin liquid Samarakoon, A. M.; Banerjee, A.; Zhang, S. -S. Physical Review B, Vol. 96, Issue 13 https://doi.org/10.1103/PhysRevB.96.134408	journal	October 2017
Classical and quantum spin dynamics of the honeycomb Γ model Samarakoon, Anjana M.; Wachtel, Gideon; Yamaji, Youhei Physical Review B, Vol. 98, Issue 4 https://doi.org/10.1103/PhysRevB.98.045121	journal	July 2018
General method for atomistic spin-lattice dynamics with first-principles accuracy Hellsvik, Johan; Thonig, Danny; Modin, Klas Physical Review B, Vol. 99, Issue 10 https://doi.org/10.1103/PhysRevB.99.104302	journal	March 2019
Spirit : Multifunctional framework for atomistic spin simulations Müller, Gideon P.; Hoffmann, Markus; Dißelkamp, Constantin Physical Review B, Vol. 99, Issue 22 https://doi.org/10.1103/PhysRevB.99.224414	journal	June 2019
Parallelization of the Wolff single-cluster algorithm Kaupužs, J.; Rimšāns, J.; Melnik, R. V. N. Physical Review E, Vol. 81, Issue 2 https://doi.org/10.1103/PhysRevE.81.026701	journal	February 2010
Nanosecond X-Ray Photon Correlation Spectroscopy on Magnetic Skyrmions Seaberg, M. H.; Holladay, B.; Lee, J. C. T. Physical Review Letters, Vol. 119, Issue 6 https://doi.org/10.1103/PhysRevLett.119.067403	journal	August 2017
Dynamical Structure Factor of the Three-Dimensional Quantum Spin Liquid Candidate NaCaNi 2 F 7 Zhang, Shu; Changlani, Hitesh J.; Plumb, Kemp W. Physical Review Letters, Vol. 122, Issue 16 https://doi.org/10.1103/PhysRevLett.122.167203	journal	April 2019
Monte Carlo Simulation with Time Step Quantification in Terms of Langevin Dynamics Nowak, U.; Chantrell, R. W.; Kennedy, E. C. Physical Review Letters, Vol. 84, Issue 1 https://doi.org/10.1103/PhysRevLett.84.163	journal	January 2000
Spontaneous fluctuations in a magnetic Fe/Gd skyrmion lattice Seaberg, M. H.; Holladay, B.; Montoya, S. A. Physical Review Research, Vol. 3, Issue 3 https://doi.org/10.1103/PhysRevResearch.3.033249	journal	September 2021
Resonant inelastic x-ray scattering studies of elementary excitations Ament, Luuk J. P.; van Veenendaal, Michel; Devereaux, Thomas P. Reviews of Modern Physics, Vol. 83, Issue 2 https://doi.org/10.1103/RevModPhys.83.705	journal	June 2011
Kokkos 3: Programming Model Extensions for the Exascale Era Trott, Christian; Lebrun-Grandie, Damien; Arndt, Daniel IEEE Transactions on Parallel and Distributed Systems https://doi.org/10.1109/TPDS.2021.3097283	journal	January 2021
Dynamics of Antiferromagnetic Heisenberg Model at Low Temperatures Takahashi, Minoru Progress of Theoretical Physics Supplement, Vol. 101 https://doi.org/10.1143/PTPS.101.487	journal	February 1990
Understanding the GPU Microarchitecture to Achieve Bare-Metal Performance Tuning Zhang, Xiuxia; Tan, Guangming; Xue, Shuangbai Proceedings of the 22nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming - PPoPP '17 https://doi.org/10.1145/3018743.3018755	conference	January 2017
A snapshot review—Fluctuations in quantum materials: from skyrmions to superconductivity Shen, L.; Seaberg, M.; Blackburn, E. MRS Advances, Vol. 6, Issue 8 https://doi.org/10.1557/s43580-021-00051-y	journal	April 2021
Improved CUDA programs for GPU computing of Swendsen–Wang multi-cluster spin flip algorithm: 2D and 3D Ising, Potts, and XY models Komura, Yukihiro Mendeley https://doi.org/10.17632/7fnfbtsgrn	dataset	January 2019
An introduction to the theory of nuclear neutron scattering in condensed matter Schober, Helmut Journal of Neutron Research, Vol. 17, Issue 3-4 https://doi.org/10.3233/JNR-140016	journal	January 2014

Similar Records

Performance Analysis of Memory Transfers and GEMM Subroutines on NVIDIA Tesla GPU Cluster

Conference · Mon Aug 31 00:00:00 EDT 2009 · OSTI ID:965387

A high-performance implementation of atomistic spin dynamics simulations on x86 CPUs

Journal Article · Tue Jul 11 00:00:00 EDT 2023 · Computer Physics Communications · OSTI ID:2308837

GPU-acceleration of the ELPA2 distributed eigensolver for dense symmetric and hermitian eigenproblems

Journal Article · Wed Dec 30 23:00:00 EST 2020 · Computer Physics Communications · OSTI ID:1773653

Related Subjects

97 MATHEMATICS AND COMPUTING
CUTLASS
GEMM
Kernel fusion
Landau-Lifshitz spin dynamics
classical Monte Carlo

Kernel fusion in atomistic spin dynamics simulations on Nvidia GPUs using tensor core

Citation Formats

References (34)

Similar Records

Related Subjects