Understanding the Design Space of Sparse/Dense Multiphase Dataflows for Mapping Graph Neural Networks on Spatial Accelerators
- Georgia Institute of Technology, Atlanta, GA (United States)
- Universidad de Murcia (Spain)
- Univ. Politecnica de Catalunya (Spain)
- Neutroon, Barcelona (Spain)
- Universidad Católica de Murcia (Spain)
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Georgia Inst. of Technology, Atlanta, GA (United States)
Graph Neural Networks (GNNs) have garnered a lot of recent interest because of their success in learning representations from graph-structured data across several critical applications in cloud and HPC. Owing to their unique compute and memory characteristics that come from an interplay between dense and sparse phases of computations, the emergence of reconfigurable dataflow (aka spatial) accelerators offers promise for acceleration by mapping optimized dataflows (i.e., computation order and parallelism) for both phases. The goal of this work is to characterize and understand the design-space of dataflow choices for running GNNs on spatial accelerators in order for the compilers to optimize the dataflow based on the workload. Specifically, we propose a taxonomy to describe all possible choices for mapping the dense and sparse phases of GNNs spatially and temporally over a spatial accelerator, capturing both the intra-phase dataflow and the inter-phase (pipelined) dataflow. Using this taxonomy, we do deep-dives into the cost and benefits of several dataflows and perform case studies on implications of hardware parameters for dataflows and value of flexibility to support pipelined execution.
- Research Organization:
- Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
- DOE Contract Number:
- NA0003525
- OSTI ID:
- 1821960
- Report Number(s):
- SAND2021-11869R; 699905
- Country of Publication:
- United States
- Language:
- English
Similar Records
FPGA Acceleration of GCN in Light of the Symmetry of Graph Adjacency Matrix
Reducing Communication in Graph Neural Network Training