Towards Precision-Aware Fault Tolerance Approaches for Mixed-Precision Applications

Conference · Sun Nov 13 04:00:00 EST 2022

DOI:https://doi.org/10.1109/FTXS56515.2022.00010· OSTI ID:1963399

Fang, Bo ^[1]; Hari, Siva ^[2]; Tsai, Timothy ^[2]; Li, Xinyi ^[3]; Gopalakrishnan, Ganesh ^[3]; Laguna, Ignacio ^[4]; Barker, Kevin J. ^[1]; Li, Ang ^[1]

---

1. BATTELLE (PACIFIC NW LAB)
2. NVIDIA
3. University of Utah
4. Lawrence Livermore National Laboratory

Graphics Processing Units (GPUs), the dominantly adopted accelerators in HPC systems, are susceptible to transient hardware fault. New generation of GPUs feature mixed-precision architectures such as NVIDIA Tensor Cores to accelerate matrix multiplications. While being widely adapted, how would they behave under transient hardware faults remain unclear. In this study, we conduct a large-scale fault injection experiments on GEMM kernels implemented with different floating-point data types on the V100 and A100 Tensor Cores, and show distinct error resilience characteristics for the GEMMS with different formats. In the future, we plan to explore this space by building precision-aware floating-point fault tolerance techniques for applications such as DNNs that exercise low-precision computations.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 1963399
• Report Number(s): PNNL-SA-177005
• Country of Publication: United States
• Language: English