Reliability Lessons Learned From GPU Experience With The Titan Supercomputer at Oak Ridge Leadership Computing Facility

Conference · Sun Nov 01 00:00:00 EDT 2015

DOI:https://doi.org/10.1145/2807591.2807666· OSTI ID:1248768

Tiwari, Devesh ^[1]; Gupta, Saurabh ^[1]; Gallarno, George ^[2]; Rogers II, Jim ^[1]; Maxwell, Don ^[1]

---

1. ORNL
2. Christian Brothers University

The high computational capability of graphics processing units (GPUs) is enabling and driving the scientific discovery process at large-scale. The world’s second fastest supercomputer for open science, Titan, has more than 18,000 GPUs that computational scientists use to perform scientific simu- lations and data analysis. Understanding of GPU reliability characteristics, however, is still in its nascent stage since GPUs have only recently been deployed at large-scale. This paper presents a detailed study of GPU errors and their impact on system operations and applications, describing experiences with the 18,688 GPUs on the Titan supercom- puter as well as lessons learned in the process of efficient operation of GPUs at scale. These experiences are helpful to HPC sites which already have large-scale GPU clusters or plan to deploy GPUs in the future.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1248768
• Country of Publication: United States
• Language: English