On the Use of GPUs in Realizing Cost-Effective Distributed RAID
- Virginia Polytechnic Institute and State University (Virginia Tech)
- Qatar Foundation
- ORNL
- Queen's University, Belfast
The exponential growth in user and application data entails new means for providing fault tolerance and protection against data loss. High Performance Computing (HPC) storage systems, which are at the forefront of handling the data deluge, typically employ hardware RAID at the backend. However, such solutions are costly, do not ensure end-to-end data integrity, and can become a bottleneck during data reconstruction. In this paper, we design an innovative solution to achieve a flexible, fault-tolerant, and high-performance RAID-6 solution for a parallel file system (PFS). Our system utilizes low-cost, strategically placed GPUs - both on the client and server sides - to accelerate parity computation. In contrast to hardware-based approaches, we provide full control over the size, length and location of a RAID array on a per file basis, end-to-end data integrity checking, and parallelization of RAID array reconstruction. We have deployed our system in conjunction with the widely-used Lustre PFS, and show that our approach is feasible and imposes acceptable overhead.
- Research Organization:
- Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE Laboratory Directed Research and Development (LDRD) Program
- DOE Contract Number:
- DE-AC05-00OR22725
- OSTI ID:
- 1048750
- Resource Relation:
- Conference: Proceedings of the IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 12), Washington, D.C., DC, USA, 20120807, 20120807
- Country of Publication:
- United States
- Language:
- English
Similar Records
ZFS on RBODs - Leveraging RAID Controllers for Metrics and Enclosure Management
Improving the Availability of Supercomputer Job Input Data Using Temporal Replication