Performance and Scalability Evaluation of the Ceph Parallel File System

Conference · Tue Jan 01 00:00:00 EST 2013

DOI:https://doi.org/10.1145/2538542.2538562· OSTI ID:1113158

Wang, Feiyi ^[1]; Nelson, Mark ^[2]; Oral, H Sarp ^[1]; Settlemyer, Bradley W ^[1]; Atchley, Scott ^[1]; Caldwell, Blake A ^[1]; Hill, Jason J ^[1]

---

1. ORNL
2. Inktank Storage, Inc.

Ceph is an open-source and emerging parallel distributed file and storage system technology. By design, Ceph assumes running on unreliable and commodity storage and network hardware and provides reliability and fault-tolerance through controlled object placement and data replication. We evaluated the Ceph technology for scientific high-performance computing (HPC) environments. This paper presents our evaluation methodology, experiments, results and observations from mostly parallel I/O performance and scalability perspectives. Our work made two unique contributions. First, our evaluation is performed under a realistic setup for a large-scale capability HPC environment using a commercial high-end storage system. Second, our path of investigation, tuning efforts, and findings made direct contributions to Ceph's development and improved code quality, scalability, and performance. These changes should also benefit both Ceph and HPC communities at large. Throughout the evaluation, we observed that Ceph still is an evolving technology under fast-paced development and showing great promises.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: DE-AC05-00OR22725
• OSTI ID: 1113158
• Resource Relation: Conference: Parallel Data Storage Workshop, Denvor, CO, USA, 20131118, 20131122
• Country of Publication: United States
• Language: English