Performance and scalability evaluation of the Ceph parallel file system. In: PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop

Conference · Tue Jan 01 00:00:00 EST 2013

DOI:https://doi.org/10.1145/2538542.2538562· OSTI ID:1567635

Wang, Feiyi ^[1]; Nelson, Mark ^[2]; Oral, Sarp ^[1]; Atchley, Scott ^[1]; Weil, Sage ^[2]; Settlemyer, Bradley W. ^[1]; Caldwell, Blake ^[1]; Hill, Jason ^[1]

---

1. Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States)
2. Inktank, Inc., Los Angeles, CA (United States)

Ceph is an emerging open-source parallel distributed file and storage system. By design, Ceph leverages unreliable commodity storage and network hardware, and provides reliability and fault-tolerance via controlled object placement and data replication. This paper presents our file and block I/O performance and scalability evaluation of Ceph for scientific high-performance computing (HPC) environments. Our work makes two unique contributions. First, our evaluation is performed under a realistic setup for a large-scale capability HPC environment using a commercial high-end storage system. Second, our path of investigation, tuning efforts, and findings made direct contributions to Ceph's development and improved code quality, scalability, and performance. These changes should benefit both Ceph and the HPC community at large.

• Research Organization: Oak Ridge National Lab. (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1567635
• Resource Relation: Conference: 8th Parallel Data Storage Workshop, Denver, Colorado, November 17 - 21, 2013
• Country of Publication: United States
• Language: English