Accelerating HDF5 I/O for Exascale Using DAOS

Soumagne, Jerome; Henderson, Jordan; Chaarawi, Mohamad; Fortner, Neil; Breitenfeld, Scot; Lu, Songyu; Robinson, Dana; Pourmal, Elena; Lombardi, Johann

doi:10.1109/tpds.2021.3097884

Accelerating HDF5 I/O for Exascale Using DAOS

Journal Article · 01 April 2022 · IEEE Transactions on Parallel and Distributed Systems

DOI:https://doi.org/10.1109/tpds.2021.3097884· OSTI ID:1982885

; ; Chaarawi, Mohamad; ; Breitenfeld, Scot; Lu, Songyu; Robinson, Dana; Pourmal, Elena; Lombardi, Johann

Not provided.

Research Organization:: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). National Energy Research Scientific Computing Center (NERSC)

Sponsoring Organization:: USDOE

DOE Contract Number:: AC02-05CH11231; AC02-06CH11357

OSTI ID:: 1982885

Journal Information:: IEEE Transactions on Parallel and Distributed Systems, Journal Name: IEEE Transactions on Parallel and Distributed Systems Journal Issue: 4 Vol. 33; ISSN 1045-9219

Publisher:: IEEE

Country of Publication:: United States

Language:: English

References (15)

An abstract-device interface for implementing portable parallel-I/O interfaces Thakur, R.; Gropp, W.; Lusk, E. Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96) https://doi.org/10.1109/FMPC.1996.558080	conference	January 1996
RADOS: a scalable, reliable storage service for petabyte-scale storage clusters Weil, Sage A.; Leung, Andrew W.; Brandt, Scott A. Proceedings of the 2nd international workshop on Petascale data storage held in conjunction with Supercomputing '07 - PDSW '07 https://doi.org/10.1145/1374596.1374606	conference	January 2007
A Transparent Server-Managed Object Storage System for HPC Mu, Jingqing; Soumagne, Jerome; Tang, Houjun 2018 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2018.00063	conference	September 2018
0.374 Pflop/s trillion-particle kinetic modeling of laser plasma interaction on roadrunner Bowers, K. J.; Albright, B. J.; Bergen, B. 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2008.5222734	conference	November 2008
Parallel I/O, analysis, and visualization of a trillion particle simulation Byna, Surendra; Chou, Jerry; Rubel, Oliver 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2012.92	conference	November 2012
Evaluation of HPC Application I/O on Object Storage Systems Liu, Jialin; Koziol, Quincey; Butler, Gregory F. 2018 IEEE/ACM 3rd International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISCS) https://doi.org/10.1109/PDSW-DISCS.2018.00005	conference	November 2018
Toward Scalable and Asynchronous Object-Centric Data Management for HPC Tang, Houjun; Byna, Suren; Tessier, Francois 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) https://doi.org/10.1109/CCGRID.2018.00026	conference	May 2018
An overview of the HDF5 technology suite and its applications Folk, Mike; Heber, Gerd; Koziol, Quincey Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases - AD '11 https://doi.org/10.1145/1966895.1966900	conference	January 2011
Mercury: Enabling remote procedure call for high-performance computing Soumagne, Jerome; Kimpe, Dries; Zounmevo, Judicael 2013 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/CLUSTER.2013.6702617	conference	September 2013
Mochi: Composing Data Services for High-Performance Computing Environments Ross, Robert B.; Amvrosiadis, George; Carns, Philip Journal of Computer Science and Technology, Vol. 35, Issue 1 https://doi.org/10.1007/s11390-020-9802-0	journal	January 2020
On the role of burst buffers in leadership-class storage systems Liu, Ning; Cope, Jason; Carns, Philip 2012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST) https://doi.org/10.1109/MSST.2012.6232369	conference	April 2012
PLFS: a checkpoint filesystem for parallel applications Bent, John; Gibson, Garth; Grider, Gary https://doi.org/10.1145/1654059.1654081	conference	January 2009
Faodel Ulmer, Craig; Mukherjee, Shyamali; Templet, Gary Proceedings of the 9th Workshop on Scientific Cloud Computing https://doi.org/10.1145/3217880.3217888	conference	June 2018
A Plugin for HDF5 Using PLFS for Improved I/O Performance and Semantic Analysis Mehta, Kshitij; Bent, John; Torres, Aaron https://doi.org/10.1109/SC.Companion.2012.102	conference	November 2012
Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories Zheng, Qing; Cranor, Charles D.; Jain, Ankush ACM Transactions on Storage, Vol. 16, Issue 4 https://doi.org/10.1145/3415581	journal	September 2020

Similar Records

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Journal Article · 2024 · The international journal of high performance computing applications · OSTI ID:2573001

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

Journal Article · 2020 · Journal of Computer Science and Technology · OSTI ID:1582374

Related Subjects

Computer Science
Engineering

Accelerating HDF5 I/O for Exascale Using DAOS

Citation Formats

References (15)

Similar Records

Related Subjects