skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

Abstract

Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.

Authors:
 [1];  [2];  [1];  [1];  [2];  [2];  [2];  [1];  [3];  [2]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  2. The HDF Group, Champaign, IL (United States)
  3. Argonne National Lab. (ANL), Lemont, IL (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); USDOE National Nuclear Security Administration (NNSA); USDOE Office of Science (SC), Basic Energy Sciences (BES). Scientific User Facilities Division
OSTI Identifier:
1582374
Grant/Contract Number:  
AC02-05CH11231; 17-SC-20-SC; AC02-06CH11357
Resource Type:
Accepted Manuscript
Journal Name:
Journal of Computer Science and Technology
Additional Journal Information:
Journal Volume: 35; Journal Issue: 1; Journal ID: ISSN 1000-9000
Publisher:
Springer Nature
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; parallel I/O; Hierarchical Data Format version 5 (HDF5); I/O performance; virtual object layer; HDF5 optimizations

Citation Formats

Byna, Suren, Breitenfeld, M. Scot, Dong, Bin, Koziol, Quincey, Pourmal, Elena, Robinson, Dana, Soumagne, Jerome, Tang, Houjun, Vishwanath, Venkatram, and Warren, Richard. ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems. United States: N. p., 2020. Web. https://doi.org/10.1007/s11390-020-9822-9.
Byna, Suren, Breitenfeld, M. Scot, Dong, Bin, Koziol, Quincey, Pourmal, Elena, Robinson, Dana, Soumagne, Jerome, Tang, Houjun, Vishwanath, Venkatram, & Warren, Richard. ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems. United States. https://doi.org/10.1007/s11390-020-9822-9
Byna, Suren, Breitenfeld, M. Scot, Dong, Bin, Koziol, Quincey, Pourmal, Elena, Robinson, Dana, Soumagne, Jerome, Tang, Houjun, Vishwanath, Venkatram, and Warren, Richard. Fri . "ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems". United States. https://doi.org/10.1007/s11390-020-9822-9. https://www.osti.gov/servlets/purl/1582374.
@article{osti_1582374,
title = {ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems},
author = {Byna, Suren and Breitenfeld, M. Scot and Dong, Bin and Koziol, Quincey and Pourmal, Elena and Robinson, Dana and Soumagne, Jerome and Tang, Houjun and Vishwanath, Venkatram and Warren, Richard},
abstractNote = {Scientific applications at exascale generate and analyze massive amounts of data. A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems. As such, parallel I/O, the key technology enables moving data between compute nodes and storage, faces monumental challenges from new applications, memory, and storage architectures considered in the designs of exascale systems. As the storage hierarchy is expanding to include node-local persistent memory, burst buffers, etc., as well as disk-based storage, data movement among these layers must be efficient. Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond. In this paper, we describe new capabilities we have developed in Hierarchical Data Format version 5 (HDF5), the most popular parallel I/O library for scientific applications. HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems. The state-of-the-art features we describe include: Virtual Object Layer (VOL), Data Elevator, asynchronous I/O, full-featured single-writer and multiple-reader (Full SWMR), and parallel querying. In this paper, we introduce these features, their implementations, and the performance and feature benefits to applications and other libraries.},
doi = {10.1007/s11390-020-9822-9},
journal = {Journal of Computer Science and Technology},
number = 1,
volume = 35,
place = {United States},
year = {2020},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share:

Works referenced in this record:

ARCHIE: Data Analysis Acceleration with Array Caching in Hierarchical Storage
conference, December 2018


Argobots: A Lightweight Low-Level Threading and Tasking Framework
journal, March 2018

  • Seo, Sangmin; Amer, Abdelhalim; Balaji, Pavan
  • IEEE Transactions on Parallel and Distributed Systems, Vol. 29, Issue 3
  • DOI: 10.1109/TPDS.2017.2766062

Data Elevator: Low-Contention Data Movement in Hierarchical Storage System
conference, December 2016

  • Dong, Bin; Byna, Suren; Wu, Kesheng
  • 2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
  • DOI: 10.1109/HiPC.2016.026

Parallel I/O, analysis, and visualization of a trillion particle simulation
conference, November 2012

  • Byna, Surendra; Chou, Jerry; Rubel, Oliver
  • 2012 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2012.92

Terascale direct numerical simulations of turbulent combustion using S3D
journal, January 2009


Parallel netCDF: A High-Performance Scientific I/O Interface
conference, January 2003

  • Li, Jianwei; Zingale, Michael; Liao, Wei-keng
  • Proceedings of the 2003 ACM/IEEE conference on Supercomputing - SC '03
  • DOI: 10.1145/1048935.1050189

ArrayUDF: User-Defined Scientific Data Analysis on Arrays
conference, January 2017

  • Dong, Bin; Wu, Kesheng; Byna, Surendra
  • Proceedings of the 26th International Symposium on High-Performance Parallel and Distributed Computing - HPDC '17
  • DOI: 10.1145/3078597.3078599

An overview of the HDF5 technology suite and its applications
conference, January 2011

  • Folk, Mike; Heber, Gerd; Koziol, Quincey
  • Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases - AD '11
  • DOI: 10.1145/1966895.1966900

Adaptable, metadata rich IO methods for portable high performance IO
conference, May 2009

  • Lofstead, Jay; Zheng, Fang; Klasky, Scott
  • Distributed Processing (IPDPS), 2009 IEEE International Symposium on Parallel & Distributed Processing
  • DOI: 10.1109/IPDPS.2009.5161052