skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

This content will become publicly available on March 9, 2021

Title: Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2];  [2];  [2];  [1]
  1. The HDF Group Champaign Illinois
  2. Lawrence Berkeley National Laboratory Berkeley California
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1603709
Grant/Contract Number:  
[AC02-05CH11231; SC0016454]
Resource Type:
Publisher's Accepted Manuscript
Journal Name:
Concurrency and Computation. Practice and Experience
Additional Journal Information:
[Journal Name: Concurrency and Computation. Practice and Experience]; Journal ID: ISSN 1532-0626
Publisher:
Wiley Blackwell (John Wiley & Sons)
Country of Publication:
United Kingdom
Language:
English

Citation Formats

Mu, Jingqing, Soumagne, Jerome, Byna, Suren, Koziol, Quincey, Tang, Houjun, and Warren, Richard. Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage. United Kingdom: N. p., 2020. Web. doi:10.1002/cpe.5715.
Mu, Jingqing, Soumagne, Jerome, Byna, Suren, Koziol, Quincey, Tang, Houjun, & Warren, Richard. Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage. United Kingdom. doi:10.1002/cpe.5715.
Mu, Jingqing, Soumagne, Jerome, Byna, Suren, Koziol, Quincey, Tang, Houjun, and Warren, Richard. Mon . "Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage". United Kingdom. doi:10.1002/cpe.5715.
@article{osti_1603709,
title = {Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage},
author = {Mu, Jingqing and Soumagne, Jerome and Byna, Suren and Koziol, Quincey and Tang, Houjun and Warren, Richard},
abstractNote = {},
doi = {10.1002/cpe.5715},
journal = {Concurrency and Computation. Practice and Experience},
number = ,
volume = ,
place = {United Kingdom},
year = {2020},
month = {3}
}

Journal Article:
Free Publicly Available Full Text
This content will become publicly available on March 9, 2021
Publisher's Version of Record

Save / Share:

Works referenced in this record:

A cost-effective, high-bandwidth storage architecture
journal, November 1998

  • Gibson, Garth A.; Zelenka, Jim; Nagle, David F.
  • ACM SIGPLAN Notices, Vol. 33, Issue 11
  • DOI: 10.1145/291006.291029

Toward Scalable and Asynchronous Object-Centric Data Management for HPC
conference, May 2018

  • Tang, Houjun; Byna, Suren; Tessier, Francois
  • 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)
  • DOI: 10.1109/CCGRID.2018.00026

On implementing MPI-IO portably and with high performance
conference, January 1999

  • Thakur, Rajeev; Gropp, William; Lusk, Ewing
  • Proceedings of the sixth workshop on I/O in parallel and distributed systems - IOPADS '99
  • DOI: 10.1145/301816.301826

BD-CATS: big data clustering at trillion particle scale
conference, January 2015

  • Patwary, Md. Mostofa Ali; Dubey, Pradeep; Byna, Suren
  • Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '15
  • DOI: 10.1145/2807591.2807616

Improving parallel I/O autotuning with performance modeling
conference, January 2014

  • Behzad, Babak; Byna, Surendra; Wild, Stefan M.
  • Proceedings of the 23rd international symposium on High-performance parallel and distributed computing - HPDC '14
  • DOI: 10.1145/2600212.2600708

A Plugin for HDF5 Using PLFS for Improved I/O Performance and Semantic Analysis
conference, November 2012


Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks: HELLO ADIOS
journal, August 2013

  • Liu, Qing; Logan, Jeremy; Tian, Yuan
  • Concurrency and Computation: Practice and Experience, Vol. 26, Issue 7
  • DOI: 10.1002/cpe.3125

Mercury: Enabling remote procedure call for high-performance computing
conference, September 2013

  • Soumagne, Jerome; Kimpe, Dries; Zounmevo, Judicael
  • 2013 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2013.6702617

Ultrahigh performance three-dimensional electromagnetic relativistic kinetic plasma simulation
journal, May 2008

  • Bowers, K. J.; Albright, B. J.; Yin, L.
  • Physics of Plasmas, Vol. 15, Issue 5
  • DOI: 10.1063/1.2840133

An overview of the HDF5 technology suite and its applications
conference, January 2011

  • Folk, Mike; Heber, Gerd; Koziol, Quincey
  • Proceedings of the EDBT/ICDT 2011 Workshop on Array Databases - AD '11
  • DOI: 10.1145/1966895.1966900

Data Elevator: Low-Contention Data Movement in Hierarchical Storage System
conference, December 2016

  • Dong, Bin; Byna, Suren; Wu, Kesheng
  • 2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
  • DOI: 10.1109/HiPC.2016.026

Persistent object management system
journal, January 1984

  • Cockshot, W. P.; Atkinson, M. P.; Chisholm, K. J.
  • Software: Practice and Experience, Vol. 14, Issue 1
  • DOI: 10.1002/spe.4380140106

The POSIX family of standards
journal, March 1995


A Transparent Server-Managed Object Storage System for HPC
conference, September 2018

  • Mu, Jingqing; Soumagne, Jerome; Tang, Houjun
  • 2018 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2018.00063

Improving I/O Forwarding Throughput with Data Compression
conference, September 2011

  • Welton, Benjamin; Kimpe, Dries; Cope, Jason
  • 2011 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2011.80

RADOS: a scalable, reliable storage service for petabyte-scale storage clusters
conference, January 2007

  • Weil, Sage A.; Leung, Andrew W.; Brandt, Scott A.
  • Proceedings of the 2nd international workshop on Petascale data storage held in conjunction with Supercomputing '07 - PDSW '07
  • DOI: 10.1145/1374596.1374606

UniviStor: Integrated Hierarchical and Distributed Storage for HPC
conference, September 2018

  • Wang, Teng; Byna, Suren; Dong, Bin
  • 2018 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2018.00025

Entering the petaflop era: The architecture and performance of Roadrunner
conference, November 2008

  • Barker, Kevin J.; Davis, Kei; Hoisie, Adolfy
  • 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis
  • DOI: 10.1109/SC.2008.5217926

Design of the Mneme persistent object store
journal, April 1990

  • Moss, J. Eliot B.
  • ACM Transactions on Information Systems (TOIS), Vol. 8, Issue 2
  • DOI: 10.1145/96105.96109

A generic persistent object store
journal, January 1992


TAPIOCA: An I/O Library for Optimized Topology-Aware Data Aggregation on Large-Scale Supercomputers
conference, September 2017

  • Tessier, Francois; Vishwanath, Venkatram; Jeannot, Emmanuel
  • 2017 IEEE International Conference on Cluster Computing (CLUSTER)
  • DOI: 10.1109/CLUSTER.2017.80

SSDUP: a traffic-aware ssd burst buffer for HPC systems
conference, January 2017

  • Shi, Xuanhua; Li, Ming; Liu, Wei
  • Proceedings of the International Conference on Supercomputing - ICS '17
  • DOI: 10.1145/3079079.3079087

Parallel netCDF: A High-Performance Scientific I/O Interface
conference, January 2003

  • Li, Jianwei; Zingale, Michael; Liao, Wei-keng
  • Proceedings of the 2003 ACM/IEEE conference on Supercomputing - SC '03
  • DOI: 10.1145/1048935.1050189

Searching for millions of objects in the BOSS spectroscopic survey data with H5Boss
conference, August 2017