skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: High-Performance I/O: HDF5 for Lattice QCD

Abstract

Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design and implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.

Authors:
 [1];  [2];  [3];  [4];  [5]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Nuclear Science Division
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Center for Theoretical Physics
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
  4. Brookhaven National Lab. (BNL), Upton, NY (United States). RIKEN Research Center
  5. College of William and Mary, Williamsburg, VA (United States). Dept. of Physics; Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1398478
Grant/Contract Number:  
AC02-05CH11231; FG02-94ER40818
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
PoS Proceedings of Science
Additional Journal Information:
Conference: 32nd International Symposium on Lattice Field Theory, Columbia Univ., New York, NY (United States), 23-28 Jun 2014; Journal ID: ISSN 1824-8039
Publisher:
SISSA
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, and Walker-Loud, Andre. High-Performance I/O: HDF5 for Lattice QCD. United States: N. p., 2017. Web.
Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, & Walker-Loud, Andre. High-Performance I/O: HDF5 for Lattice QCD. United States.
Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, and Walker-Loud, Andre. Tue . "High-Performance I/O: HDF5 for Lattice QCD". United States. doi:. https://www.osti.gov/servlets/purl/1398478.
@article{osti_1398478,
title = {High-Performance I/O: HDF5 for Lattice QCD},
author = {Kurth, Thorsten and Pochinsky, Andrew and Sarje, Abhinav and Syritsyn, Sergey and Walker-Loud, Andre},
abstractNote = {Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design and implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.},
doi = {},
journal = {PoS Proceedings of Science},
number = ,
volume = ,
place = {United States},
year = {Tue May 09 00:00:00 EDT 2017},
month = {Tue May 09 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
The DOI is not currently available

Save / Share: