skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: High-Performance I/O: HDF5 for Lattice QCD

Abstract

Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design and implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.

Authors:
 [1];  [2];  [3];  [4];  [5]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Nuclear Science Division
  2. Massachusetts Inst. of Technology (MIT), Cambridge, MA (United States). Center for Theoretical Physics
  3. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Computational Research Division
  4. Brookhaven National Lab. (BNL), Upton, NY (United States). RIKEN Research Center
  5. College of William and Mary, Williamsburg, VA (United States). Dept. of Physics; Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)
OSTI Identifier:
1398478
Grant/Contract Number:
AC02-05CH11231; FG02-94ER40818
Resource Type:
Journal Article: Accepted Manuscript
Journal Name:
PoS Proceedings of Science
Additional Journal Information:
Conference: 32nd International Symposium on Lattice Field Theory, Columbia Univ., New York, NY (United States), 23-28 Jun 2014; Journal ID: ISSN 1824-8039
Publisher:
SISSA
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING

Citation Formats

Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, and Walker-Loud, Andre. High-Performance I/O: HDF5 for Lattice QCD. United States: N. p., 2017. Web.
Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, & Walker-Loud, Andre. High-Performance I/O: HDF5 for Lattice QCD. United States.
Kurth, Thorsten, Pochinsky, Andrew, Sarje, Abhinav, Syritsyn, Sergey, and Walker-Loud, Andre. Tue . "High-Performance I/O: HDF5 for Lattice QCD". United States. doi:. https://www.osti.gov/servlets/purl/1398478.
@article{osti_1398478,
title = {High-Performance I/O: HDF5 for Lattice QCD},
author = {Kurth, Thorsten and Pochinsky, Andrew and Sarje, Abhinav and Syritsyn, Sergey and Walker-Loud, Andre},
abstractNote = {Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design and implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.},
doi = {},
journal = {PoS Proceedings of Science},
number = ,
volume = ,
place = {United States},
year = {Tue May 09 00:00:00 EDT 2017},
month = {Tue May 09 00:00:00 EDT 2017}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
The DOI is not currently available

Save / Share:
  • Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design andmore » implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.« less
  • Direct observations of local lattice aluminum environments have been a major challenge for aluminum -bearing Li ion battery materials, such as LiNi1-y-zCoyAlzO2 Al(NCA) and aluminum-doped LiNixMnyCozO2 (NMC). Al-27 magic angle spinning (MAS) nuclear magnetic resonance (NMR) spectroscopy is the only structural probe currently available that can qualitatively and quantitatively characterize lattice and nonlattice (i.e., surface, coatings, segregation, secondary phase etc.) aluminum coordination and provide information that helps discern its effect in the lattice. In the present study, we use NMR to gain new insights into transition metal (TM)-O-Al coordination and evolution of lattice aluminum sites upon cycling. With the aidmore » of first-principles DFT calculations, we show direct evidence of lattice Al sites, nonpreferential Ni/Co-O-Al ordering in NCA, and the lack of bulk lattice aluminum in aluminum -"doped" NMC. Aluminum coordination of the paramagnetic (lattice) and diamagnetic (nonlattice) nature is investigated for Al-doped NMC and NCA. For the latter, the evolution of the lattice site(s) upon cycling is also studied. A clear reordering of lattice aluminum environments due to nickel migration is observed in NCA upon extended cycling.« less
  • There is a growing interest in using Java as the language for developing high-performance computing applications. To be successful in the high-performance computing domain, however, Java must not only be able to provide high computational performance, but also high-performance I/O. In this paper, we first examine several approaches that attempt to provide high-performance I/O in Java - many of which are not obvious at first glance - and evaluate their performance on two parallel machines, the IBM SP and the SGI Origin2000. We then propose extensions to the Java I/O library that address the deficiencies in the Java I/O APImore » and improve performance dramatically. The extensions add bulk (array) I/O operations to Java, thereby removing much of the overhead currently associated with array I/O in Java. We have implemented the extensions in two ways: in a standard JVM using the Java Native Interface (JNI) and in a high-performance parallel dialect of Java called Titanium. We describe the two implementations and present performance results that demonstrate the benefits of the proposed extensions.« less
  • Cited by 28