High-Performance I/O: HDF5 for Lattice QCD

Conference · Thu Jan 01 00:00:00 EST 2015 · PoS LATTICE2014

OSTI ID:1183734

Kurth, Thorsten; Pochinsky, Andrew; Sarje, Abhinav; Syritsyn, Sergey; Walker-Loud, Andre

Practitioners of lattice QCD/QFT have been some of the primary pioneer users of the state-of-the-art high-performance-computing systems, and contribute towards the stress tests of such new machines as soon as they become available. As with all aspects of high-performance-computing, I/O is becoming an increasingly specialized component of these systems. In order to take advantage of the latest available high-performance I/O infrastructure, to ensure reliability and backwards compatibility of data files, and to help unify the data structures used in lattice codes, we have incorporated parallel HDF5 I/O into the SciDAC supported USQCD software stack. Here we present the design and implementation of this I/O framework. Our HDF5 implementation outperforms optimized QIO at the 10-20% level and leaves room for further improvement by utilizing appropriate dataset chunking.

• Research Organization: Thomas Jefferson National Accelerator Facility (TJNAF), Newport News, VA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)
• DOE Contract Number: AC05-06OR23177; AC02-05CH11231
• OSTI ID: 1183734
• Report Number(s): JLAB-THY-15-2007; DOE/OR/23177-3292; arxiv:1501.06992; NT@WM-15-02; FG02-94ER40818
• Journal Information: PoS LATTICE2014, Conference: Lattice 2014, 23-28 Jun 2014. Brookhaven, NY, USA
• Country of Publication: United States
• Language: English