Performance Impact of I/O on QMCPack Simulations at the Petascale and Beyond

Conference · Tue Dec 31 23:00:00 EST 2013

DOI:https://doi.org/10.1109/CSE.2013.24· OSTI ID:1126967

Herbein, Stephen N ^[1]; Matheny, Michael E ^[1]; Wezowicz, Matthew R ^[1]; Krogel, Jaron T ^[1]; Kim, Jeongnim ^[1]; Klasky, Scott A ^[1]; Taufer, Michela ^[1]

---

1. ORNL

Traditional petascale applications, such as QMCPack, can scale their computations to completely utilize modern supercomputers like Titan, but they cannot scale their I/O. To preserve scalability, scientists cannot save data at the granularity needed to enable scientific discovery and are forced to use large intervals between two checkpoint calls. In this paper, we work to increase the granularity of the I/O in QMCPack simulations without increasing the I/O associated overhead or compromising the scalability of the simulations. Our solution redesigns the I/O algorithms used by QMCPack to gather finer-grained data at high frequencies and integrate the ADIOS API to select effective I/O methods without major code changes. The extension of a tool such as Skel to mimic the variable I/O in QMCPack allows us to predict the I/O performance of the code when using ADIOS methods at the petascale. We show how I/O libraries like ADIOS allow us to increase the amount of scientific data extracted from QMCPack simulations at the granularity desired by the scientists while keeping the I/O overhead below 10%. We also show how the impact of checkpoint I/O for the QMCPack code using ADIOS is below 5% when using preventive tactics for checkpointing at the petascale and beyond.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
• Sponsoring Organization: ORNL LDRD Director's R&D; USDOE Office of Science (SC)
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1126967
• Country of Publication: United States
• Language: English