AMRIC: A Novel In Situ Lossy Compression Framework for Efficient I/O in Adaptive Mesh Refinement Applications

Conference · Sun Nov 12 04:00:00 EST 2023

DOI:https://doi.org/10.1145/3581784.3613212· OSTI ID:2323332

Wang, Daoce ^[1]; Pulido, Jesus ^[2]; Grosset, Pascal ^[2]; Tian, Jianan ^[1]; Jin, Sian ^[1]; Tang, Houyun ^[3]; Sexton, Jean ^[3]; Di, Sheng ^[4]; Zhao, Kai ^[5]; Fang, Bo ^[6]; Lukic, Zarija ^[3]; Cappello, Franck ^[4]; Ahrens, James ^[2]; Tao, Dingwen ^[1]

---

1. Indiana University-Bloomington
2. Los Alamos National Laboratory
3. Lawrence Berkeley National Laboratory
4. Argonne National Laboratory
5. University of Alabama - Birmingham
6. BATTELLE (PACIFIC NW LAB)

As supercomputers advance towards exascale capabilities, computational intensity increases significantly, and the volume of data requiring storage and transmission experiences exponential growth. Adaptive Mesh Refinement (AMR) has emerged as an effective solution to address these two challenges. Concurrently, error-bounded lossy compression is recognized as one of the most efficient approaches to tackle the latter issue. Despite their respective advantages, few attempts have been made to investigate how AMR and error-bounded lossy compression can function together. To this end, this study presents a novel in-situ lossy compression framework that employs the HDF5 filter to improve both I/O costs and boost compression quality for AMR applications. We implement our solution into the AMReX framework and evaluate on two real-world AMR applications, Nyx and WarpX, on the Summit supercomputer. Experiments with 512 cores demonstrate that AMRIC improves the compression ratio by 81X and the I/O performance by 39X over AMReX's original compression solution.

• Research Organization: Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC05-76RL01830
• OSTI ID: 2323332
• Report Number(s): PNNL-SA-183902
• Country of Publication: United States
• Language: English