AMRZone: A Runtime AMR Data Sharing Framework For Scientific Applications

Conference · Thu Aug 31 04:00:00 EDT 2017

DOI:https://doi.org/10.1109/CCGrid.2016.62· OSTI ID:1377848

Zhang, Wenzhao; Tang, Houjun; Harenberg, Steven; Byna, Suren; Zou, Xiaocheng; Devendran, Dharshi; Martin, Daniel; Wu, Kesheng; Dong, Bin; Klasky, Scott; Samatova, Nagiza

Frameworks that facilitate runtime data sharing across multiple applications are of great importance for scientific data analytics. Although existing frameworks work well over uniform mesh data, they can not effectively handle adaptive mesh refinement (AMR) data. Among the challenges to construct an AMR-capable framework include: (1) designing an architecture that facilitates online AMR data management; (2) achieving a load-balanced AMR data distribution for the data staging space at runtime; and (3) building an effective online index to support the unique spatial data retrieval requirements for AMR data. Towards addressing these challenges to support runtime AMR data sharing across scientific applications, we present the AMRZone framework. Experiments over real-world AMR datasets demonstrate AMRZone's effectiveness at achieving a balanced workload distribution, reading/writing large-scale datasets with thousands of parallel processes, and satisfying queries with spatial constraints. Moreover, AMRZone's performance and scalability are even comparable with existing state-of-the-art work when tested over uniform mesh data with up to 16384 cores; in the best case, our framework achieves a 46% performance improvement.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: DOE Office of Science; Computational Research Division
• OSTI ID: 1377848
• Report Number(s): LBNL-1005694; ir:1005694
• Country of Publication: United States
• Language: English