A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

Zhang, Yang; Guo, Hanqi; Shang, Lanyu; Wang, Dong; Peterka, Tom

doi:10.1109/tbdata.2021.3092174

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

Conference · 30 November 2022

DOI:https://doi.org/10.1109/tbdata.2021.3092174· OSTI ID:2427985

Zhang, Yang; Guo, Hanqi; Shang, Lanyu; Wang, Dong; Peterka, Tom

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on a quality-aware adaptive temporal data selection and reconstruction problem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we develop DeepSample , an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

View Conference

Research Organization:: Argonne National Laboratory (ANL)

Sponsoring Organization:: USDOE Office of Science - Office of Advanced Scientific Computing Research (ASCR) - Scientific Discovery through Advanced Computing (SciDAC); USDOE Office of Science; USDOE Office of Science - Office of Advanced Scientific Computing Research (ASCR)

DOE Contract Number:: AC02-06CH11357

OSTI ID:: 2427985

Country of Publication:: United States

Language:: English

References (32)

In Situ Visualization at Extreme Scale: Challenges and Opportunities No authors listed IEEE Computer Graphics and Applications, Vol. 29, Issue 6 https://doi.org/10.1109/MCG.2009.120	journal	November 2009
Access Pattern Learning with Long Short-Term Memory for Parallel Particle Tracing Hong, Fan; Zhang, Jiang; Yuan, Xiaoru 2018 IEEE Pacific Visualization Symposium (PacificVis) https://doi.org/10.1109/PacificVis.2018.00018	conference	April 2018
In situ data-driven adaptive sampling for large-scale simulation data summarization Biswas, Ayan; Dutta, Soumya; Pulido, Jesus Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization - ISAV '18 https://doi.org/10.1145/3281464.3281467	conference	January 2018
Robust Reference Frame Extraction from Unsteady 2D Vector Fields with Convolutional Neural Networks Kim, Byungsoo; Günther, Tobias Computer Graphics Forum, Vol. 38, Issue 3 https://doi.org/10.1111/cgf.13689	journal	June 2019
In-situ Sampling of a Large-Scale Particle Simulation for Interactive Visualization and Analysis Woodring, J.; Ahrens, J.; Figg, J. Computer Graphics Forum, Vol. 30, Issue 3 https://doi.org/10.1111/j.1467-8659.2011.01964.x	journal	June 2011
A U-Net Deep Learning Framework for High Performance Vessel Segmentation in Patients With Cerebrovascular Disease Livne, Michelle; Rieger, Jana; Aydin, Orhun Utku Frontiers in Neuroscience, Vol. 13 https://doi.org/10.3389/fnins.2019.00097	journal	February 2019
Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization Tao, Dingwen; Di, Sheng; Chen, Zizhong 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2017.115	conference	May 2017
Image Quality Metrics: PSNR vs. SSIM Hore, Alain; Ziou, Djemel 2010 20th International Conference on Pattern Recognition (ICPR) https://doi.org/10.1109/ICPR.2010.579	conference	August 2010
Workshop report on In Situ Data Management Peterka, Tom; Bard, Deborah; Bennett, Janine https://doi.org/10.2172/1493245	report	February 2019
Computational fluid dynamics modelling in cardiovascular medicine Morris, Paul D.; Narracott, Andrew; von Tengg-Kobligk, Hendrik Heart, Vol. 102, Issue 1 https://doi.org/10.1136/heartjnl-2015-308044	journal	October 2015
Computing Just What You Need: Online Data Analysis and Reduction at Extreme Scales Foster, Ian 2017 IEEE 24th International Conference on High Performance Computing (HiPC) https://doi.org/10.1109/HiPC.2017.00042	conference	December 2017
Adaptive Sampling and Diversity Reception in Multi-hop Wireless Audio Sensor Networks Luo, Hong; Wang, Jinge; Sun, Yan 2010 IEEE 30th International Conference on Distributed Computing Systems https://doi.org/10.1109/ICDCS.2010.32	conference	June 2010
Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets Liang, Xin; Di, Sheng; Tao, Dingwen 2018 IEEE International Conference on Big Data (Big Data) https://doi.org/10.1109/BigData.2018.8622520	conference	December 2018
Fast Error-Bounded Lossy HPC Data Compression with SZ Di, Sheng; Cappello, Franck 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS) https://doi.org/10.1109/IPDPS.2016.11	conference	May 2016
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis Sajjadi, Mehdi S. M.; Scholkopf, Bernhard; Hirsch, Michael 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.481	conference	October 2017
Volume upscaling with convolutional neural networks Zhou, Zhenglei; Hou, Yule; Wang, Qirui Proceedings of the Computer Graphics International Conference https://doi.org/10.1145/3095140.3095178	conference	June 2017
Visualization and Visual Analysis of Ensemble Data: A Survey Wang, Junpeng; Hazarika, Subhashis; Li, Cheng IEEE Transactions on Visualization and Computer Graphics, Vol. 25, Issue 9 https://doi.org/10.1109/TVCG.2018.2853721	journal	September 2019
Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow Wiewel, S.; Becher, M.; Thuerey, N. Computer Graphics Forum, Vol. 38, Issue 2 https://doi.org/10.1111/cgf.13620	journal	May 2019
ADIOS 2: The Adaptable Input Output System. A framework for high-performance data management Godoy, William F.; Podhorszki, Norbert; Wang, Ruonan SoftwareX, Vol. 12 https://doi.org/10.1016/j.softx.2020.100561	journal	July 2020
A Comparison of Shear- and Buoyancy-Driven Planetary Boundary Layer Flows Moeng, Chin-Hoh; Sullivan, Peter P. Journal of the Atmospheric Sciences, Vol. 51, Issue 7 https://doi.org/10.1175/1520-0469(1994)051<0999:ACOSAB>2.0.CO;2	journal	April 1994
Deep visual domain adaptation: A survey Wang, Mei; Deng, Weihong Neurocomputing, Vol. 312 https://doi.org/10.1016/j.neucom.2018.05.083	journal	October 2018
InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations He, Wenbin; Wang, Junpeng; Guo, Hanqi IEEE Transactions on Visualization and Computer Graphics https://doi.org/10.1109/TVCG.2019.2934312	journal	January 2019
tempoGAN: a temporally coherent, volumetric GAN for super-resolution fluid flow Xie, You; Franz, Erik; Chu, Mengyu ACM Transactions on Graphics, Vol. 37, Issue 4 https://doi.org/10.1145/3197517.3201304	journal	August 2018
Taming massive distributed datasets Su, Yu; Agrawal, Gagan; Woodring, Jonathan Proceedings of the 22nd international symposium on High-performance parallel and distributed computing https://doi.org/10.1145/2462902.2462906	conference	June 2013
Salient time steps selection from large scale time-varying data sets with dynamic time warping Tong, Xin; Lee, Teng-Yok; Shen, Han-Wei 2012 IEEE Symposium on Large Data Analysis and Visualization (LDAV) https://doi.org/10.1109/LDAV.2012.6378975	conference	October 2012
Fully convolutional networks for semantic segmentation Long, Jonathan; Shelhamer, Evan; Darrell, Trevor 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2015.7298965	conference	June 2015
Scalable Compression of Stream Cipher Encrypted Images Through Context-Adaptive Sampling Zhou, Jiantao; Au, Oscar C.; Zhai, Guangtao IEEE Transactions on Information Forensics and Security, Vol. 9, Issue 11 https://doi.org/10.1109/TIFS.2014.2352455	journal	November 2014
Deep Cytometry: Deep learning with Real-time Inference in Cell Sorting and Flow Cytometry Li, Yueqin; Mahjoubfar, Ata; Chen, Claire Lifan Scientific Reports, Vol. 9, Issue 1 https://doi.org/10.1038/s41598-019-47193-6	journal	July 2019
Simulations of ultra-high energy cosmic rays in the local Universe and the origin of cosmic magnetic fields Hackstein, S.; Vazza, F.; Brüggen, M. Monthly Notices of the Royal Astronomical Society, Vol. 475, Issue 2 https://doi.org/10.1093/mnras/stx3354	journal	January 2018
Key Time Steps Selection for Large‐Scale Time‐Varying Volume Datasets Using an Information‐Theoretic Storyboard Zhou, Bo; Chiang, Yi‐Jen Computer Graphics Forum, Vol. 37, Issue 3 https://doi.org/10.1111/cgf.13399	journal	June 2018
A Large Eddy Simulation Intercomparison Study of Shallow Cumulus Convection Siebesma, A. Pier; Bretherton, Christopher S.; Brown, Andrew Journal of the Atmospheric Sciences, Vol. 60, Issue 10 https://doi.org/10.1175/1520-0469(2003)60<1201:ALESIS>2.0.CO;2	journal	May 2003
Future challenges for ensemble visualization Obermaier, Harald; Joy, Kenneth I. IEEE Computer Graphics and Applications, Vol. 34, Issue 3 https://doi.org/10.1109/MCG.2014.52	journal	May 2014

Similar Records

U-Net Decoder CRF v0.1.0

Software · 2022 · OSTI ID:code-140540

Spatiotemporally Adaptive Compression for Scientific Dataset with Feature Preservation – A Case Study on Simulation Data with Extreme Climate Events Analysis

Conference · 2023 · OSTI ID:2282939

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Journal Article · 2022 · IEEE Transactions on Parallel and Distributed Systems · OSTI ID:1908133

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

Citation Formats

References (32)

Similar Records

Related Subjects