Deep Hierarchical Super Resolution for Scientific Data

Wurster, Skylar W.; Guo, Hanqi; Shen, Han-Wei; Peterka, Tom; Xu, Jiayi

doi:10.1109/tvcg.2022.3214420

Deep Hierarchical Super Resolution for Scientific Data

Journal Article · Mon Oct 17 00:00:00 EDT 2022 · IEEE Transactions on Visualization and Computer Graphics

DOI:https://doi.org/10.1109/tvcg.2022.3214420· OSTI ID:2477201

^[1]; ^[1]; ^[1]; Peterka, Tom ^[2]; ^[1]

The Ohio State Univ., Columbus, OH (United States)
Argonne National Laboratory (ANL), Argonne, IL (United States)

We present a novel technique for hierarchical super resolution (SR) with neural networks (NNs), which upscales volumetric data represented with an octree data structure to a high-resolution uniform gridwith minimal seam artifacts on octree node boundaries. Our method uses existing state-of-the-art SR models and adds flexibility to upscale input data with varying levels of detail across the domain, instead of only uniform grid data that are supported in previous approaches.The key is to use a hierarchy of SR NNs, each trained to perform 2x SR between two levels of detail, with a hierarchical SR algorithm that minimizes seam artifacts by starting from the coarsest level of detail and working up.We show that our hierarchical approach outperforms baseline interpolation and hierarchical upscaling methods, and demonstrate the usefulness of our proposed approach across three use cases including data reduction using hierarchical downsampling+SR instead of uniform downsampling+SR, computation savings for hierarchical finite-time Lyapunov exponent field calculation, and super-resolving low-resolution simulation results for a high-resolution approximation visualization.

View Accepted Manuscript (DOE)

Research Organization:: Argonne National Laboratory (ANL), Argonne, IL (United States)

Sponsoring Organization:: National Science Foundation (NSF); USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: AC02-06CH11357; SC0021360

OSTI ID:: 2477201

Journal Information:: IEEE Transactions on Visualization and Computer Graphics, Journal Name: IEEE Transactions on Visualization and Computer Graphics Journal Issue: 12 Vol. 29; ISSN 1077-2626

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (57)

A comparative study of convolutional neural network models for wind field downscaling Höhlein, Kevin; Kern, Michael; Hewson, Timothy Meteorological Applications, Vol. 27, Issue 6 https://doi.org/10.1002/met.1961	journal	November 2020
Introducing Enzo, an AMR Cosmology Application O'shea, Brian W.; Bryan, Greg; Bordner, James Adaptive Mesh Refinement - Theory and Applications https://doi.org/10.1007/3-540-27039-6_24	book	January 2006
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks Wang, Xintao; Yu, Ke; Wu, Shixiang Lecture Notes in Computer Science https://doi.org/10.1007/978-3-030-11021-5_5	book	January 2019
Learning a Deep Convolutional Network for Image Super-Resolution Dong, Chao; Loy, Chen Change; He, Kaiming Computer Vision – ECCV 2014 https://doi.org/10.1007/978-3-319-10593-2_13	book	January 2014
A single-pass GPU ray casting framework for interactive out-of-core rendering of massive volumetric datasets Gobbetti, Enrico; Marton, Fabio; Iglesias Guitián, José Antonio The Visual Computer, Vol. 24, Issue 7-9 https://doi.org/10.1007/s00371-008-0261-9	journal	June 2008
Multilevel techniques for compression and reduction of scientific data—the univariate case Ainsworth, Mark; Tugluk, Ozan; Whitney, Ben Computing and Visualization in Science, Vol. 19, Issue 5-6 https://doi.org/10.1007/s00791-018-00303-9	journal	November 2018
Adaptive mesh refinement for hyperbolic partial differential equations Berger, Marsha J.; Oliger, Joseph Journal of Computational Physics, Vol. 53, Issue 3 https://doi.org/10.1016/0021-9991(84)90073-1	journal	March 1984
Local adaptive mesh refinement for shock hydrodynamics Berger, M. J.; Colella, P. Journal of Computational Physics, Vol. 82, Issue 1 https://doi.org/10.1016/0021-9991(89)90035-1	journal	May 1989
Geometric modeling using octree encoding Meagher, Donald Computer Graphics and Image Processing, Vol. 19, Issue 2 https://doi.org/10.1016/0146-664X(82)90104-6	journal	June 1982
Gerris: a tree-based adaptive solver for the incompressible Euler equations in complex geometries Popinet, Stéphane Journal of Computational Physics, Vol. 190, Issue 2 https://doi.org/10.1016/S0021-9991(03)00298-5	journal	September 2003
Super-resolution reconstruction of turbulent flows with machine learning Fukami, Kai; Fukagata, Koji; Taira, Kunihiko Journal of Fluid Mechanics, Vol. 870 https://doi.org/10.1017/jfm.2019.238	journal	May 2019
Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows Fukami, Kai; Fukagata, Koji; Taira, Kunihiko Journal of Fluid Mechanics, Vol. 909 https://doi.org/10.1017/jfm.2020.948	journal	December 2020
A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence Li, Yi; Perlman, Eric; Wan, Minping Journal of Turbulence, Vol. 9 https://doi.org/10.1080/14685240802376389	journal	January 2008
Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY Almgren, Ann S.; Bell, John B.; Lijewski, Mike J. The Astrophysical Journal, Vol. 765, Issue 1 https://doi.org/10.1088/0004-637X/765/1/39	journal	February 2013
Deformable Non-Local Network for Video Super-Resolution Wang, Hua; Su, Dewei; Liu, Chuangchuang IEEE Access, Vol. 7 https://doi.org/10.1109/ACCESS.2019.2958030	journal	January 2019
Accurate Image Super-Resolution Using Very Deep Convolutional Networks Kim, Jiwon; Lee, Jung Kwon; Lee, Kyoung Mu 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.182	conference	June 2016
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network Shi, Wenzhe; Caballero, Jose; Huszar, Ferenc 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.207	conference	June 2016
Deep Residual Learning for Image Recognition He, Kaiming; Zhang, Xiangyu; Ren, Shaoqing 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2016.90	conference	June 2016
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network Ledig, Christian; Theis, Lucas; Huszar, Ferenc 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.19	conference	July 2017
OctNet: Learning Deep 3D Representations at High Resolutions Riegler, Gernot; Ulusoy, Ali Osman; Geiger, Andreas 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2017.701	conference	July 2017
Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes Takikawa, Towaki; Litalien, Joey; Yin, Kangxue 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR46437.2021.01120	conference	June 2021
Enhanced Deep Residual Networks for Single Image Super-Resolution Lim, Bee; Son, Sanghyun; Kim, Heewon 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) https://doi.org/10.1109/CVPRW.2017.151	conference	July 2017
Exploring Autoencoder-based Error-bounded Compression for Scientific Data Liu, Jinyang; Di, Sheng; Zhao, Kai 2021 IEEE International Conference on Cluster Computing (CLUSTER) https://doi.org/10.1109/Cluster48925.2021.00034	conference	September 2021
Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs Tatarchenko, Maxim; Dosovitskiy, Alexey; Brox, Thomas 2017 IEEE International Conference on Computer Vision (ICCV) https://doi.org/10.1109/ICCV.2017.230	conference	October 2017
An analysis of scalable GPU-based ray-guided volume rendering Fogal, Thomas; Schiewe, Alexander; Kruger, Jens 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV) https://doi.org/10.1109/LDAV.2013.6675157	conference	October 2013
SSR-VFD: Spatial Super-Resolution for Vector Field Data Analysis and Visualization Guo, Li; Ye, Shaojie; Han, Jun 2020 IEEE Pacific Visualization Symposium (PacificVis) https://doi.org/10.1109/PacificVis48177.2020.8737	conference	June 2020
Interactive Isosurface Ray Tracing of Large Octree Volumes Knoll, Aaron; Wald, Ingo; Parker, Steven 2006 IEEE Symposium on Interactive Ray Tracing https://doi.org/10.1109/RT.2006.280222	conference	September 2006
MGARD+: Optimizing Multilevel Methods for Error-Bounded Scientific Data Reduction Liang, Xin; Whitney, Ben; Chen, Jieyang IEEE Transactions on Computers, Vol. 71, Issue 7 https://doi.org/10.1109/TC.2021.3092201	journal	July 2022
Image Quality Assessment: From Error Visibility to Structural Similarity Wang, Z.; Bovik, A. C.; Sheikh, H. R. IEEE Transactions on Image Processing, Vol. 13, Issue 4 https://doi.org/10.1109/TIP.2003.819861	journal	April 2004
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks Lai, Wei-Sheng; Huang, Jia-Bin; Ahuja, Narendra IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, Issue 11 https://doi.org/10.1109/TPAMI.2018.2865304	journal	November 2019
Deep Learning for Image Super-Resolution: A Survey Wang, Zhihao; Chen, Jian; Hoi, Steven C. H. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, Issue 10 https://doi.org/10.1109/TPAMI.2020.2982166	journal	October 2021
Efficient Visualization of Lagrangian Coherent Structures by Filtered AMR Ridge Extraction Sadlo, Filip; Peikert, Ronald IEEE Transactions on Visualization and Computer Graphics, Vol. 13, Issue 6 https://doi.org/10.1109/TVCG.2007.70554	journal	November 2007
Octree Rasterization: Accelerating High-Quality Out-of-Core GPU Volume Rendering No authors listed IEEE Transactions on Visualization and Computer Graphics, Vol. 19, Issue 10 https://doi.org/10.1109/TVCG.2012.151	journal	October 2013
Interactive Volume Exploration of Petascale Microscopy Data Streams Using a Visualization-Driven Virtual Memory Approach Hadwiger, M.; Beyer, J. IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 12 https://doi.org/10.1109/TVCG.2012.240	journal	December 2012
Interactive Computation and Rendering of Finite-Time Lyapunov Exponent Fields Barakat, Samer; Garth, C.; Tricoche, X. IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 8 https://doi.org/10.1109/TVCG.2012.33	journal	August 2012
CPU Isosurface Ray Tracing of Adaptive Mesh Refinement Data Wang, Feng; Wald, Ingo; Wu, Qi IEEE Transactions on Visualization and Computer Graphics, Vol. 25, Issue 1 https://doi.org/10.1109/TVCG.2018.2864850	journal	January 2019
A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization Hoang, Duong; Klacansky, Pavol; Bhatia, Harsh IEEE Transactions on Visualization and Computer Graphics, Vol. 25, Issue 1 https://doi.org/10.1109/TVCG.2018.2864853	journal	January 2019
TTHRESH: Tensor Compression for Multidimensional Visual Data Ballester-Ripoll, Rafael; Lindstrom, Peter; Pajarola, Renato IEEE Transactions on Visualization and Computer Graphics, Vol. 26, Issue 9 https://doi.org/10.1109/TVCG.2019.2904063	journal	September 2020
TSR-TVD: Temporal Super-Resolution for Time-Varying Data Analysis and Visualization Han, Jun; Wang, Chaoli IEEE Transactions on Visualization and Computer Graphics https://doi.org/10.1109/TVCG.2019.2934255	journal	January 2019
InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations He, Wenbin; Wang, Junpeng; Guo, Hanqi IEEE Transactions on Visualization and Computer Graphics https://doi.org/10.1109/TVCG.2019.2934312	journal	January 2019
Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution Weiss, Sebastian; Chu, Mengyu; Thuerey, Nils IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 6 https://doi.org/10.1109/TVCG.2019.2956697	journal	June 2021
A Fluid Flow Data Set for Machine Learning and its Application to Neural Flow Map Interpolation Jakob, Jakob; Gross, Markus; Gunther, Tobias IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 2 https://doi.org/10.1109/TVCG.2020.3028947	journal	February 2021
Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution Hoang, Duong; Summa, Brian; Bhatia, Harsh IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 2 https://doi.org/10.1109/TVCG.2020.3030381	journal	February 2021
SSR-TVD: Spatial Super-Resolution for Time-Varying Data Analysis and Visualization Han, Jun; Wang, Chaoli IEEE Transactions on Visualization and Computer Graphics https://doi.org/10.1109/TVCG.2020.3032123	journal	January 2020
Learning Adaptive Sampling and Reconstruction for Volume Visualization Weiss, Sebastian; Isk, Mustafa; Thies, Justus IEEE Transactions on Visualization and Computer Graphics, Vol. 28, Issue 7 https://doi.org/10.1109/TVCG.2020.3039340	journal	July 2022
STNet: An End-to-End Generative Framework for Synthesizing Spatiotemporal Super-Resolution Volumes Han, Jun; Zheng, Hao; Chen, Danny Z. IEEE Transactions on Visualization and Computer Graphics, Vol. 28, Issue 1 https://doi.org/10.1109/TVCG.2021.3114815	journal	January 2022
AMM: Adaptive Multilinear Meshes Bhatia, Harsh; Hoang, Duong; Morrical, Nate IEEE Transactions on Visualization and Computer Graphics, Vol. 28, Issue 6 https://doi.org/10.1109/TVCG.2022.3165392	journal	June 2022
Simulating water and smoke with an octree data structure Losasso, Frank; Gibou, Frédéric; Fedkiw, Ron ACM SIGGRAPH 2004 Papers https://doi.org/10.1145/1186562.1015745	conference	August 2004
Octrees for faster isosurface generation Wilhelms, Jane; Van Gelder, Allen ACM Transactions on Graphics, Vol. 11, Issue 3 https://doi.org/10.1145/130881.130882	journal	July 1992
Generic objective vortices for flow visualization Günther, Tobias; Gross, Markus; Theisel, Holger ACM Transactions on Graphics, Vol. 36, Issue 4 https://doi.org/10.1145/3072959.3073684	journal	July 2017
Volume upscaling with convolutional neural networks Zhou, Zhenglei; Hou, Yule; Wang, Qirui Proceedings of the Computer Graphics International Conference https://doi.org/10.1145/3095140.3095178	conference	June 2017
CPU volume rendering of adaptive mesh refinement data Wald, Ingo; Brownlee, Carson; Usher, Will SIGGRAPH Asia 2017 Symposium on Visualization on - SA '17 https://doi.org/10.1145/3139295.3139305	conference	January 2017
tempoGAN: a temporally coherent, volumetric GAN for super-resolution fluid flow Xie, You; Franz, Erik; Chu, Mengyu ACM Transactions on Graphics, Vol. 37, Issue 4 https://doi.org/10.1145/3197517.3201304	journal	August 2018
Acorn Martel, Julien N. P.; Lindell, David B.; Lin, Connor Z. ACM Transactions on Graphics, Vol. 40, Issue 4 https://doi.org/10.1145/3450626.3459785	journal	July 2021
AMReX: a framework for block-structured adaptive mesh refinement Zhang, Weiqun; Almgren, Ann; Beckner, Vince Journal of Open Source Software, Vol. 4, Issue 37 https://doi.org/10.21105/joss.01370	journal	May 2019
The LAVA Computational Fluid Dynamics Solver Kiris, Cetin C.; Barad, Michael F.; Housman, Jeffrey A. 52nd Aerospace Sciences Meeting https://doi.org/10.2514/6.2014-0070	conference	January 2014
Homogeneous Buoyancy Driven Turbulence Data Set Livescu, Daniel; Canada, Curtis; Kanov, Kalin Johns Hopkins Turbulence Databases https://doi.org/10.7281/t1vx0dpc	dataset	January 2014

Similar Records

Strictly Enforcing Invertibility and Conservation in CNN-Based Super Resolution for Scientific Datasets

Journal Article · Sat Dec 31 19:00:00 EST 2022 · Artificial Intelligence for the Earth Systems · OSTI ID:1958686

Hierarchical Volume Representation with 3{radical}2 Subdivision and Trivariate B-Spline Wavelets

Conference · Thu Jan 10 23:00:00 EST 2002 · OSTI ID:15006860

Related Subjects

97 MATHEMATICS AND COMPUTING
computational modeling
computer graphics
data models
deep learning
hierarchical data
interpolation
isosuraces
octrees
rendering
spatial resolution
super-resolution

Deep Hierarchical Super Resolution for Scientific Data

Citation Formats

References (57)

Similar Records

Related Subjects