FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

Bauer, David; Wu, Qi; Ma, Kwan-Liu

doi:10.1109/tvcg.2022.3209498

FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

Journal Article · 31 December 2022 · IEEE Transactions on Visualization and Computer Graphics

DOI:https://doi.org/10.1109/tvcg.2022.3209498· OSTI ID:2417700

Bauer, David ^[1]; Wu, Qi ^[2]; Ma, Kwan-Liu ^[2]

University of California at Davis, USA; OSTI
University of California at Davis, USA

Not provided.

Research Organization:: Univ. of California, Davis, CA (United States)

Sponsoring Organization:: USDOE Office of Science (SC)

DOE Contract Number:: SC0019486

OSTI ID:: 2417700

Journal Information:: IEEE Transactions on Visualization and Computer Graphics, Journal Name: IEEE Transactions on Visualization and Computer Graphics Journal Issue: 1 Vol. 29; ISSN 1077-2626

Publisher:: IEEE

Country of Publication:: United States

Language:: English

References (52)

Fast global illumination for interactive volume visualization Zhang, Yubo; Ma, Kwan-Liu Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games - I3D '13 https://doi.org/10.1145/2448196.2448205	conference	January 2013
Deep convolutional reconstruction for gradient-domain rendering Kettunen, Markus; Härkönen, Erik; Lehtinen, Jaakko ACM Transactions on Graphics, Vol. 38, Issue 4 https://doi.org/10.1145/3306346.3323038	journal	July 2019
Interactive Volume Rendering with Dynamic Ambient Occlusion and Color Bleeding Ropinski, Timo; Meyer‐Spradow, Jennis; Diepenbrock, Stefan Computer Graphics Forum, Vol. 27, Issue 2 https://doi.org/10.1111/j.1467-8659.2008.01154.x	journal	April 2008
Foveated Real‐Time Ray Tracing for Head‐Mounted Displays Weier, Martin; Roth, Thorsten; Kruijff, Ernst Computer Graphics Forum, Vol. 35, Issue 7 https://doi.org/10.1111/cgf.13026	journal	October 2016
A Directional Occlusion Shading Model for Interactive Direct Volume Rendering Schott, Mathias; Pegoraro, Vincent; Hansen, Charles Computer Graphics Forum, Vol. 28, Issue 3 https://doi.org/10.1111/j.1467-8659.2009.01464.x	journal	June 2009
Quantizing deep convolutional networks for efficient inference: A whitepaper Krishnamoorthi, Raghuraman arXiv https://doi.org/10.48550/arXiv.1806.08342	preprint	January 2018
DMCR-GAN: Adversarial Denoising for Monte Carlo Renderings with Residual Attention Networks and Hierarchical Features Modulation of Auxiliary Buffers Lu, YiFan; Xie, Ning; Shen, Heng Tao SIGGRAPH Asia 2020 Technical Communications https://doi.org/10.1145/3410700.3425426	conference	November 2020
Robust deep residual denoising for Monte Carlo rendering Wong, Kin-Ming; Wong, Tien-Tsin SIGGRAPH Asia 2018 Technical Briefs https://doi.org/10.1145/3283254.3283261	conference	December 2018
Parallel distributed, GPU-accelerated, advanced lighting calculations for large-scale volume visualization Shih, Min; Rizzi, Silvio; Insley, Joseph 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV) https://doi.org/10.1109/LDAV.2016.7874309	conference	October 2016
Spatiotemporal Blue Noise Masks Wolfe, Alan; Morrical, Nathan; Akenine-Möller, Tomas The Eurographics Association https://doi.org/10.2312/sr.20221161	conferencepaper	January 2022
Local and Global Illumination in the Volume Rendering Integral Max, Nelson; Chen, Min Schloss Dagstuhl – Leibniz-Zentrum für Informatik https://doi.org/10.4230/DFU.SciViz.2010.259	bookchapter	January 2010
Generative adversarial networks Goodfellow, Ian; Pouget-Abadie, Jean; Mirza, Mehdi Communications of the ACM, Vol. 63, Issue 11 https://doi.org/10.1145/3422622	journal	October 2020
Deep Volumetric Ambient Occlusion Engel, Dominik; Ropinski, Timo IEEE Transactions on Visualization and Computer Graphics, Vol. 27, Issue 2 https://doi.org/10.1109/TVCG.2020.3030344	journal	February 2021
Neural Denoising for Path Tracing of Medical Volumetric Data Hofmann, Nikolai; Martschinke, Jana; Engel, Klaus Proceedings of the ACM on Computer Graphics and Interactive Techniques, Vol. 3, Issue 2 https://doi.org/10.1145/3406181	journal	August 2020
Sample-based Monte Carlo denoising using a kernel-splatting network Gharbi, Michaël; Li, Tzu-Mao; Aittala, Miika ACM Transactions on Graphics, Vol. 38, Issue 4 https://doi.org/10.1145/3306346.3322954	journal	July 2019
Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering Stengel, Michael; Grogorick, Steve; Eisemann, Martin Computer Graphics Forum, Vol. 35, Issue 4 https://doi.org/10.1111/cgf.12956	journal	July 2016
Out-of-core visualization of time-varying hybrid-grid volume data No authors listed 2014 IEEE 4th Symposium on Large Data Analysis and Visualization (LDAV) https://doi.org/10.1109/LDAV.2014.7013209	conference	November 2014
Exposure Render: An Interactive Photo-Realistic Volume Rendering Framework Kroes, Thomas; Post, Frits H.; Botha, Charl P. PLoS ONE, Vol. 7, Issue 7 https://doi.org/10.1371/journal.pone.0038586	journal	July 2012
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Jacob, Benoit; Kligys, Skirmantas; Chen, Bo 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) https://doi.org/10.1109/CVPR.2018.00286	conference	June 2018
FovVideoVDP Mantiuk, Rafał K.; Denes, Gyorgy; Chapiro, Alexandre ACM Transactions on Graphics, Vol. 40, Issue 4 https://doi.org/10.1145/3450626.3459831	journal	July 2021
Image Plane Sweep Volume Illumination Sunden, E.; Ynnerman, A.; Ropinski, T. IEEE Transactions on Visualization and Computer Graphics, Vol. 17, Issue 12 https://doi.org/10.1109/TVCG.2011.211	journal	December 2011
Progressive light volume for interactive volumetric illumination Liu, Ning; Zhu, Dengming; Wang, Zhaoqi Computer Animation and Virtual Worlds, Vol. 27, Issue 3-4 https://doi.org/10.1002/cav.1706	journal	May 2016
Kernel-predicting convolutional networks for denoising Monte Carlo renderings Bako, Steve; Vogels, Thijs; Mcwilliams, Brian ACM Transactions on Graphics, Vol. 36, Issue 4 https://doi.org/10.1145/3072959.3073708	journal	July 2017
A survey on deep learning-based Monte Carlo denoising Huo, Yuchi; Yoon, Sung-eui Computational Visual Media, Vol. 7, Issue 2 https://doi.org/10.1007/s41095-021-0209-9	journal	March 2021
A Survey of GPU-Based Large-Scale Volume Visualization Beyer, Johanna; Hadwiger, Markus; Pfister, Hanspeter The Eurographics Association https://doi.org/10.2312/eurovisstar.20141175	null	January 2014
From Pairwise Comparisons and Rating to a Unified Quality Scale Perez-Ortiz, Maria; Mikhailiuk, Aliaksei; Zerman, Emin IEEE Transactions on Image Processing, Vol. 29 https://doi.org/10.1109/TIP.2019.2936103	journal	January 2020
A Multidirectional Occlusion Shading Model for Direct Volume Rendering Šoltészová, Veronika; Patel, Daniel; Bruckner, Stefan Computer Graphics Forum, Vol. 29, Issue 3 https://doi.org/10.1111/j.1467-8659.2009.01695.x	journal	June 2010
Massively parallel Lucas Kanade optical flow for real-time video processing applications Plyer, Aurélien; Le Besnerais, Guy; Champagnat, Frédéric Journal of Real-Time Image Processing, Vol. 11, Issue 4 https://doi.org/10.1007/s11554-014-0423-0	journal	April 2014
Perception-driven Accelerated Rendering Weier, M.; Stengel, M.; Roth, T. Computer Graphics Forum, Vol. 36, Issue 2 https://doi.org/10.1111/cgf.13150	journal	May 2017
Two-Frame Motion Estimation Based on Polynomial Expansion Farnebäck, Gunnar Image Analysis https://doi.org/10.1007/3-540-45103-X_50	book	January 2003
DeepFovea Kaplanyan, Anton S.; Sochenov, Anton; Leimkühler, Thomas ACM Transactions on Graphics, Vol. 38, Issue 6 https://doi.org/10.1145/3355089.3356557	journal	November 2019
Dense optical flow by iterative local window registration Le Besnerais, G.; Champagnat, F. IEEE International Conference on Image Processing 2005 https://doi.org/10.1109/ICIP.2005.1529706	conference	January 2005
Obscurance-based Volume Rendering Framework Ruiz, M.; Boada, I.; Viola, I. The Eurographics Association https://doi.org/10.2312/VG/VG-PBG08/113-120	null	January 2008
Cinematic rendering – an alternative to volume rendering for 3D computed tomography imaging Dappa, Evelyn; Higashigaito, Kai; Fornaro, Jürgen Insights into Imaging, Vol. 7, Issue 6 https://doi.org/10.1007/s13244-016-0518-1	journal	September 2016
U-Net: Convolutional Networks for Biomedical Image Segmentation Ronneberger, Olaf; Fischer, Philipp; Brox, Thomas Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III https://doi.org/10.1007/978-3-319-24574-4_28	book	November 2015
TV-L1 Optical Flow Estimation Sánchez Pérez, Javier; Meinhardt-Llopis, Enric; Facciolo, Gabriele Image Processing On Line, Vol. 3 https://doi.org/10.5201/ipol.2013.26	journal	January 2013
Optical models for direct volume rendering Max, N. IEEE Transactions on Visualization and Computer Graphics, Vol. 1, Issue 2 https://doi.org/10.1109/2945.468400	journal	June 1995
A reduced-precision network for image reconstruction Thomas, Manu Mathew; Vaidyanathan, Karthik; Liktor, Gabor ACM Transactions on Graphics, Vol. 39, Issue 6 https://doi.org/10.1145/3414685.3417786	journal	November 2020
Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder Chaitanya, Chakravarty R. Alla; Kaplanyan, Anton S.; Schied, Christoph ACM Transactions on Graphics, Vol. 36, Issue 4 https://doi.org/10.1145/3072959.3073601	journal	July 2017
Foveated 3D graphics Guenter, Brian; Finch, Mark; Drucker, Steven ACM Transactions on Graphics, Vol. 31, Issue 6 https://doi.org/10.1145/2366145.2366183	journal	November 2012
Learning Adaptive Sampling and Reconstruction for Volume Visualization Weiss, Sebastian; Isk, Mustafa; Thies, Justus IEEE Transactions on Visualization and Computer Graphics, Vol. 28, Issue 7 https://doi.org/10.1109/TVCG.2020.3039340	journal	July 2022
Denoising with kernel prediction and asymmetric loss functions Vogels, Thijs; Rousselle, Fabrice; Mcwilliams, Brian ACM Transactions on Graphics, Vol. 37, Issue 4 https://doi.org/10.1145/3197517.3201388	journal	July 2018
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Zhang, Richard; Isola, Phillip; Efros, Alexei A. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition https://doi.org/10.1109/CVPR.2018.00068	conference	June 2018
Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation Xu, Bing; Zhang, Junfei; Wang, Rui ACM Transactions on Graphics, Vol. 38, Issue 6 https://doi.org/10.1145/3355089.3356547	journal	November 2019
Multi-Material Volume Rendering with a Physically-Based Surface Reflection Model Igouchkine, Oleg; Zhang, Yubo; Ma, Kwan-Liu IEEE Transactions on Visualization and Computer Graphics, Vol. 24, Issue 12 https://doi.org/10.1109/TVCG.2017.2784830	journal	December 2018
A model for volume lighting and modeling Kniss, J.; Premoze, S.; Hansen, C. IEEE Transactions on Visualization and Computer Graphics, Vol. 9, Issue 2 https://doi.org/10.1109/TVCG.2003.1196003	journal	April 2003
Neural Temporal Adaptive Sampling and Denoising Hasselgren, J.; Munkberg, J.; Salvi, M. Computer Graphics Forum, Vol. 39, Issue 2 https://doi.org/10.1111/cgf.13919	journal	May 2020
OSPRay - A CPU Ray Tracing Framework for Scientific Visualization Wald, I.; Johnson, Gp; Amstutz, J. IEEE Transactions on Visualization and Computer Graphics, Vol. 23, Issue 1 https://doi.org/10.1109/TVCG.2016.2599041	journal	January 2017
Interactive volumetric lighting simulating scattering and shadowing Ropinski, Timo; Doring, Christian; Rezk-Salama, Christof 2010 IEEE Pacific Visualization Symposium (PacificVis) https://doi.org/10.1109/PACIFICVIS.2010.5429594	conference	March 2010
Real-time ambient occlusion and halos with Summed Area Tables Díaz, José; Vázquez, Pere-Pau; Navazo, Isabel Computers & Graphics, Vol. 34, Issue 4 https://doi.org/10.1016/j.cag.2010.03.005	journal	August 2010
Local Ambient Occlusion in Direct Volume Rendering Hernell, Frida; Ljung, Patric; Ynnerman, Anders IEEE Transactions on Visualization and Computer Graphics, Vol. 16, Issue 4 https://doi.org/10.1109/TVCG.2009.45	journal	July 2010
Interactive Volume Exploration of Petascale Microscopy Data Streams Using a Visualization-Driven Virtual Memory Approach Hadwiger, M.; Beyer, J. IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 12 https://doi.org/10.1109/TVCG.2012.240	journal	December 2012

Similar Records

Background rejection in NEXT using deep neural networks

Journal Article · 2017 · Journal of Instrumentation · OSTI ID:1346933

Bond order predictions using deep neural networks

Journal Article · 2021 · Journal of Applied Physics · OSTI ID:1805727

Related Subjects

Computer Science

FoVolNet: Fast Volume Rendering using Foveated Deep Neural Networks

Citation Formats

References (52)

Similar Records

Related Subjects