A Virtual Reality Visualization Tool for Neuron Tracing

Usher, Will; Klacansky, Pavol; Federer, Frederick; Bremer, Peer-Timo; Knoll, Aaron; Yarch, Jeff; Angelucci, Alessandra; Pascucci, Valerio

doi:10.1109/tvcg.2017.2744079

A Virtual Reality Visualization Tool for Neuron Tracing

Journal Article · Sun Dec 31 23:00:00 EST 2017 · IEEE Transactions on Visualization and Computer Graphics

DOI:https://doi.org/10.1109/tvcg.2017.2744079· OSTI ID:1756051

Usher, Will ^[1]; Klacansky, Pavol ^[2]; Federer, Frederick ^[3]; Bremer, Peer-Timo ^[4]; Knoll, Aaron ^[2]; Yarch, Jeff ^[3]; Angelucci, Alessandra ^[3]; Pascucci, Valerio ^[2]

Univ. of Utah, Salt Lake City, UT (United States). SCI Institute; University of Utah
Univ. of Utah, Salt Lake City, UT (United States). SCI Institute
Univ. of Utah, Salt Lake City, UT (United States). Moran Eye Institute
Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)

Tracing neurons in large-scale microscopy data is crucial to establishing a wiring diagram of the brain, which is needed to understand how neural circuits in the brain process information and generate behavior. Automatic techniques often fail for large and complex datasets, and connectomics researchers may spend weeks or months manually tracing neurons using 2D image stacks. We present a design study of a new virtual reality (VR) system, developed in collaboration with trained neuroanatomists, to trace neurons in microscope scans of the visual cortex of primates. We hypothesize that using consumer-grade VR technology to interact with neurons directly in 3D will help neuroscientists better resolve complex cases and enable them to trace neurons faster and with less physical and mental strain. We discuss both the design process and technical challenges in developing an interactive system to navigate and manipulate terabyte-sized image volumes in VR. Using a number of different datasets, we demonstrate that, compared to widely used commercial software, consumer-grade VR presents a promising alternative for scientists.

View Accepted Manuscript (DOE)

Research Organization:: Univ. of Utah, Salt Lake City, UT (United States)

Sponsoring Organization:: USDOE National Nuclear Security Administration (NNSA); National Science Foundation (NSF); National Institutes of Health (NIH); University Of Utah Research Foundation

Grant/Contract Number:: NA0002375; SC0007446; SC0010498

OSTI ID:: 1756051

Journal Information:: IEEE Transactions on Visualization and Computer Graphics, Journal Name: IEEE Transactions on Visualization and Computer Graphics Journal Issue: 1 Vol. 24; ISSN 1077-2626

Publisher:: IEEECopyright Statement

Country of Publication:: United States

Language:: English

References (37)

Neuron tracing in perspective Meijering, Erik Cytometry Part A, Vol. 77A, Issue 7 https://doi.org/10.1002/cyto.a.20895	journal	March 2010
Neuroinformatics Grand Challenges Ascoli, Giorgio A. Neuroinformatics, Vol. 6, Issue 1 https://doi.org/10.1007/s12021-008-9010-5	journal	February 2008
Proof-editing is the Bottleneck Of 3D Neuron Reconstruction: The Problem and Solutions Peng, Hanchuan; Long, Fuhui; Zhao, Ting Neuroinformatics, Vol. 9, Issue 2-3 https://doi.org/10.1007/s12021-010-9090-x	journal	December 2010
The DIADEM and Beyond Liu, Yuan Neuroinformatics, Vol. 9, Issue 2-3 https://doi.org/10.1007/s12021-011-9102-5	journal	March 2011
Automated Neuron Tracing Methods: An Updated Account Acciai, Ludovica; Soda, Paolo; Iannello, Giulio Neuroinformatics, Vol. 14, Issue 4 https://doi.org/10.1007/s12021-016-9310-0	journal	July 2016
The Visual Haptic Workbench Ikits, Milan; Brederson, J. Dean Visualization Handbook https://doi.org/10.1016/B978-012387582-2/50024-1	book	January 2005
Single-Cell Phenotyping within Transparent Intact Tissue through Whole-Body Clearing Yang, Bin; Treweek, Jennifer B.; Kulkarni, Rajan P. Cell, Vol. 158, Issue 4 https://doi.org/10.1016/j.cell.2014.07.017	journal	August 2014
Simple, Scalable Proteomic Imaging for High-Dimensional Profiling of Intact Systems Murray, Evan; Cho, Jae Hun; Goodwin, Daniel Cell, Vol. 163, Issue 6 https://doi.org/10.1016/j.cell.2015.11.025	journal	December 2015
Three-dimensional neuron tracing by voxel scooping Rodriguez, Alfredo; Ehlenberger, Douglas B.; Hof, Patrick R. Journal of Neuroscience Methods, Vol. 184, Issue 1 https://doi.org/10.1016/j.jneumeth.2009.07.021	journal	October 2009
Cell Type-Specific Structural Plasticity of Axonal Branches and Boutons in the Adult Neocortex De Paola, Vincenzo; Holtmaat, Anthony; Knott, Graham Neuron, Vol. 49, Issue 6 https://doi.org/10.1016/j.neuron.2006.02.017	journal	March 2006
Genetic Dissection of Neural Circuits Luo, Liqun; Callaway, Edward M.; Svoboda, Karel Neuron, Vol. 57, Issue 5 https://doi.org/10.1016/j.neuron.2008.01.002	journal	March 2008
Network anatomy and in vivo physiology of visual cortical neurons Bock, Davi D.; Lee, Wei-Chung Allen; Kerlin, Aaron M. Nature, Vol. 471, Issue 7337 https://doi.org/10.1038/nature09802	journal	March 2011
Wiring specificity in the direction-selectivity circuit of the retina Briggman, Kevin L.; Helmstaedter, Moritz; Denk, Winfried Nature, Vol. 471, Issue 7337 https://doi.org/10.1038/nature09818	journal	March 2011
Structural and molecular interrogation of intact biological systems Chung, Kwanghun; Wallace, Jenelle; Kim, Sung-Yon Nature, Vol. 497, Issue 7449, p. 332-337 https://doi.org/10.1038/nature12107	journal	April 2013
V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets Peng, Hanchuan; Ruan, Zongcai; Long, Fuhui Nature Biotechnology, Vol. 28, Issue 4 https://doi.org/10.1038/nbt.1612	journal	March 2010
Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis Peng, Hanchuan; Tang, Jianyong; Xiao, Hang Nature Communications, Vol. 5, Issue 1 https://doi.org/10.1038/ncomms5342	journal	July 2014
Rayburst sampling, an algorithm for automated three-dimensional shape analysis from laser scanning microscopy images Rodriguez, Alfredo; Ehlenberger, Douglas B.; Hof, Patrick R. Nature Protocols, Vol. 1, Issue 4 https://doi.org/10.1038/nprot.2006.313	journal	November 2006
The Structure of the Nervous System of the Nematode Caenorhabditis elegans White, J. G.; Southgate, E.; Thomson, J. N. Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 314, Issue 1165 https://doi.org/10.1098/rstb.1986.0056	journal	November 1986
Parallel volume rendering using binary-swap compositing No authors listed IEEE Computer Graphics and Applications, Vol. 14, Issue 4 https://doi.org/10.1109/38.291532	journal	July 1994
An open-source VAA3D plugin for real-time 3D visualization of terabyte-sized volumetric images Bria, Alessandro; Iannello, Giulio; Peng, Hanchuan 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI) https://doi.org/10.1109/ISBI.2015.7163925	conference	April 2015
Visualizing large-scale atomistic simulations in ultra-resolution immersive environments Reda, Khairi; Knoll, Aaron; Nomura, Ken-ichi 2013 IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV) https://doi.org/10.1109/LDAV.2013.6675159	conference	October 2013
Exploring the Connectome: Petascale Volume Visualization of Microscopy Data Streams Beyer, J.; Hadwiger, M.; Al-Awami, A. IEEE Computer Graphics and Applications, Vol. 33, Issue 4 https://doi.org/10.1109/MCG.2013.55	journal	July 2013
FluoRender: An application of 2D image space methods for 3D and 4D confocal microscopy data visualization in neurobiology research Wan, Yong; Otsuna, Hideo; Chien, Chi-Bin 2012 IEEE Pacific Visualization Symposium https://doi.org/10.1109/PacificVis.2012.6183592	conference	February 2012
A Semi-Automatic Computer-Microscope for the Analysis of Neuronal Morphology Glaser, E. M.; Van Der Loos, H. IEEE Transactions on Biomedical Engineering, Vol. BME-12, Issue 1 https://doi.org/10.1109/TBME.1965.4502337	journal	January 1965
Haptic Rendering of Dynamic Volumetric Data Palmerius, K. L.; Cooper, M.; Ynnerman, A. IEEE Transactions on Visualization and Computer Graphics, Vol. 14, Issue 2 https://doi.org/10.1109/TVCG.2007.70409	journal	March 2008
A Comparative Study of Desktop, Fishtank, and Cave Systems for the Exploration of Volume Rendered Confocal Data Sets No authors listed IEEE Transactions on Visualization and Computer Graphics, Vol. 14, Issue 3 https://doi.org/10.1109/TVCG.2007.70433	journal	May 2008
Volume Ray Casting with Peak Finding and Differential Sampling Knoll, A.; Hijazi, Y.; Westerteiger, R. IEEE Transactions on Visualization and Computer Graphics, Vol. 15, Issue 6 https://doi.org/10.1109/TVCG.2009.204	journal	November 2009
Local Ambient Occlusion in Direct Volume Rendering Hernell, Frida; Ljung, Patric; Ynnerman, Anders IEEE Transactions on Visualization and Computer Graphics, Vol. 16, Issue 4 https://doi.org/10.1109/TVCG.2009.45	journal	July 2010
Design Study Methodology: Reflections from the Trenches and the Stacks Sedlmair, Michael; Meyer, Miriah; Munzner, Tamara IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 12 https://doi.org/10.1109/TVCG.2012.213	journal	December 2012
Effects of Immersion on Visual Analysis of Volume Data Laha, B.; Sensharma, K.; Schiffbauer, J. D. IEEE Transactions on Visualization and Computer Graphics, Vol. 18, Issue 4 https://doi.org/10.1109/TVCG.2012.42	journal	April 2012
Effects of VR System Fidelity on Analyzing Isosurface Visualization of Volume Datasets Laha, Bireswar; Bowman, Doug A.; Socha, John J. IEEE Transactions on Visualization and Computer Graphics, Vol. 20, Issue 4 https://doi.org/10.1109/TVCG.2014.20	journal	April 2014
A Survey of Volumetric Illumination Techniques for Interactive Volume Rendering Jönsson, Daniel; Sundén, Erik; Ynnerman, Anders Computer Graphics Forum, Vol. 33, Issue 1 https://doi.org/10.1111/cgf.12252	journal	October 2013
Two-photon laser scanning fluorescence microscopy Denk, W.; Strickler, J.; Webb, W. Science, Vol. 248, Issue 4951 https://doi.org/10.1126/science.2321027	journal	April 1990
The CAVE: audio visual experience automatic virtual environment Cruz-Neira, Carolina; Sandin, Daniel J.; DeFanti, Thomas A. Communications of the ACM, Vol. 35, Issue 6 https://doi.org/10.1145/129888.129892	journal	June 1992
Advanced illumination techniques for GPU volume raycasting Hadwiger, Markus; Ljung, Patric; Salama, Christof Rezk ACM SIGGRAPH ASIA 2008 courses on - SIGGRAPH Asia '08 https://doi.org/10.1145/1508044.1508045	conference	January 2008
Global static indexing for real-time exploration of very large regular grids Pascucci, Valerio; Frank, Randall J. Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM) - Supercomputing '01 https://doi.org/10.1145/582034.582036	conference	January 2001
The incremental garbage collection of processes Baker, Henry C.; Hewitt, Carl Proceedings of the 1977 symposium on Artificial intelligence and programming languages - https://doi.org/10.1145/800228.806932	conference	January 1977

Cited By (6)

BigTop: a three-dimensional virtual reality tool for GWAS visualization Westreich, Samuel T.; Nattestad, Maria; Meyer, Christopher BMC Bioinformatics, Vol. 21, Issue 1 https://doi.org/10.1186/s12859-020-3373-5	journal	January 2020
Experimental and Computational Methods for the Study of Cerebral Organoids: A Review Poli, Daniele; Magliaro, Chiara; Ahluwalia, Arti Frontiers in Neuroscience, Vol. 13 https://doi.org/10.3389/fnins.2019.00162	journal	March 2019
TeraVR empowers precise reconstruction of complete 3-D neuronal morphology in the whole brain Wang, Yimin; Li, Qi; Liu, Lijuan Nature Communications, Vol. 10, Issue 1 https://doi.org/10.1038/s41467-019-11443-y	journal	August 2019
Classroom misbehaviour management: an SVVR-based training system for preservice teachers Ye, Xindong; Liu, Peng-Fei; Lee, Xiao-Zhi Interactive Learning Environments https://doi.org/10.1080/10494820.2019.1579235	journal	February 2019
TeraVR Empowers Precise Reconstruction of Complete 3-D Neuronal Morphology in the Whole Brain Wang, Yimin; Li, Qi; Liu, Lijuan https://doi.org/10.1101/621011	posted_content	April 2019
BigTop: A Three-Dimensional Virtual Reality tool for GWAS Visualization Westreich, Samuel T.; Nattestad, Maria; Meyer, Christopher https://doi.org/10.1101/650176	posted_content	June 2019

Similar Records

ViRPET--combination of virtual reality and PET brain imaging

Patent · Tue May 23 00:00:00 EDT 2017 · OSTI ID:1358274

ViRPET—combination of virtual reality and PET brain imaging

Patent · Tue Mar 27 00:00:00 EDT 2018 · OSTI ID:1532056

Augmented and Virtual Reality Navigation for Interventions in the Musculoskeletal System

Journal Article · Sat Sep 15 00:00:00 EDT 2018 · Current Radiology Reports · OSTI ID:22876169

Related Subjects

97 MATHEMATICS AND COMPUTING
Virtual reality
design studies
interaction design

A Virtual Reality Visualization Tool for Neuron Tracing

Citation Formats

References (37)

Cited By (6)

Similar Records

Related Subjects