Skip to main content
U.S. Department of Energy
Office of Scientific and Technical Information

State of the Art of Molecular Visualization in Immersive Virtual Environments

Journal Article · · Computer Graphics Forum
DOI:https://doi.org/10.1111/cgf.14738· OSTI ID:1958690
 [1];  [2];  [3];  [4];  [5];  [6];  [7];  [8]
  1. Visitlab, Masaryk University Brno Czech Republic, AIT Austrian Institute of Technology Vienna Austria
  2. ViRVIG group, Universitat Politècnica de Catalunya Barcelona Spain
  3. Université Paris‐Saclay, CNRS, Inria, LISN Paris France
  4. Department of Computer Science and Engineering Tandon School of Engineering, New York University New York United States, Big Data Visual Analytics, University of Tübingen Tübingen Germany
  5. Laboratoire de Biochimie Théorique, UPR 9080 CNRS Paris France
  6. Visitlab, Masaryk University Brno Czech Republic, Department of Informatics University of Bergen Bergen Norway
  7. Visitlab, Masaryk University Brno Czech Republic
  8. Center for Applied Scientific Computing, Lawrence Livermore National Laboratory Livermore California USA
Abstract

Visualization plays a crucial role in molecular and structural biology. It has been successfully applied to a variety of tasks, including structural analysis and interactive drug design. While some of the challenges in this area can be overcome with more advanced visualization and interaction techniques, others are challenging primarily due to the limitations of the hardware devices used to interact with the visualized content. Consequently, visualization researchers are increasingly trying to take advantage of new technologies to facilitate the work of domain scientists. Some typical problems associated with classic 2D interfaces, such as regular desktop computers, are a lack of natural spatial understanding and interaction, and a limited field of view. These problems could be solved by immersive virtual environments and corresponding hardware, such as virtual reality head‐mounted displays. Thus, researchers are investigating the potential of immersive virtual environments in the field of molecular visualization. There is already a body of work ranging from educational approaches to protein visualization to applications for collaborative drug design. This review focuses on molecular visualization in immersive virtual environments as a whole, aiming to cover this area comprehensively. We divide the existing papers into different groups based on their application areas, and types of tasks performed. Furthermore, we also include a list of available software tools. We conclude the report with a discussion of potential future research on molecular visualization in immersive environments.

Research Organization:
Lawrence Livermore National Laboratory (LLNL), Livermore, CA (United States)
Sponsoring Organization:
Carl Zeiss Stiftung; French National Research Agency; Ministerio de Ciencia e Innovación; Spanish Ministerio de Economía y Competitividad; USDOE; USDOE National Nuclear Security Administration (NNSA)
Grant/Contract Number:
AC52-07NA27344
OSTI ID:
1958690
Alternate ID(s):
OSTI ID: 1958692
OSTI ID: 1973195
Report Number(s):
LLNL-JRNL-847822; e14738
Journal Information:
Computer Graphics Forum, Journal Name: Computer Graphics Forum Journal Issue: 6 Vol. 42; ISSN 0167-7055
Publisher:
Wiley-BlackwellCopyright Statement
Country of Publication:
Netherlands
Language:
English

References (151)

A Protein in the palm of your hand through augmented reality journal June 2014
A practical guide to developing virtual and augmented reality exercises for teaching structural biology: VR and AR for Teaching Structural Biology journal November 2018
An accessible, open‐source mobile application for macromolecular visualization using augmented reality journal February 2020
An idea to explore: Use of augmented reality for teaching three‐dimensional biomolecular structures journal March 2020
UCSF ChimeraX: Meeting modern challenges in visualization and analysis: UCSF ChimeraX Visualization System journal September 2017
Peppy: A virtual reality environment for exploring the principles of polypeptide structure journal November 2019
Visualizing biomolecular electrostatics in virtual reality with UnityMol‐APBS journal November 2019
Immersive virtual reality in computational chemistry: Applications to the analysis of QM and MM data journal July 2016
Wielding the power of interactive molecular simulations journal December 2021
Immersive Analytics: An Introduction book January 2018
Immersive Analytics: Time to Reconsider the Value of 3D for Information Visualisation book January 2018
Collaborative Immersive Analytics book January 2018
Head Mounted Display Interaction Evaluation: Manipulating Virtual Objects in Augmented Reality book January 2019
Visual Abstraction book January 2020
Distributed, Immersive and Multi-platform Molecular Visualization for Chemistry Learning book January 2017
Immersion Versus Embodiment: Embodied Cognition for Immersive Analytics in Mixed Reality Environments book January 2018
Molecular Visualization with Supports of Interaction, Immersion and Collaboration among Geographically-Separated Research Groups book January 2011
Time Critical Computing and Rendering of Molecular Surfaces Using a Zonal Map book January 2000
Interactive Immersive Virtual Reality to Enhance Students’ Visualisation of Complex Molecules book January 2019
Use of Virtual-Reality in Teaching and Learning Molecular Biology book January 2013
Virtual Reality in Chemistry journal September 1997
Cybersickness in current-generation virtual reality head-mounted displays: systematic review and outlook journal April 2021
Learning with augmented reality: Impact of dimensionality and spatial abilities journal December 2022
LifeBrush: Painting, simulating, and visualizing dense biomolecular environments journal August 2019
VRdeo: Creating engaging educational material for asynchronous student-teacher exchange using virtual reality journal August 2021
NOMAD VR: Multiplatform virtual reality viewer for chemistry simulations journal April 2019
Development and use of augmented reality models to teach medicinal chemistry journal August 2021
An immersive journey to the molecular structure of SARS-CoV-2: Virtual reality in COVID-19 journal January 2020
Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts journal June 2020
Understanding differences between combinations of 2D and 3D input and output devices for 3D data visualization journal July 2022
Augmented Reality and Virtual Reality Displays: Perspectives and Challenges journal August 2020
Molecular Visualization on the Holodeck journal October 2018
Virtual Reality: Beyond Visualization journal March 2019
iBET: Immersive visualization of biological electron-transfer dynamics journal April 2016
ChemPreview : an augmented reality-based molecular interface journal May 2017
Development of a virtual reality platform for effective communication of structural data in drug discovery journal June 2019
Online tools to easily build virtual molecular models for display in augmented and virtual reality on the web journal July 2022
Virtual reality: A survey of enabling technologies and its applications in IoT journal March 2021
Immersive virtual reality in the age of the Metaverse: A hybrid-narrative review based on the technology affordance perspective journal June 2022
A never-ending story in the sky: The secrets of chemical evolution journal March 2020
Game-Engine-Assisted Research platform for Scientific computing (GEARS) in Virtual Reality journal January 2019
Molecular Graphics: Bridging Structural Biologists and Computer Scientists journal November 2019
Extended reality applications in industry 4.0. – A systematic literature review journal September 2022
Virtual and augmented reality for biomedical applications journal July 2021
Fast, Simple, Student Generated Augmented Reality Approach for Protein Visualization in the Classroom and Home Study journal June 2020
Interacting with Three-Dimensional Molecular Structures Using an Augmented Reality Mobile App journal August 2020
Exploring Chemistry with Wireless, PC-Less Portable Virtual Reality Laboratories journal December 2020
Developing and Using BioSIMAR, an Augmented Reality Program to Visualize and Learn about Chemical Structures in a Virtual Environment on Any Internet-Connected Device journal April 2021
MoleculARweb: A Web Site for Chemistry and Structural Biology Education through Interactive Augmented Reality out of the Box in Commodity Devices journal June 2021
Designing and Implementing VR2E2C, a Virtual Reality Remote Education for Experimental Chemistry System journal July 2021
InteraChem: Virtual Reality Visualizer for Reactive Interactive Molecular Dynamics journal October 2021
BiochemAR: An Augmented Reality Educational Tool for Teaching Macromolecular Structure and Function journal December 2019
Chemical Exploration with Virtual Reality in Organic Teaching Laboratories journal July 2019
Teaching Enzyme Catalysis Using Interactive Molecular Dynamics in Virtual Reality journal July 2019
Visualizing 3D Molecular Structures Using an Augmented Reality App journal April 2020
Real-Time Interactive Simulation and Visualization of Organic Molecules journal September 2020
Combining Virtual Reality Visualization with Ensemble Molecular Dynamics to Study Complex Protein Conformational Changes journal November 2020
Interactive Molecular Dynamics in Virtual Reality Is an Effective Tool for Flexible Substrate and Inhibitor Docking to the SARS-CoV-2 Main Protease journal November 2020
MOF-VR: A Virtual Reality Program for Performing and Visualizing Immersive Molecular Dynamics Simulations of Guest Molecules in Metal–Organic Frameworks journal February 2022
Molecular Rift: Virtual Reality for Drug Designers journal November 2015
Exploring DrugBank in Virtual Reality Chemical Space journal August 2018
Study of Commodity VR for Computational Material Sciences journal February 2019
Accessible virtual reality of biomolecular structural models using the Autodesk Molecule Viewer journal December 2017
Molecular visualization in chemistry education: the role of multidisciplinary collaboration journal January 2005
VR in chemistry, a review of scientific research on advanced atomic/molecular visualization journal January 2022
Interactive molecular dynamics in virtual reality from quantum chemistry to drug binding: An open-source multi-person framework journal June 2019
Flipping the laboratory: improving student engagement and learning outcomes in second year science courses journal October 2018
Using virtual reality for drug discovery: a promising new outlet for novel leads journal November 2018
Augmented reality in scientific visualization and communications: a new dawn of looking at antibody interactions journal July 2020
GPU-powered tools boost molecular visualization journal February 2011
VRmol: an integrative web-based virtual reality system to explore macromolecular structure journal August 2020
CSynth: an interactive modelling and visualization tool for 3D chromatin structure journal August 2020
RealityConvert: a tool for preparing 3D models of biochemical structures for augmented and virtual reality journal August 2017
iCAVE: an open source tool for visualizing biomolecular networks in 3D, stereoscopic 3D and immersive 3D journal July 2017
Development and assessment of CootVR, a virtual reality computer program for model building journal January 2021
UnityMol prototype for FAIR sharing of molecular-visualization experiences: from pictures in the cloud to collaborative virtual reality exploration in immersive 3D environments journal May 2021
Viewing geometric protein structures from inside a CAVE journal July 1996
Immersive VR for scientific visualization: a progress report journal January 2000
What do you do when two hands are not enough? interactive selection of bonds between pairs of tangible molecules conference March 2010
Visual Data Mining by Virtual Reality for Protein-Protein Interaction Networks conference October 2018
A novel virtual environment for molecular system design
  • Alsayegh, Rajab; Paramonov, Leonid; Makatsoris, Charalampos
  • 2013 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA) https://doi.org/10.1109/CIVEMSA.2013.6617392
conference July 2013
StereoChem: Augmented Reality 3D Molecular Model Visualization App for Teaching and Learning Stereochemistry conference July 2018
Learning organic chemistry with virtual reality conference October 2020
Interactive visual analytics of molecular data in immersive environments via a semantic definition of the content and the context conference March 2016
Immersive Molecular Visualization with Omnidirectional Stereoscopic Ray Tracing and Remote Rendering conference May 2016
Immersive environment for molecular visualization to interaction between research groups geographically dispersed
  • Correa, Moacyr Francischetti; Schpector, Julio Zukerman; Trevelin, Luis Carlos
  • 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2010) https://doi.org/10.1109/ISABEL.2010.5702889
conference November 2010
A Collaborative Virtual Reality Environment for Molecular Biology conference July 2009
Co-Coot: A real-time collaborative tool for bio-molecular modeling and visualization conference March 2011
Interactive Drug Design in Virtual Reality conference July 2011
ProteinScanAR - An Augmented Reality Web Application for High School Education in Biomolecular Life Sciences conference July 2012
Visualizing large-scale atomistic simulations in ultra-resolution immersive environments conference October 2013
CellPAINT: Interactive Illustration of Dynamic Mesoscale Cellular Environments journal November 2018
An Immersive Visualization Study on Molecules Manipulation conference November 2017
Investigating the Nature of Students' Reasoning in Connecting Molecular Structures of Stereoisomers with their Physical Properties Using an AR App conference December 2018
Ionic and Metallic Bonding Visualization Using Augmented Reality conference December 2018
Empirical Studies in Information Visualization: Seven Scenarios journal September 2012
Effects of Immersion on Visual Analysis of Volume Data journal April 2012
Towards An Understanding of Mobile Touch Navigation in a Stereoscopic Viewing Environment for 3D Data Exploration journal May 2016
Real-Time Molecular Visualization Supporting Diffuse Interreflections and Ambient Occlusion journal January 2016
Molecular Surface Maps journal January 2017
Multiscale Visualization and Scale-Adaptive Modification of DNA Nanostructures journal January 2018
Instant Construction and Visualization of Crowded Biological Environments journal January 2018
Pondering the Concept of Abstraction in (Illustrative) Visualization journal September 2018
Survey of Immersive Analytics journal March 2021
Scale Trotter: Illustrative Visual Travels Across Negative Scales journal January 2020
HyperLabels: Browsing of Dense and Hierarchical Molecular 3D Models journal August 2021
ChemVA: Interactive Visual Analysis of Chemical Compound Similarity in Virtual Screening journal February 2021
Collaborative Work in Augmented Reality: A Survey journal January 2020
Vivern–A Virtual Environment for Multiscale Visualization and Modeling of DNA Nanostructures journal December 2022
Nanotilus: Generator of Immersive Guided-Tours in Crowded 3D Environments journal March 2023
UnityMol: interactive and ludic visual manipulation of coarse-grained RNA and other biomolecules
  • Doutreligne, Sebastien; Gageat, Cedric; Cragnolini, Tristan
  • 2015 IEEE 1st International Workshop on Virtual and Augmented Reality for Molecular Science (VARMS@IEEEVR) https://doi.org/10.1109/VARMS.2015.7151718
conference March 2015
Virtual structure reconstruction and energy estimation of a peptide from a physical tangible interface conference March 2015
Enhancing visualization of molecular simulations using sonification conference March 2015
Immersive design of DNA molecules with a tangible interface conference January 2004
The eyes have it: a task by data type taxonomy for information visualizations conference January 1996
Optimised Molecular Graphics on the HoloLens conference March 2019
Immersive sonification of protein surface conference March 2020
Visual Analysis of Biomolecular Cavities: State of the Art journal June 2016
Visualization of Biomolecular Structures: State of the Art Revisited: Visualization of Biomolecular Structures journal November 2016
Analysis of Long Molecular Dynamics Simulations Using Interactive Focus+Context Visualization journal June 2019
Tasks, Techniques, and Tools for Genomic Data Visualization journal June 2019
Visual Analysis of Large‐Scale Protein‐Ligand Interaction Data journal August 2021
Exploring 3-D virtual reality technology for spatial ability and chemistry achievement journal June 2013
Journey to the centre of the cell: Virtual reality immersion into scientific data journal November 2017
Sampling molecular conformations and dynamics in a multiuser virtual reality framework journal June 2018
Learning molecular biology by VR playing
  • Lu, Bf; Lim, Kt; Zheng, Jm
  • Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry - VRCAI '04 https://doi.org/10.1145/1044588.1044622
conference January 2004
Virtual and augmented reality as spatial ability training tools
  • Dünser, Andreas; Steinbügl, Karin; Kaufmann, Hannes
  • Proceedings of the 6th ACM SIGCHI New Zealand chapter's international conference on Computer-human interaction design centered HCI - CHINZ '06 https://doi.org/10.1145/1152760.1152776
conference January 2006
The CAVE: audio visual experience automatic virtual environment journal June 1992
A head-mounted three dimensional display conference January 1968
The ImmersaDesk and Infinity Wall projection-based virtual reality displays journal May 1997
Eukaryo conference March 2016
A compact, wide-FOV optical design for head-mounted displays conference November 2016
Exploring Data in Virtual Reality conference April 2018
LearnDNA conference April 2018
Immersive Intelligence Genomic Data Visualisation
  • Lau, Chng Wei; Nguyen, Quang Vinh; Qu, Zhonglin
  • ACSW 2019: Australasian Computer Science Week 2019, Proceedings of the Australasian Computer Science Week Multiconference https://doi.org/10.1145/3290688.3290722
conference January 2019
Towards an Understanding of Augmented Reality Extensions for Existing 3D Data Analysis Tools conference April 2020
Dynamic Virtual Proteins: Visualization, Interaction and Collaboration in Virtual Reality conference November 2019
Molecular Genomics Education Through Gamified Cell Exploration in Virtual Reality
  • Patterson, Kate; Lilja, Andrew; Arrebola, Mark
  • Proceedings of the 17th International Conference on Virtual-Reality Continuum and its Applications in Industry https://doi.org/10.1145/3359997.3365724
conference November 2019
Narupa iMD: A VR-Enabled Multiplayer Framework for Streaming Interactive Molecular Simulations conference August 2020
ProMVR - Protein Multiplayer Virtual Reality Tool conference December 2021
Seven Myths on Crowding and Peripheral Vision journal May 2020
ProteinVR: Web-based molecular visualization in virtual reality journal March 2020
Game On, Science - How Video Game Technology May Help Biologists Tackle Visualization Challenges journal March 2013
Interactive molecular dynamics in virtual reality for accurate flexible protein-ligand docking journal March 2020
Eukaryo: An AR and VR Application for Cell Biology journal January 2016
Teaching Biochemistry and Molecular Biology with Virtual Reality — Lesson Creation and Student Response journal May 2020
VSEPR Theory, An Interactive and Immersive Virtual Reality conference June 2020
Pepblock Builder VR – An Open-Source Tool for Gaming-Based Bio-Edutainment in Interactive Protein Design journal May 2021
Identifying the Characteristics of Virtual Reality Gamification for Complex Educational Topics journal September 2021
Virtual reality visual feedback for hand-controlled scanning probe microscopy manipulation of single molecules journal November 2015
3D Printing of Molecular Models journal February 2016

Similar Records

Visualizing biomolecular electrostatics in virtual reality with UnityMol‐APBS
Journal Article · Sun Nov 24 19:00:00 EST 2019 · Protein Science · OSTI ID:1575785