wiki:Other/Summer/2024/aRvR

AR/VR Visualization of Data

Members: Alexander Kim, Amanda Lau, Catzby Paul, Samhitha Sangaraju, Vir Vaidya, David Zhao

Faculty advisors: Wade Trappe, Jennifer Shane, Ivan Seskar

PROJECT OVERVIEW

Protein Data Visualization

Interactive visualization of the protein structures from the Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) in AR/VR using Microsoft Hololens and Unity

Oceanographic Data Visualization

Visualizing sensor data from the Rutgers-deployed Slocum glider using the Meta Quest 3 and Unity

PROGRESS LOG

Week 1 (5/28 - 5/30)

Week 1 Slides

  • Installed and familiarized ourselves with the Unity Engine
  • Successfully connected the Hololens to the Unity engine and created a simple test program

Week 2 (6/03 - 6/06)

Week 2 Slides

Week 3 (6/10 - 6/13)

Week 3 Slides

General

  • Developed scripts for comprehensive manipulation of game objects for the Meta Quest 3, including movement (grab and hold), rotation, and resizing, to allow users to better interact with areas of interest

RCSB PDB

  • Explored .pdb files on the RCSB PDB website to get a sense of what features are available to us
  • Successfully visualized proteins through PyMol and imported models into Unity
  • Began work on direct conversion of PDB files into Unity without use of external PyMol program due to compatibility and licensing issues

RUCOOL

  • Wrote a Python script to access, search, download, and preprocess relevant datasets from Rutgers' ERDDAP database
  • Successfully imported and visualized these datasets in Unity with human assistance

Week 4 (6/17 - 6/20)

Week 4 Slides

RCSB PDB

  • No more PyMol, importing .pdb files directly into Unity
  • Finalized approach to building protein structures —> layer by layer approach
  • Visualized and refined assets of proteins in Unity, streamlining code and procedures

RUCOOL

  • Automated the process of downloading, cleaning, and visualizing glider data by pulling directly from ERDDAP into Unity
  • Added navigational functionality and toggle to accommodate for multiple variables of interest

Week 5 (6/24 - 6/27)

Week 5 Slides

RCSB PDB

  • Continued work on protein development, building atoms, helices, links, connections
  • Optimized protein generation to improve performance
  • Implemented voice commands for viewing options
    • Toggle different structures
    • Isolate specific elements
    • Rescale the protein
    • Summon the protein in front of user

RUCOOL

  • Connected plot points by interpolating color
  • Implemented a basic GUI with a minimap and a variable readout
    • Began implementation of gauge to go along with variable readout
  • Added a toggle to "follow" the path of the glider
  • Worked on creating VR-compatible gauge in Unity to display ocean data (ex: temperature, salinity, and pressure)

Week 6 (7/01 - 7/03)

Week 6 Slides

RCSB PDB

  • Cleaned up code, made it easier to read and commented it out
  • Redid Helix algorithm, allowing for improved Helix generation —> ribbon like look comparable to professional visualizations
  • Met with RCSB PDB experts and discussed goals and project direction; They suggested we look into importing structures directly from Mol*

RUCOOL

  • Continued working on the gauge for the gliders path
  • Worked on adding references to the needle game object
  • Attached the gauge script to the needle game object
  • Ran into some issues regarding the rotation of the needle relative to the dataset.
  • Realized that the problem was regarding the pivot point of the needles and its position on the gauge and fix that.

Week 7 (7/8 - 7/11)

Week 7 Slides

RCSB PDB

  • Explored Mol* as advised, but realized it was not compatible with our approach —> sticking to building proteins from scratch
  • Did some research into amino acid ribbons (wavy line-like structures going through the proteins in professional visualizations) and created them ourselves
  • Finished creation of the proteins (all major elements visualized!)

RUCOOL

  • Took a field trip to the RUCOOL lab!
    • Learned about the gliders
    • Got guidance on anomalies to explore
  • Implemented quality of life improvements (new colormaps, GUI variable readout)
  • Ran into issues when trying to get the gauge needle to receive the data from different variable datas
  • Created and inserted CAD model of Slocum Glider
  • Modeled ocean floor and topography using bathymetry data

Week 8 (7/15 - 7/18)

Week 8 Slides

RCSB PDB

  • Coded a web scraper that can fetch PDB and CIF files directly from the RCSB PDB website.
    • No longer have to download individual files
  • Created a user interface with a holographic keyboard, allowing users to search for proteins and load them in directly

RUCOOL

  • Added glider and bathymetry data into unified project
    • Glider follows path in real-time
    • Added toggle for both pausing and free-explore
  • Added toggleable menu/help guide in GUI

Week 9 (7/22 - 7/25)

Week 9 Slides

RCSB PDB

  • Implemented CIF file functionality
    • Previously, we could only load in PDB files, but some proteins are incomplete and have CIF files instead
  • Optimized program to decrease load time of proteins
    • Takes about 1/3 of the time to load proteins now

RUCOOL

  • Fixed UI Overlay and world projection
  • Implemented some features requested by the RUCOOL team (Minimap location indicator and path and solid display modes)

Week 10 (7/29 - 8/01)

Week 10 Slides

RCSB PDB

  • Improved loading times for larger proteins
  • Demo Video for final Presentation
  • Commented code and uploaded to GitLab

RUCOOL

  • Implemented the ability for the user to create a subset and save new data in the VR environment
  • Wrote documentation for our code
  • Made poster and demo video for open house
Last modified 2 months ago Last modified on Aug 6, 2024, 3:43:34 PM
Note: See TracWiki for help on using the wiki.