MSDS Capstone Project

"Mathematical reasoning may be regarded rather schematically as the exercise of a combination of two facilities, which we may call intuition and ingenuity." -Alan Turing

      My Data Science capstone project focused on transforming real-world 3D coordinates into a virtual 3D space (i.e., point clouds) by leveraging a cutting-edge approach with dense prediction transformers and comparing results with more traditional approaches (e.g., fully connected convolutional neural networks) in order to extract, transform, and append Z-depth dimension onto the original 2D (X, Y) image. Essentially, this was an approach to reverse engineer the process of 3D coordinate locations on the 2D image so that an image’s pixels could be reprojected into voxels, which are 3D pixels. My approach was simple:

  1. obtain an image with the associated intrinsic camera matrix
  2. apply MiDaS vision transformer to obtain depth maps
  3. combine the original image RGB features with the depth map through OpenCV Q-matrix operation
  4. create a polygon file (PLY) where each row represents one pixel with six features (RGB, XYZ)
      For analysis, I compared the depth map creation from two dense prediction transformers and one convolutional neural network. Four areas to focus analysis on were production rates, outlier detection levels, Y-axis perpendicularity, and general visual point cloud comparison.
      My research paper is being submitted to (ICECCME) International Conference on Electrical, Computer, Communications and Mechatronics Engineering for 2022 conference. While this project only consisted of 8-weeks of work, I was proud of the results. If you have a virtual reality device, like Oculus, you can go to the below link and see what it's like to walk through a 2D image.
      For the most comprehensive guide into my capstone project, check out my paper and presentation. Click the below link to download the paper. Go to my GitHub repository to see my 4-part presentation PowerPoint slides. Enjoy!

2D Image to Virtual Reality: Link to Virtual Reality
MSDS Capstone Research Paper: Click HERE to download .pdf