3D computer vision for virtual reality and robotics

3D computer vision involves teaching the computer to understand the three-dimensional world around it. Because images are only two-dimensional projections of the three-dimensional world, the computer must use intelligent strategies to infer the missing third dimension in an image. Similarly, video captured from a moving camera gives clues about the trajectory of the camera, but some form of intelligence is needed to infer the motion from the video alone.

These types of 3D computer vision problems have great relevance to many interesting applications such as robotics, augmented reality, and virtual reality. In my research I have focused on three main areas of 3D computer vision:

  • Fast solvers for ego-motion estimation
  • Camera localization and tracking in large areas
  • Unsupervised learning of 3D reconstruction from panoramic video

Sponsor: National Science Foundation

Recent projects:

PanoSynthVR: View Synthesis from a Single Input Panorama with Multi-Cylinder Images

View Synthesis In Casually Captured Scenes Using a Cylindrical Neural Radiance Field With Exposure Compensation

Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video

CasualStereo: Casual Capture of Stereo Panoramas with Spherical Structure-from-Motion