3D computer vision for virtual reality and robotics
3D computer vision involves teaching the computer to understand the three-dimensional world around it. Because images are only two-dimensional projections of the three-dimensional world, the computer must use intelligent strategies to infer the missing third dimension in an image. Similarly, video captured from a moving camera gives clues about the trajectory of the camera, but some form of intelligence is needed to infer the motion from the video alone.
These types of 3D computer vision problems have great relevance to many interesting applications such as robotics, augmented reality, and virtual reality. In my research I have focused on three main areas of 3D computer vision:
- Fast solvers for ego-motion estimation
- Camera localization and tracking in large areas
- Unsupervised learning of 3D reconstruction from panoramic video
Sponsor: National Science Foundation
Recent projects:
3D Pano Inpainting: Building a VR Environment from a Single Input Panorama (VR 2023 Poster)
Absolute Pose From One or Two Scaled and Oriented Features (CVPR 2024)
P1AC: Revisiting Absolute Pose from a Single Affine Correspondence (ICCV 2023)
PanoSynthVR: View Synthesis from a Single Input Panorama with Multi-Cylinder Images (ISMAR 2022)
CasualStereo: Casual Capture of Stereo Panoramas with Spherical Structure-from-Motion (VR 2020)
Unsupervised Learning of Depth and Ego-Motion from Cylindrical Panoramic Video (AIVR 2019)