logo Idiap Research Institute        
 [BibTeX] [Marc21]
DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation
Type of publication: Conference paper
Citation: Johari_ICCV_2021
Publication status: Accepted
Booktitle: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)
Year: 2021
Month: October
Pages: 6039-6048
URL: https://openaccess.thecvf.com/...
Abstract: We present DepthInSpace, a self-supervised deep-learning method for depth estimation using a structured-light camera. The design of this method is motivated by the commercial use case of embedded depth sensors in nowadays smartphones. We first propose to use estimated optical flow from ambient information of multiple video frames as a complementary guide for training a single-frame depth estimation network, helping to preserve edges and reduce over-smoothing issues. Utilizing optical flow, we also propose to fuse the data of multiple video frames to get a more accurate depth map. In particular, fused depth maps are more robust in occluded areas and incur less in flying pixels artifacts. We finally demonstrate that these more precise fused depth maps can be used as self-supervision for fine-tuning a single-frame depth estimation network to improve its performance. Our models' effectiveness is evaluated and compared with state-of-the-art models on both synthetic and our newly introduced real datasets. The implementation code, training procedure, and both synthetic and captured real datasets are available at https://www.idiap.ch/paper/depthinspace.
Projects Idiap
Authors Johari, Mohammad Mahdi
Carta, Camilla
Fleuret, Francois
Added by: [UNK]
Total mark: 0