Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing

Yan Dong; Chunyu Lin; Yao Zhao; Chao Yao

doi:10.1117/1.JEI.26.4.043004

11 July 2017 Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing

Yan Dong, Chunyu Lin, Yao Zhao, Chao Yao

Author Affiliations +

Journal of Electronic Imaging, Vol. 26, Issue 4, 043004 (July 2017). https://doi.org/10.1117/1.JEI.26.4.043004

Abstract

In texture-plus-depth format of three-dimensional visual data, both texture and depth maps are required to synthesize a desired view via depth-image-based rendering. However, the depth maps captured or estimated always exist with low resolution compared to their corresponding texture images. We introduce a joint edge-guided convolutional neural network that upsamples the resolution of a depth map on the premise of synthesized view quality. The network takes the low-resolution depth map as an input using a joint edge feature extracted from the depth map and the registered texture image as a reference, and then produces a high-resolution depth map. We further use local constraints that preserve smooth regions and sharp edges so as to improve the quality of the depth map and synthesized view. Finally, a global looping optimization is performed with virtual view quality as guidance in the recovery process. We train our model using pairs of depth maps and texture images and then make tests on other depth maps and video sequences. The experimental results demonstrate that our scheme outperforms existing methods both in the quality of the depth maps and synthesized views.

Citation Download Citation

Yan Dong, Chunyu Lin, Yao Zhao, and Chao Yao "Depth map upsampling using joint edge-guided convolutional neural network for virtual view synthesizing," Journal of Electronic Imaging 26(4), 043004 (11 July 2017). https://doi.org/10.1117/1.JEI.26.4.043004

Received: 24 January 2017; Accepted: 19 June 2017; Published: 11 July 2017

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available