Depth upsampling method using the confidence map for a fusion of a high resolution color sensor and low resolution time-of-flight depth sensor

Kwanghyuk Bae; Kyu-Min Kyung; Tae-Chan Kim

doi:10.1117/12.872163

27 January 2011 Depth upsampling method using the confidence map for a fusion of a high resolution color sensor and low resolution time-of-flight depth sensor

Kwanghyuk Bae, Kyu-Min Kyung, Tae-Chan Kim

Author Affiliations +

Proceedings Volume 7864, Three-Dimensional Imaging, Interaction, and Measurement; 786406 (2011) https://doi.org/10.1117/12.872163
Event: IS&T/SPIE Electronic Imaging, 2011, San Francisco Airport, California, United States

Abstract

This paper proposes a depth up-sampling method using the confidence map for a fusion of a high resolution color sensor and low resolution time-of-flight depth sensor. The confidence map represents the accuracy of depth depending on the reflectance of a measured object and is estimated with amplitude, offset, and reconstructed error of a received signal. The proposed method suppresses the depth artifacts that are caused by difference between low and high reflective materials on an object at a distance. Although the surface of an object is located at the same distance, the reflectance of small regions within the surface depends on constituent materials. Weighted filter generated by confidence map is added to the modified noise-aware filter for depth up-sampling that is proposed by Derek et al., and is adaptively selected. The proposed method consists of followings; the normalization, the reconstruction, the confidence map estimation, and the modified noise-aware filtering. In the normalization, amplitudes and offsets of received signals are calculated and received signals are normalized by those. The phase shifts are measured between transmitted and received signals. In the reconstruction, received signals are reconstructed using only the values of phase shifts and the reconstruction errors are calculated. The confidence map is estimated with amplitudes, offsets, and reconstruction errors. The coefficients of a modified noise-aware filter are adaptively selected by referring to the confidence map. The proposed method shows the enhanced results of removing depth artifacts in the experiments.

Citation Download Citation

Kwanghyuk Bae, Kyu-Min Kyung, and Tae-Chan Kim "Depth upsampling method using the confidence map for a fusion of a high resolution color sensor and low resolution time-of-flight depth sensor", Proc. SPIE 7864, Three-Dimensional Imaging, Interaction, and Measurement, 786406 (27 January 2011); https://doi.org/10.1117/12.872163

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available