A novel region-based approach for the fusion of combined stereo and spectral series

I. Gheta; S. Höfer; M. Heizmann; J. Beyerer

doi:10.1117/12.840166

28 January 2010 A novel region-based approach for the fusion of combined stereo and spectral series

I. Gheta, S. Höfer, M. Heizmann, J. Beyerer

Author Affiliations +

Proceedings Volume 7538, Image Processing: Machine Vision Applications III; 75380G (2010) https://doi.org/10.1117/12.840166
Event: IS&T/SPIE Electronic Imaging, 2010, San Jose, California, United States

Abstract

This contribution proposes a novel approach for image fusion of combined stereo and spectral series acquired simultaneously with a camera array. To this purpose, nine cameras are equipped with spectral filters (50 nm spectral bandwidth) such that the visible and near infrared parts of the spectrum (400-900 nm) are observed. The resulting image series is fused in order to obtain two types of information: the 3D shape of the scene and its spectral properties. For the registration of the images, a novel region based registration approach which evaluates the gray value invariant features (e.g. edges) of regions in segmented images is proposed. The registration problem is formulated by means of energy functionals. The data term of our functional compares features of a region in one image with features of an area in another image, such that an additional independency of the form and size of the regions in the segmented images is obtained. As regularization, a smoothness term is proposed, which models the fact that disparity discontinuities should only occur at edges in the images. In order to minimize the energy functional, we use graph cuts. The minimization is carried out simultaneously over all image pairs in the series. Even though the approach is region based, a label (e.g. disparity) is assigned to each pixel. The result of the minimization approach consists of a disparity map. By means of calibration, we use the disparity map to compute a depth map. Once pixel depths are determined, the images can be warped to a common view, such that a pure spectral series is obtained. This can be used to classify different materials of the objects in the scene based on real spectral information, which cannot be acquired with a common RGB camera.

Citation Download Citation

I. Gheta, S. Höfer, M. Heizmann, and J. Beyerer "A novel region-based approach for the fusion of combined stereo and spectral series", Proc. SPIE 7538, Image Processing: Machine Vision Applications III, 75380G (28 January 2010); https://doi.org/10.1117/12.840166

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE