17 November 2017 Indoor calibration for stereoscopic camera STC: a new method
Author Affiliations +
Proceedings Volume 10563, International Conference on Space Optics — ICSO 2014; 105634E (2017) https://doi.org/10.1117/12.2304188
Event: International Conference on Space Optics — ICSO 2014, 2014, Tenerife, Canary Islands, Spain
Abstract
In the framework of the ESA-JAXA BepiColombo mission to Mercury, the global mapping of the planet will be performed by the on-board Stereo Camera (STC), part of the SIMBIO-SYS suite [1]. In this paper we propose a new technique for the validation of the 3D reconstruction of planetary surface from images acquired with a stereo camera.

STC will provide a three-dimensional reconstruction of Mercury surface. The generation of a DTM of the observed features is based on the processing of the acquired images and on the knowledge of the intrinsic and extrinsic parameters of the optical system.

The new stereo concept developed for STC needs a pre-flight verification of the actual capabilities to obtain elevation information from stereo couples: for this, a stereo validation setup to get an indoor reproduction of the flight observing condition of the instrument would give a much greater confidence to the developed instrument design.

STC is the first stereo satellite camera with two optical channels converging in a unique sensor. Its optical model is based on a brand new concept to minimize mass and volume and to allow push-frame imaging. This model imposed to define a new calibration pipeline to test the reconstruction method in a controlled ambient. An ad-hoc indoor set-up has been realized for validating the instrument designed to operate in deep space, i.e. in-flight STC will have to deal with source/target essentially placed at infinity.

This auxiliary indoor setup permits on one side to rescale the stereo reconstruction problem from the operative distance in-flight of 400 km to almost 1 meter in lab; on the other side it allows to replicate different viewing angles for the considered targets.

Neglecting for sake of simplicity the Mercury curvature, the STC observing geometry of the same portion of the planet surface at periherm corresponds to a rotation of the spacecraft (SC) around the observed target by twice the 20° separation of each channel with respect to nadir. The indoor simulation of the SC trajectory can therefore be provided by two rotation stages to generate a dual system of the real one with same stereo parameters but different scale.

The set of acquired images will be used to get a 3D reconstruction of the target: depth information retrieved from stereo reconstruction and the known features of the target will allow to get an evaluation of the stereo system performance both in terms of horizontal resolution and vertical accuracy.

To verify the 3D reconstruction capabilities of STC by means of this stereo validation set-up, the lab target surface should provide a reference, i.e. should be known with an accuracy better than that required on the 3D reconstruction itself. For this reason, the rock samples accurately selected to be used as lab targets have been measured with a suitable accurate 3D laser scanner.

The paper will show this method in detail analyzing all the choices adopted to lead back a so complex system to the indoor solution for calibration.
Simioni, Re, Deppo, Naletto, Borrelli, Dami, Veltroni, and Cremonese: INDOOR CALIBRATION FOR STEREOSCOPIC CAMERA STC, A NEW METHOD

I.

INTRODUCTION

The Stereo Camera (STC) [2] is the first stereoptic satellite push-frame camera designed for the study of Mercury. STC is onboard of the satellite BepiColombo. Scope of the mission is the study the surface composition and morphology, the geology and the magnetosphere of the planet. Many objectives will be provided by BepiColombo Mission: global mapping with stereo imaging and spectroscopy, color mapping with multi-band filters and hyperspectral imaging of selected regions and high spatial resolution imaging of at least 20% of the planet surface.

II.

THE STEREO CAMERA (STC)

STC has a brand-new optical design (see Fig. 1), it’s a catadrioptic solution which allows to supply two optical wide angle channels with the orientation of +20° and -20° from the nadir direction maintaining limited mass and power. The solution to this minimization is the use of a common detector for both the channels and the adoption of mirrors to have the two different perspectives re-directed almost on the same optical path. [3]

Fig. 1

STC optical layout. In (a) the configuration is viewed in the plane defined by the along track and nadir directions; in (b) the projection in the orthogonal plane, the one including across track and nadir directions, is depicted. In the inset, an enlarged view of the focal plane region helps to better follow the rays optical path.

00151_PSISDG10563_105634E_page_3_1.jpg

On the other side, five different filters in 410-930 nm spectral range permit multi-band acquisitions. Panchromatic ones are used for acquire stereo-pair guaranteeing resulting DTM (Digital Terrain Models) with a stereo vertical accuracy of 80 meters and a grid size of 50-110 m/pixel at periherm over the equator.

A.

Optical design

Six months will be the maximum period in which global stereo mapping, first scientific goal of STC, will be completed. To reach this objective the optical design (proprieties are shown in Table 1) was conceived to reach a coverage of 40×19 km2 for the panchromatic filter and 40×3 km2 for the color ones (periherm configuration on equator).

Table 1

Optical and on-ground propriety of STC instrument.

Focal length95 mm
Focal ratioF/6.3
Pixel size10 μm SiPIN CMOS
Pupil size15 mm
IFoV22”/px (105 μrad/px)
FoV (cross-track dir.)
Spectral range410-930 nm
 panchromatic700 ± 100 nm
 other filters420 ±10 nm
  550 ±10 nm
  750 ± 10 nm
  920 ± 10 nm
Ground pixel scale50 m/px (@400km periherm)
Stereo accuracy80 m (@400km periherm)
Coverturepanchromatic40×19 km2
 other filters40×3 km2
Swath30 km 

Two independent elements compose the STC optical solution [4]: two folding mirrors per each channel, and a common telescope unit. The couple of folding mirrors redirects the ±20° wide beams along direction much closer to the system optical axis (±3.75°). The optical path is in common for both the channel and the rays are focused on a 10 μm pixel size SiPIN CMOS detector, useful in terms of radiation hardness and for the capability of snapshot image acquisition, allowing very short exposure times (< 1 ms).

Result of this innovative design is a light instrument which cuts down on energy and permits stereo reconstruction with a narrow focal length of 95 mm to reach the 50 m/pixel scale factor at periherm.

B.

Stereo Validation Set-up (SVS)

Considering the brand-new design adopted a pre-flight verification was needed. Base of the pipeline for the generation of the DTM of Mercury surface is the knowledge of the intrinsic and extrinsic parameters of the optical system. With this aim a stereo validation setup was realized for an indoor reproduction of the flight observing condition [5] [6]. See Fig. 2 for a scheme of the stereo validation set-up concept.

Fig. 2

Setup for STC stereo validation concept: a. STC observing Mercury surface at periherm (flight conditions). b. In laboratory simulation of STC observing with “Forward channel” (F) and “Backward channel”

00151_PSISDG10563_105634E_page_4_1.jpg

Two principal problems were engaged: the need of acquire satellite images in a limited indoor space and the necessity of supply two different perspectives of a surface to use both the optical channels.

First hindrance was solved by the use of an auxiliary optical system to observe with the STC instrument not a planetary surface from an altitude greater than 400 km but a selected small stone target at about 1 meter distance: a lens (collimator) that collimates the light rays coming from the target (essentially placing it at infinity) has been foreseen. Optical analysis by means of ray-tracing software led to the choice of an achromatic doublet with focal length of 1000 mm with a diameter of 80 mm.

Neglecting for the sake of simplicity the Mercury curvature, multi-perspective acquisition was supplied by the use of two high precision rotational stages (see Fig. 3), which permit the acquisition of images from two different points of view. These stages allow to rotate the instrument and the target without the need of additional space not available in the close clean room environment. The first rotation stage, where the STC instrument (in a thermo-vacuum chamber) is positioned, has the function of aligning the optical axis of the active STC channel with the collimated light coming from the collimator lens; the second rotation stage has the function of rotating a target positioned on the collimator lens focusing plane, reproducing the angle between a pair of homologous rays.

Fig. 3

In (a) Setup for the stereo validation: mechanical layout. From left side: on the optical bench the collimator, on the rotation stage the light source and the rock sample. In (b) example of target (stone) acquisition with lamp in extreme orientation position and rotational stage on optical path of high channel in panchromatic (-21°).

00151_PSISDG10563_105634E_page_5_1.jpg

With the aim of reproduce different light conditions, a narrow-spot halogen lamp coupled to a collimator has been assembled on a curve rail system. Considering that the target illumination condition has to be maintained in the two acquisitions with STC channels, the rail was mounted on the target rotational stage in order to be moved together with the target.

C.

Scientific requirements scaling

The ratio between the observable feature size (horizontal and vertical accuracy) in the laboratory images and in the in-flight ones are proportional to the ratio between the collimator focal length and the distance of the SC from the Mercury surface during the acquisition. At periherm the observing distance is about 400 km: considering a collimator focal length of 1 m the scale ratio will be 1:400000.

So, considering that the area of the Mercury surface that will be imaged by STC is approximately 40x20 km2, the corresponding target area that should be imaged with this setup in the laboratory is of approximately 9x4 cm2, referring to the panchromatic FoV.

Rescaling parameters of Table 1 to obtain the spatial resolution corresponding to that of STC (50 m) with a 1 m collimator focal length, the target surface should be sampled with a step of 105 μm and the elevation should be determined with an accuracy of 190 μm.

D.

Rock target definition

Scaling factor obtained permitted to choose a correct dimension for the target. To reproduce the same albedo condition of the ones attended from the Mercury surface basalt and anorthosite were chosen as composition material of the target stones.

These rock samples were acquired by scanning by a CAM2® FaroArm Platinum, a portable Coordinate Measurement Machine (CMM) with a Z axis accuracy of 0.02 mm which supplied a DTM with point spacing of 0.05 mm, i.e. about half of the image resolution [5][6].

This permits to have a reference DTM with an accuracy better than the one required by scientific requirements. DTM obtained by laser scanner is than use to compare stereo reconstruction and to define the effective validity of the instrument approach.

III.

INTRINSIC AND EXTRINSIC CALIBRATION

As explained before, if laser scanner data are available directly for stereo reconstruction, intrinsic and extrinsic coordinates are needed on one side to reduce time and error in reconstruction by using epipolar geometry, on the other side to obtain system geometry or triangulate points and to pass from image coordinates to metrical data.

A first estimation of these parameters is obtained by an “ad hoc” calibration procedure which has been used for the two channels.

Two important limits have to be taken in account in this process: on one side, the channels have a very narrow field of view (about 5.3°x2.3°), and in addition the limited depth of focus (reduced by the fact that any target has to be rotated by the rotational stage) makes difficult the use of strongly convergent images of a calibration target. A standard analytical calibration procedure would therefore lead to large estimation uncertainties as well as to strong correlations between intrinsic and extrinsic parameters. To partially cope with the problem, the principal point position of each channel (defined in this particular race by the collimator axis) has been optically measured using a laser target; moreover, since no significant radial distortion was apparent in the images of the calibration target (see Fig. 4), the corresponding parameters were neglected. Zhang’s algorithm [7] has been used to compute, as the only unknown, the focal length; the estimated value, although with a large uncertainty, turned out to be rather close to the expected nominal value of the focal length of the STC camera – collimator combination.

Fig. 4

In (a) laboratory acquisition of the stereo target well lighted by halogen lamp. In (b) the chessboard calibration target imaged with STC FM. In (c) the re-projection error from a calibration session. The 2-σ curve of the error obtained in calibration is close to 0.1 pixel both along x and y directions.

00151_PSISDG10563_105634E_page_6_1.jpg

As far as image orientation is concerned, two different approaches have been followed in cascade: Zhang camera calibration and Bundle adjustment refinements.

A first set of orientation parameters has been obtained by using the stereo rig module of the well-known Camera Calibration Toolbox [7], introducing part of intrinsic parameters of each camera as known values.

A.

Zhang camera calibration

A sequence of about 50 image pairs of the chessboard target has been used to compute the relative orientation (extrinsic parameters) of the camera pair for STC FM. Since the relative geometry between the cameras is nominally unchanged, such parameters can be used in the derivation of the DTM, when the stone samples are substituted for the calibration target; however, a not trivial co-registration of the reconstructed DTM with the laser measured one is necessary to compare the two surfaces.

Results of the Zhang calibration impose the rotation between the two optical axes of the channels angle are equal to 42.28° (with a relative error around 2%) and stereo base line equal to 1.23 m. These data correspond to a distance pupil-target equal to 1.59 m.

In the case of intrinsic parameters while principal point and skew factors are determined, both the channels present a focal length around 164 mm, but the error band is greater and reach the 8%. It has to be considered that these parameters identify just a starting point for bundle adjustment refinement.

B.

Bundle adjustment refinements

In the second part of the pipeline, the orientation is computed by means of a bundle block adjustment of a series of automatically identified tie points and some Ground Control Points (GCP) extracted from the reference DTM. A stereo pair of the rock target in the region of the tie-points is shown in Fig. 5.

Fig. 5

The stereo pair of one of the rock samples with markers.

00151_PSISDG10563_105634E_page_7_1.jpg

Tie points are automatically extracted and matched by structure from motion algorithms [8], [9]. GCP’s are object points of known coordinates, independently determined with an accuracy better than that obtainable by photogrammetric techniques; using them in a bundle adjustment allows to fix the reference system to the object and to control residual errors in intrinsic parameters.

Tie points have two important scope: on one side they are well-recognizable both on images, DTM results and on points cloud results of the laser scanner acquisition; on the other side they permit a fast and precise comparison between stereo results and laser scanner acquisition without resorting to closest point algorithms.

IV.

RECONSTRUCTION RESULTS

As intrinsic and extrinsic parameters are recomputed by bundle adjustment, the DTM generation of the rock samples has been performed with the program Dense Matcher being developed at the University of Parma since 2006 [10]; this software implements the NCC (Normalized Cross Correlation) method, the Least Squares Matching (LSM) method [11] and the Multi-photo Geometrically Constrained Matching (MGCM) method [12]. LSM is a local area-based method, where a linear radiometric transformation and the affine mapping are introduced to minimize the squared sum of the grey values differences between the patch image and the template image. As input data the program requires interior and exterior orientation parameters of the images. The matching stage is embedded in a multi-resolution approach where three levels of an image pyramid are processed. A parallel dense matching procedure is implemented where the initial disparity map is computed by performing a feature based matching followed by interpolation on a grid.

As described in Fig. 6 different tests were followed out varying the shape function in the LSM (Affine, Projective and Polynomial) and the template size (T17: 17×17 pix, T21: 21×21 pix, T25:25×25 pix).

Fig. 6

Comparison of the resulting standard deviation and RMS using three different shape functions: Affine, Projective and Polynomial and different template size in the SVS test.

00151_PSISDG10563_105634E_page_7_2.jpg

An example of the results of the different choices undertaken is shown in Fig. 7. The image shows that high spatial frequency are better reconstructed by the use of Polynomial shape function (more details are visible in Fig. 6b). Metrical analysis otherwise demonstrates how convergence to the laser scanner acquisitions is performed by all the models in the same way making useless the implementation of a higher degree functional mode.

Fig. 7

Level of detail of the DTMs processed in the example case of T25 by the using of (a) the Affine model approach and (b) the Polynomial model.

00151_PSISDG10563_105634E_page_8_1.jpg

Several target stereo couples of images obtained with the STC functional breadboard and the SVS are currently under analysis. The point cloud produced by Dense Matcher has been imported, together with the reference DTM, in a 3D modeling software for the comparison.

V.

CONCLUSIONS

An innovative setup for the validation of stereo reconstruction capabilities of the SIMBIO-SYS STC has been developed and tested. The preliminary results obtained presented an overall reconstruction error that is compatible with the specifications. Aim of the work was in fact the achievement of a stereo accuracy less than 80 m on the Mercury Surface derived from the instrument scientific requirements. This upper bound was scaled in the controlled ambient of a clean room to 190 μm.

A validation set-up to reproduce geometrical and stereo proprieties of the observatory strategy of BepiColombo was produced respecting light proprieties and material albedo characteristics.

A step by step pipe-line was implemented with the aim of calibrate the system and obtain DTM reconstruction of the target stone reaching a RMS (compared to more fitted and precise laser reconstruction) less than 70 μm.

The fact that RMS obtained by this method is less than half the required stereo accuracy demonstrated both the capabilities of the testing method and the validity of STC project.

Work is still in progress in order both to better refine the stereo reconstruction techniques and also to use a novel approach to stereo reconstruction [13], [14]; moreover, additional sets of data will be acquired, to evaluate the impact of different sample morphologies and lighting conditions on stereo performance.

The set up validated the reconstruction method for STC and supplied a bench of stereo images with different light condition that will permits to test different sun direction condition on Mercury and the appropriate parameter for stereo algorithm. These bench will be used even to improve the semi-global SIM algorithm [15] projected for this mission.

ACKNOWLEDGMENT

This activity has been realized under the BepiColombo Agenzia Spaziale Italiana (ASI) contract to the Istituto Nazionale di Astrofisica (INAF) no I/022/10/0 in collaboration with Selex Electronic Systems S.p.A, Campi Bisenzio (FI).

REFERENCES

[1] 

E. Flamini et al., “SIMBIO-SYS: the spectrometer and imagers integrated observatory system for the BepiColombo planetary orbiter”, Planet. Space Sci., vol. 58, pp. 125–143, 2010.Google Scholar

[2] 

G. Cremonese et al, “The stereo camera on the BepiColombo ESA/JAXA mission: a novel approach”, Advances in Geosciences, vol. 15, 305, 2009.Google Scholar

[3] 

V. Da Deppo, G.Naletto, G.Cremonese, L.Calamai., “Optical design of the single-detector planetary stereo camera for the BepiColombo European Space Agency mission to Mercury”, Appl. Opt., vol. 49, pp. 2910–2919, 2010.Google Scholar

[4] 

V. Da Deppo et al., “Optical Design performance of the stereo channel for SIMBIOSYS onboard the BepiColombo ESA mission”, Proceedings of the International Conference on Space Optics (ICSO2010), Rhodes Greece, 1981911, 2010.Google Scholar

[5] 

G. Naletto et al., “Innovative optical setup for testing a stereo camera for space applications”, Proc. SPIE 8442, Space Telescope and Instrumentation 2012: Optical, Infrared and Millimeter Wave, vol. 84421M 2012.Google Scholar

[6] 

E.Simioni, V.Da Deppo, G.Cremonese, C.Re, G.Naletto, “Stereo Camera for satellite application: A new testing method”, IEEE Workshop on Metrology for AeroSpace, in press.Google Scholar

[7] 

Z. Zhang., “A flexible new technique for camera calibration”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), pp. 1330–1334 (2000).Google Scholar

[8] 

Bouguet J.-Y., “Camera Calibration Toolbox for Matlab”, Available online, http://www.vision.caltech.edu/bouguetj/calib doc (2010)Google Scholar

[9] 

Bay H., T. Tuytelaars, and L. Van Gool., “SURF: Speeded Up Robust Features”, CVIU, vol. 110, No. 3, pp. 346–359, 2008.Google Scholar

[10] 

Nister D., “An efficient solution to the five-point relative pose problem”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26(6), pp. 756–7701, 2004.Google Scholar

[11] 

Re C., R. Roncella, G. Forlani, G. Cremonese, G. Naletto, “Evaluation of area-based image matching applied to DTM generation with Hirise images”, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol I-4, 2012.Google Scholar

[12] 

Grün A., “Adaptive least squares correlations: a powerful matching techniques”, South African J. Photogramm. Remote Sensing and Cartography, vol. 14(3), pp. 175–187, 1985.Google Scholar

[13] 

A. Grün, E.P. Baltsavias, “Geometrically constrained multiphoto matching”, PE&RS, vol. 54(5), pp. 633–641, 1988.Google Scholar

[14] 

M. Massironi et al., “Simulations using terrestrial geological analogues to assess interpretability of potential geological features of the Hermean surface restituted by the STereo imaging Camera of the SIMBIOSYS package (BepiColombo mission)”, Planetary and Space Science, vol. 56(8), pp. 1079–1092, 2008.Google Scholar

[15] 

E. Simioni et al, “A new stereo algorithm based on snakes”, Photogrammetric Engineering & Remote Sensing, vol. 77(5), pp. 495–507, 2011.Google Scholar

© (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
E. Simioni, E. Simioni, C. Re, C. Re, V. Da Deppo, V. Da Deppo, G. Naletto, G. Naletto, D. Borrelli, D. Borrelli, M. Dami, M. Dami, I. Ficai Veltroni, I. Ficai Veltroni, G. Cremonese, G. Cremonese, } "Indoor calibration for stereoscopic camera STC: a new method", Proc. SPIE 10563, International Conference on Space Optics — ICSO 2014, 105634E (17 November 2017); doi: 10.1117/12.2304188; https://doi.org/10.1117/12.2304188
PROCEEDINGS
9 PAGES


SHARE
Back to Top