Translator Disclaimer
2 June 2020 Development of a reconstruction quality metric for optical three-dimensional measurement systems in use for hot-state measurement object
Author Affiliations +
Abstract

Optical three-dimensional (3-D) geometry measurements are state of the art when it comes to contactless quality control and maintenance of the shape of technical components that exclude tactile measurements due to filigree or internal structures. Optical inspection methods are also characterized by a fast and high-resolution 3-D inspection of complex geometries. And due to their noncontact principle, they can carry out measurements in places that would otherwise not be accessible due to harsh environmental conditions or specimens such as hot forged parts. However, there are currently no methods to estimate the reconstruction quality for the optical 3-D geometry measurements of hot objects. The mainly used geometric measurement standards cannot be used for the characterization of hot measurements since the calibrated geometrical values are not transferable to high temperatures. For the development of such a metric, we present the fundamentals of the concepts and algorithms for an estimation of the reconstruction quality are presented and evaluated using a two-dimensional simulation model. The generated findings were applied to the 3-D geometry measurement of a hot object in a laboratory environment. The results are compared with general state-of-the-art reconstruction quality metrics.

1.

Introduction

In this paper, we introduced a suitable metric to estimate the reconstruction accuracy of an optical three-dimensional (3-D) measurement system used under the influence of a refractive index gradient. Such conditions may be caused by the heat transfer from a hot measurement object into the ambient air, which leads to a deflection of the measurement light from its assumed linear path. This light deflection effect is an additional source of uncertainty when measuring the 3-D geometry of a hot-state object. The uncertainty estimation cannot be based on the comparison of the measured data with the calibrated geometry of a measurement standard, as is regular practice, since the calibrated features correspond to precisely the calibration at defined conditions, usually ambient pressure p=101325  Pa and temperature T=293  K. Also, the geometry of any component at hot-state cannot be measured by a tactile coordinate measurement machine since it cannot be excluded that the elevated temperatures have damaging effects on the tip or may cause conditions for which the system is no longer calibrated.

Members of this research group have tried to estimate the added uncertainty by analyzing the acquired data by comparing it with a mathematical model1 and have conducted a multiphysics simulation2 to estimate the effect of the light deflection on a simulated 3-D geometry measurement. In addition, we have measured hot objects in different ambient pressure situations3 and from different angles4 to try to estimate metrics to evaluate the reconstruction quality. An indirect metric was found in the estimation of the inhomogeneous refractive index field induced by the heat transfer of the hot object into the ambient air using the background-oriented schlieren method.5,6 Nevertheless, a direct proof and estimation of the influence of the light deflection effect was not conducted.

Other groups have also developed 3-D measurement techniques for hot-state components, but they omitted a reconstruction quality estimation or based it on the predicted geometry of a heated component. The geometry prediction was based on a combination of the measured object temperature, the thermal expansion coefficient of the components material, and the measured geometry of the component at room temperature. Standard quality estimation methods for optical 3-D geometry measurement systems, e.g., the backprojection error from Hartley and Zisserman,7 were not applied. The results from the literature are recapitulated in the following to give an impression of the objectives of other researchers and the achieved uncertainties and methods to estimate those uncertainties.

Liu et al.8 used a line projection device combined with a two-camera stereo vision setup to calculate the diameter of a hot cylindrical forged part. The radiation from self-emission of the hot object was blocked by a lowpass filter. There was no hot reference object available for the experiment, wherefore the accuracy was not estimated. Their proposed solution was based on the calculation of the fundamental matrix from the corresponding pixel points7 and the matching of points to the epipolar constraint. Their second validation approach was based on a heating simulation and the expected expansion of the measurand. Here, their results match with an error of 0.7%. Zhang et al.9 used a laser light section method for the geometry reconstruction of cylindrical and rectangular hot heavy forged parts. The main research topic was the investigation of movement of the measurement system and the width of the laser line projection. An accuracy estimation was not conducted. Zatočilová et al.10 developed a stereo vision system for rotationally symmetric forgings under high temperature. However, since they were not able to identify homologous points in both cameras, they opted to reconstruct the hull of the object by fitting two perperdicular planes in the images. They estimated that their possible accuracy was in the millimeter range.

Schöch et al.11,12 developed a holistic geometry measurement system for arbitrary component shapes based on multiple light section scanners and a linear motion platform. They estimated the accuracy of their system at room temperature to be around 0.15 mm for a diameter measurement of 300 mm. For validation under hot conditions, they measured low-expansion workpieces (lengths l={70  mm,130  mm} with a thermal expansion coefficient of 0.63×106  K1) at about 850°C. While the reconstructed object lengths were within the expected range, there was no explicit statement of the achieved accuracy or the influence of the light deflection effect by the inhomogeneous refractive index field. They additionally estimated the theoretical maximum influence of the light deflection effect on light-section measurements to be about 30  μm for objects at a temperature of 100°C. Their calculation was based on the light deflection caused by a homogeneous refractive index field of a slightly different refractive index outside the sensor housing. They concluded that the light deflection effect can be neglected due to its diminishing importance compared to the size of the measurand of about 300 mm in diameter.

Du et al.13 used a two-dimensional (2-D) laser radar, comprising a single laser beam and two servo-motors, to measure the geometry of cylindrical hot heavy forged parts. The measurement error was estimated to about 2 mm for a cylindrical reference object of 450  mm×275  mm (height×diameter) at room temperature. For the same object at hot state, they estimated the error to be 5 mm for the diameter and 8 mm for the height. The single point error was not investigated, and the reference method was not stated. Zhang et al.14 combined a light section system with reflection models to reconstruct the edges of a hot component. This simplifies the evaluation of the acquired 3-D data to calculate the length of an object. The accuracy of the method was estimated by comparing the reconstructed the geometry of the cold object with a reference measurement; therefore, neither a single point accuracy nor a hot reference method was given. Hawryluk and Dworzak et al.15,16 used the combination of a light-section laser-scanner with a measuring arm, which is a six-axis serial arm reconstructing the 3-D position of its tip by measuring its joint angles. The system was used to reconstruct the geometry of the tools used in a hot-forming process. A wear analysis of the tools was conducted based on the acquired geometry. In our opinion, the wear thus described is not sufficiently examined in terms of its plausibility due to uncertainties caused by inhomogeneous ambient conditions or inherent uncertainties. Bračun et al.17 used a laser-scanner to reconstruct the geometry of a hot specimen. They focused on the compensation of different lighting conditions due to inhomogeneous emission coefficients on the object’s surface as well as the measurement accuracy. They identified scale chips on the object’s surface and the surface mircotopography as the most significant influences. Zhou et al.18 used a stereo vision system in combination with a line projector. The main research was focused on the line detection along measurement object edges in the stereoscopic image pairs. The detected lines were used to reconstruct the object’s geometry edges through epipolar geometry.7 Zhang et al.19 introduced a laser scanner-based shape reconstruction method for large ring forgings. The main aspects of their approach were given by data fusion of consecutive measurements from different sensors to the holistic geometry of the component. For the validation of their data, they ompared the measurement results to the components radius, which was measured by a caliper. The estimated errors were within 1.2 mm for a radius of about 450 mm.

2.

Basics of Optical 3-D Geometry Measurement

Optical 3-D geometry measurement techniques are state of the art when it comes to contactless quality control for the shape of technical components, especially for free-form objects.20 The mapping of an arbitrary 3-D point x=(x,y,z)T onto a pixel location ub=(ub,vb)T on the sensor of a camera is described by the perspective projection of the pinhole camera model, according to

Eq. (1)

λ(ub,vb,1)T=Ki·(Rt)i·(x,y,z,1)T.
Here, Ki and (Rt)i describe the projection matrix and the pose of the used camera i, respectively. There is a loss of depth information in the process, i.e., a reduction of a point in 3-D space x to a pixel on a 2-D sensor ub. The information about the z-location of the 3-D point x is stored in λ.

The optical measurement of a 3-D geometry therefore needs to compensate for the loss of dimensions, e.g., through the use of two or more cameras observing the same scene. The reconstruction is then based on the triangulation of homologous 2-D points in multiple cameras. A successful reconstruction needs the satisfaction of Eq. (1) for each corresponding point; therefore, the identification of the system-inherit parameters [Ki, (Rt)i for each camera i] and the detection of the homologous points are necessary.

A calibration routine using a known target is used to compute those camera parameters, e.g., the one developed by Zhang.21 The reconstruction itself is then based on the intersection of two lines from two cameras i,j, resulting in a reconstructed 3-D point x(uc,i,uc,j). Since these lines in 3-D space are usually skewed due to a leftover calibration error or numerical uncertainties, different algorithms were developed to reconstruct the 3-D point. A popular method is an optimization based on the epipolar geometry.7

However, homologous points may not be detectable on measurement objects without a structured surface. An active triangulation system, which incorporates an additional illumination unit, is used to counter this. One approach uses different projection images to add an artificial structure to the surface, e.g., by projecting a pseudorandom pattern to create homologous points. Such structured surface can then be reconstructed using the described triangulation technique with multiple cameras. A different approach uses a coded projection sequence to retrieve the corresponding projector pixel up in a given camera pixel location up(uc), e.g., the multifrequency phase-shift method developed by Peng et al.22 The projector is modeled here as an inverse camera to then reconstruct 3-D-points from the given pixel values of camera c and projector p, i.e., x(up,uc), using the described triangulation techniques.

For the researched metric to estimate the reconstruction accuracy of an optical 3-D geometry measurement technique, in our case a fringe projection system, it is necessary to identify corresponding points in a setup comprising one projector and multiple cameras. To this means, Bräuer-Burchard et al.23 and Reich et al.24 investigated the additional knowledge gained from a system with more than one camera nc>1 and a single projector np=1. They developed an algorithm to change the basis of the calculation from the camera sensor to the virtual projector sensor up(uc)uc(up). Using this method unlocks additional stereo pairs for triangulation by finding the corresponding camera pixels, i.e., not only reconstructing the 3-D points xi=f(up,uc,i) but also using xi,j=f(uc,i,uc,j). There are m=(n1)! stereo pairs in such a system with n=np+nc optical devices. Therefore, there are m reconstructed 3-D points xi,j(up) for each projector pixel. To benefit from the fractional projector pixel locations retrieved by the phase-shift algorithm, they developed a method to arbitrarily scale the projector resolution. The main advantages of this method are the optimization of the point cloud density by changing the projector resolution in a virtual raster (VR).

3.

Proposed Method

First, a simplified model of the influence of refractive index fields on the reconstruction of 3-D points is described. The model is based on the compression of an inhomogeneous refractive index field to a medium containing two refractive indices with a discreet and sharp boundary layer. The assumptions are used to explain the background of the proposed method. Based on this, the presented method is brought into relation with the known backprojection error and the deviation of corresponding object points.

3.1.

Theoretical Background

Suppose a ray of light, traveling through a medium with refractive index n1 and a medium with n2, is subject to refraction at the boundary interface between n1 and n2. The angle of refraction β is calculated from the incident angle α and the quotient for refractive indices n1n2, according to

Eq. (2)

sinβ=n1n2sinα.
Suppose the refraction is caused by a discreet and plane parallel medium of thickness dw; the length lr of the refracted ray in n2 is

Eq. (3)

lr=dw/cosβ.
The refracted ray of light is shifted parallel by p compared with the virtual nonrefracted ray, according to

Eq. (4)

p=lrsin(αβ).
Let us suppose the ray of light is connecting one arbitrary 3-D point xk and the projection center of a camera ci (see Fig. 1). The view ray ak,i for the camera with center ci then connects the camera center and the first intersection of the ray of light with the interface layer dk,i, according to

Eq. (5)

ak,i=dkcidkci2.
This view ray is subject to a different amount of light deflection pk,i for each different camera position ci and for each different point xk.

Fig. 1

Refraction of a light ray connecting xk and ci based on an interface layer with thickness dw and refractive indices n1 and n2. The solid dashed line represents the virtual view ray for the camera ci, as its path is assumed to being linear. The pale line represents the actual path the light takes, including two refractions at the boundary layers between n1 and n2.

OE_59_6_064103_f001.png

For a 3-D geometry measurement system based on stereo triangulation, the reconstruction of an arbitrary point xk in a system with more than two cameras n>2 results in m=(n1)! reconstructed points xk,i,j, each one from a combination of cameras to a stereo pair i,jm. For assumed homogeneous refractive index conditions, all reconstructed points are, theoretically, equal to the corresponding and observed point in space, according to

Eq. (6)

xk,i,j=xk,
for each camera pair i,jm and n1=n2. In inhomogeneous conditions n1n2, each reconstructed point xk,i,j is subjected to the combined light deflections pk,i and pk,j of both light paths, which, again, is different for each camera, i.e., pk,ipk,j. Therefore, the distance bk,i,j between the point in space xk and the reconstructed points xk,i,j is expected to increase with larger thicknesses dw and larger refractive index differences Δn1,2=|n1n2|. While being different for different used camera pairs, this distance bk,i,j cannot not be measured in a real setup because it is not feasible to accurately reconstruct the point xk. Therefore, it is proposed to base the estimation of the reconstruction quality on the spatial extent of the set of reconstructed points xk,i,j. A natural choice for an easy-to-calculate representation for said extent is the statistical deviation of all of the reconstructed points xk,i,j for i,jm.

3.2.

Estimation of the Reconstruction Quality for Multistereo-Pair Systems

The main requirement for all of the proposed methods is the unambiguity of the corresponding pixel locations. This is easily achievable for random-pattern or single-point triangulation systems due to the unambiguous nature of these systems. For fringe projection systems, this requirement can be achieved using Bräuer-Burchardt’s VR method (see Sec. 2), in which all camera pixels are calculated as a function of the projector pixel uc,i=f(up).

Different calculation methods for a quality estimation are shown by Hartley and Zisserman.7 The commonly used method is the backprojection, i.e., the recalculation of the pixel location ub from the reconstructed 3-D point x [see Eq. (1)]. The Euclidean distance eb to the activated pixel used for the reconstruction u is then called backprojection error, according to

Eq. (7)

eb=uub2.
When applying this method to a setup with more than two cameras n>2, all stereo pairs m are taken into account. The backprojection error is then calculated separately for each pair i,jm, according to

Eq. (8)

eb(xi,j)=||ui,h1λiKi·(Rt)i·xi,j,h||2+||uj,h1λjKj·(Rt)j·xi,j,h||2.
To generalize the results, the average norm of the reprojection error for each reconstructed point within all observing cameras is computed, according to

Eq. (9)

eb,s=1meb(xi,j)for  i,jm.
To take the assumptions from Sec. 3.1 into account, the backprojection can also be based on the mean of correspondingly reconstructed points, i.e., on

Eq. (10)

xm=1mxi,jfor  i,jm.
Now, we calculate the backprojection error eb,i,m for each camera or projector in from that mean point xm, according to

Eq. (11)

eb,i,m=||ui,h1λiKi·(Rt)i·xm,h||2.
To make this error comparable throughout systems with different numbers of cameras and projectors n, the average of this value is calculated, according to

Eq. (12)

eb,m=1neb,i,mfor  in.
However, this value might be a function of the used camera resolutions. Therefore, we also propose basing the error in the metric system, i.e., by calculating the reconstruction quality as the statistical variance of the reconstructed points, according to

Eq. (13)

Em=1m1||xi,jxm||2for  i,jm.
A summary of the proposed reconstruction quality metrics as a flowgraph is shown in Fig. 2.

Fig. 2

Flowchart of the developed reconstruction quality metrics.

OE_59_6_064103_f002.png

4.

Experimental Setup

In this section, the setup for the experiments is shown. The proposed methods are tested as a 2-D simulation model under homogeneous refractive index conditions, i.e., n1=const., n2=const., n1n2. In addition, an optical 3-D fringe projection setup is used in a laboratory environment under homogeneous conditions, i.e., with a glass window, and under inhomogeneous conditions, i.e., measuring a hot object. In the simulation model, the light propagation is assumed to be in one plane as it simplifies the refraction and line–line intersection calculation.

4.1.

Model Setup

To test the proposed approach, a simplified 2-D simulation is set up (see Fig. 3). For the model, a homogeneous and plane parallel refractive index field of thickness dw and refractive index n2 is inserted into an otherwise also homogeneous propagation medium with a refractive index of n1. The path of light from a point xk=(x,y)T in 2-D space to the center ci=(cx,i,cy,i)T of a virtual camera i is calculated using Snellius’ law (dashed lines) and a brute-force method. The mean of the leftover point-line distance from camera center ci to the refracted path of light is about 104. The vector from the camera center ci to the first intersection dk,i between n1n2 is then used as directional vector ak,i (dashed lines) to calculate the virtual intersections

Eq. (14)

xk,i,j:ci+λiak,i=cj+λjak,j,
for any combinations i,j of the present camera pairs. The reconstructed points are then evaluated according to Eq. (13).

Fig. 3

Setup of the 2-D model experiment. All lengths are without units. The refraction of view rays ak,i in an ambient refractive index field n1=1.2 from the camera centers ci is calculated through an inserted, homogeneous refractive index field n2=1.4 of thickness dw=5. The refracted view rays are marked with dashed/dotted lines and all intersect in xk. The reconstructed points xk,i,j are on the intersection of the virtual nonrefracted view rays. For n2<n1, the points xk,i,j are reconstructed with a greater y value, i.e., are above the backprojected point xk.

OE_59_6_064103_f003.png

4.2.

3-D Measurement Setup

The used optical 3-D fringe projection setup combines one green-LED projector (Wintech PRO 4500 based on TI DLP LightCrafter 4500) and four cameras (Allied Vision Prosilica GT 2050 and Prosilica GT 2300 with MeViS-C lens) to a multicamera fringe projection system [see Figs. 4(a) and 5(a)]. There are bandpass filters on the camera lenses used to block incoming radiation from the self-emission of the hot objects. The triangulation bases and triangulation angles are listed in Table 1.

Fig. 4

The used measurement setup. (a) Image of four-camera one-projector 3-D geometry measurement system. The short code for the identification of the cameras is also shown; (b) Image of the used measurement object at Tc1300  K. One pattern of the sinusoidal sequence is projected onto the object. The object lays in a prism holder with an anchor point on the tail end.

OE_59_6_064103_f004.png

Table 1

Triangulation base and triangulation angle for all available stereo pairs in the fringe projection setup. The code for the stereo combination is taken from Fig. 4(a).

Stereo pair combinationTriangulation base in mmTriangulation angle in deg
clL:p27139
clR:p28540
cuL:p24033
cuR:p20929
clL:clR14121
clL:cuL46465
clL:cuR45265
clR:cuL50171
clR:cuR15321
cuL:cuR44862

For an areal measurement, Peng et al.’s22 multifrequency phase-shift method is used to calculate the projector pixel in a given camera’s pixel up=f(uc,i) by projecting a coded sinusoidal sequence onto a specimen. The measurement object with a pattern of the projected sequence is shown in Fig. 4(b). Bräuer’s23 VR method is then used to calculate the phasemaps uc,i(up), therefore having calculated nc=4 corresponding pixels for each projector pixel up.

The 3-D-points are reconstructed from each available view-ray pair using epipolar geometry.7 Camera–camera pairs on the same side of the projector (clR:cuR and clL:cuL) are not used here since the expected uncertainty is higher compared with the other pairs due to a smaller triangulation angle and triangulation base (see Table 1). The number of measured 3-D points for each projector pixel is shown in Fig. 5(b).

Fig. 5

Measurement setup from the projector perspective. (a) Image of the used measurement object at Tc300  K in front of the measurement system. From the measurement system, only one camera and the projector are displayed; (b) Number of measured points for each projector pixel up. The theoretical maximum number is m=(n1)!=10 for n=5 cameras/projectors. In the presented case, this number is reduced to m=8 for excluding the camera-camera pairs on the same side of the setup. The red rectangle represents the examined area for the experiments.

OE_59_6_064103_f005.png

A series of experiments is conducted to compare and evaluate the proposed methods. In these experiments, the geometry of a cylinder of diameter dc=50  mm is measured, which is placed in a prism holder with an anchor point (see Fig. 4(b)). The reconstruction quality is estimated for this cylinder under regular conditions Tc300  K and with the cylinder at the forging temperature of Tc1300  K. To test the methods under the influence of a known refractive index field, a perspex disc with a thickness of dw=5  mm and a refractive index of n21.49 is placed between the measurement unit and the cold specimen (not shown). Each experiment consists of five consecutive measurements per measurement scenario, each taking about a span of 2 s. The experimental setup is kept constant between each set of measurements except for the measurement object, which is placed in an oven with Toven1300  K for about 1 h. Each measurement contains the reconstruction of xi,j(up) for each stereo pair i,jm. The proposed reconstruction quality metrics are then calculated from the identical measurement results, the per-pixel mean of those metrics are shown in the result maps (Figs. 7Fig. 109), and all measured points are evaluated for the histograms in Fig. 10.

5.

Results

The results for the 2-D simulation model and the 3-D fringe projection system are shown in this section. The expectations for the measurement results are based on the experience of this research group in regards to the geometry measurement of hot objects. In general, the reconstruction quality is expected to decrease when measuring a hot object through the heat-induced refractive index field compared with the measurement of the same cylinder at room temperature, as long as the cylinder is placed in a similar pose for both measurements. The exact same placement is not reached nor is the same diameter of the cylinder measured since both are influenced by the thermal expansion of the cylinder from Tc300  K to Tc1300  K. The measurement with the inserted glass window should yield a lower quality reconstruction value than both other measurements. This is mainly due to Eq. (4), by which the amount of light deflection is described. Overall, it is not expected to reach a perfect reconstruction quality since there is an intrinsic leftover error from the (always) imperfect calibration routine.

5.1.

Results for the 2-D Simulation Model

The points of investigation are tested in the 2-D simulation model (see Sec. 4.1 and Fig. 3). To this end, different points xk with k{0,1,79} and window thicknesses dw={1,3,5,7} are compared over variations of the refractive index difference Δn1,2=n2n1.

The results for a constant surrounding refractive index n1=1.2 are displayed in Fig. 6(a). Here, the variances em are nearly zero for Δn1,2=0 and increase with increasing refractive index differences em|Δn1,2|. The deviations increase equally for both Δn1,2<0 and Δn1,2>0 until a threshold of 0.1<Δn1,2<0.1. Afterward, the deviation gradient stays constant for Δn1,2<0 and decreases for Δn1,2>0 until a saddle point at about Δn1,2=0.25.

Fig. 6

Results for the 2-D simulation model as the Euclidean distance of the variances em over refractive index differences Δn1,2 and thickness of the window dw{1,3,5,7}. The solid lines are the result of the arithmetic mean of 81 reprojected points xk. The results of the single-point reconstruction are in the background. (a) Result for n1=const.=1.2 and n2 from 1.0 to 2.4; (b) result for n1=1.00028 and n2 from 1.0 to 2n1=1.00056. The ambient refractive index n1 in the presented case is the refractive index of air at standard conditions (ambient pressure 1 bar and 273 K). All length and thickness information is given in arbitrary units.

OE_59_6_064103_f006.png

For varying thicknesses dw, the deviations em increase with increasing thickness dw. There are also larger interpoint variances for an increasing thickness dw based on the location of the backprojected point xk. These are detected by observing the thickness of the more transparent points in the background.

The deviations em for a scenario with an ambient refractive index of n1=1.00028 (air at standard conditions) are shown in Fig. 6(b). Here, similar observations can be made compared to Fig. 6(a). However, the decrease of deviations em when overcoming the threshold Δn1,2>0.25 is not a part of the discussion of results. The relative interpoint differences are larger compared with those in Fig. 6(a), while being smaller on an absolute scale. The minimum deviation is at the expected place of Δn1,2=0.

The results from the laboratory experiment are split into the three proposed quality estimation methods {eb,s,eb,m,Em} since they vary in magnitude. The maps of the single camera backprojection error eb,s(up) [from Eq. (9)] are shown in Fig. 7. The region of interest, from which the data are extracted, is shown in Fig. 5(b). The regular backprojection error is not homogeneously distributed and has a mean value of about 0.443 pixel for Tc300  K. There seem to be few differences compared with the quality estimation of the cylinder at Tc1300  K with a mean deviation of 0.445  pixel [see Fig. 7(b)]. There are larger differences in the measurement with the inserted window in the light path of about 2.05 pixel [see Fig. 7(c)].

Fig. 7

Mean of backprojection error for each single camera. (a) Cold object at Tc300  K; (b) warm object at Tc1300  K; and (c) cold object with window in the path of the measurement light.

OE_59_6_064103_f007.png

The maps of the backprojection error eb,m(up) for each averaged corresponding object point xm and Eq. (12) are shown in Fig. 8, while the maps from the metric deviations of the corresponding 3-D points Em and Eq. (13) are shown in Fig. 9. In both cases, the differences from the measurements of the cold cylinder Tc300  K to the measurements of the warm cylinder Tc1300  K [compare Figs. 8(a) to 8(b) and Figs. 9(a) to 9(b)] seem to be small but detectable, while the differences between deviations from the cold measurement to the measurement with the inserted window are larger.

Fig. 8

Mean of backprojection error from mean of 3-D points for each single camera. (a) Cold object at Tc300  K; (b) warm object at Tc1300  K; and (c) cold object with window in the path of the measurement light.

OE_59_6_064103_f008.png

Fig. 9

Metric deviations from the mean of the 3-D points Em. (a) Cold object at Tc300  K; (b) warm object at Tc1300  K; and (c) cold object with window in the path of the measurement light.

OE_59_6_064103_f009.png

The histograms of these results (see Fig. 8) are sorted by the used evaluation metric. Therefore, the comparable results from eb,s are shown in Fig. 10(a), and the results from eb,m and Em are shown in Figs. 10(b) and 10(c).

Fig. 10

Histograms for the different quality estimation methods. (a) For eb,s. Mean value for cold object is 0.443 pixel, for warm object is 0.445 pixel, and for cold object with window is 2.047 pixel; (b) for eb,m. Mean value for cold object is 1.499 pixel, for warm object is 1.664 pixel, and for cold object with window is 6.197 pixel; (c) for Em. Mean value for cold object is 0.262 mm, for warm object is 0.306 mm, and for cold object with window is 1.152 mm.

OE_59_6_064103_f010.png

6.

Discussion of Results

Considering the results of the 2-D simulation model in Sec. 5.1, the proposed quality metric as the deviations em of the redundantly reconstructed points xk,i,j for i,jm meets the expectations. The single point deviation is highly dependent on the incident angle αk,i of each of the view rays for a camera i, as is the parallel shift pk,i. However, the average of the reconstruction error em for all points xk is strictly monotonously increasing with increasing nominal optical wavelength differences dwΔn1,2 for |Δn1,2|<0.1. The relation between single point deviation and em is recognizable in the thickness of the lines in Fig. 6, where each single-point reconstruction error is displayed in the background. The simulation model has therefore proven the validity of the proposed method for the detection of small refractive index differences in the reconstruction of points by triangulation.

In the case of the real experiment in Fig. 4, which includes a 3-D reconstruction of a hot specimen, it is demonstrated that the metrics introduced for quality estimation have a significant sensitivity for the detection of significant homogeneous refractive index influences [compare Figs. 7(a), 8(a), 9(a) to 7(c), 8(c), 9(c), respectively]. The differences are easily detectable in both the error maps eb,s(up), eb,m(up), Em(up) as well as in the histograms in Fig. 10.

For the overall objective of detecting the influence of the heat induced refractive index gradient, the pixelwise differences between Tc300  K and Tc1300  K on the respective estimation method seem to be insignificantly small. Even a statistical analysis of the acquired results yielded no comprehensible conclusion for eb,s [see Fig. 10(a)], while being subject to interpretation for eb,m and Em [see Figs. 10(b) and 10(c)]. In the histograms, the distribution of both error metrics seems to be (approximately) Gaussian, and there is a difference of 11% for eb,m and 17% for Em. The difference is both significant and robust considering the evaluated number of points of about k106. The relative small reconstruction quality difference is mainly due to the relative large error in the reference results, which are a consequence of an imperfect calibration process. Extrapolating from the 2-D results, the measurement with dw=5 and Δn1,2=0.5 yields em1.3 and the measurement with dw=5 and Δn1,2=0.00028 yields em7×103, i.e., a relative difference of em,Δn1,2=0.5/em,Δn1,2=0.00028185. Comparing this with the relative differences in eb,m of 3.72 and in Em of 3.76 yields a much larger sensitivity of the model experiment compared with the measurements under laboratory conditions. This is mainly due to the additional influence of the calibration error in the workshop experiments.

The intent was to directly compare the displayed results as a difference map, e.g., as Em(up,vp)Tc=300  KEm(up,vp)Tc=1300  K. This task proved to be impractical. The reasons for this effect are manifold.

The occurring and to be quantified light deflection moves the reconstructed points on the VR, so up(xTc=300  K)up(xTc=1300  K). In addition, the distribution of the reconstruction errors is inhomogeneous, even for the experiment in standard conditions [see, e.g., Fig. 7(a)]. Therefore, a small change in the object’s geometry or placement results in a change of the reconstruction error at that pixel position. These changes occur easily, mainly due to the different measurement conditions of the experiment at different states. Another reason is the surface of the object, which is constantly changing throughout heating and cooling periods as is its diameter. In addition, the position of the object cannot be assumed to be fixed, even though the cylindrical measurement object is placed in a prism holder with an anchor point.

7.

Conclusion and Future Work

In this paper, a method for the estimation of the reconstruction quality of an optical 3-D geometry measurement system under inhomogeneous refractive index conditions was presented. The method is based on a multicamera single-projector fringe projection system without using additional geometric standards, e.g., a sphere or a cylinder with a calibrated diameter. The reconstruction metrics are based on redundantly measured 3-D points and the analysis of the geometric mean of these points.

The theoretical background for the proposed methods, as well as the results of a 2-D simulation model, was laid out. In the experiment, the influence of the inhomogeneous refractive index gradient was joined in a homogeneous refractive index field of varying thicknesses and refractive index gradients. The deviation of the reconstructed points em was introduced as a reconstruction quality metric for the 2-D case, and its feasibility was verified by the results of the simulation model. Here, the mean deviation is proportional to the optical wavelength difference emdwΔn1,2 for small refractive index differences Δn1,2<0.1.

For the 3-D laboratory experiment, the feasibility of the proposed quality metrics was proven by the comparison of an uninfluenced measurement with a measurement in which a glass plate was inserted into the optical path. However, the comparison of a measurement of a cylinder at Tc300  K with the measurement at Tc1300  K revealed no differences for the state-of-the-art quality metric of the single-camera backprojection error eb,s. The evaluation of the results for the quality metric based on redundantly reconstructed points showed a small but significant difference between the hot and cold measurements. Here, the metric deviation Em yields a slightly higher sensitivity to the influence of the refractive index gradient than the backprojection eb,m of the mean points xm.

Overall, the results from the proposed reconstruction quality metric yield detectable differences when measuring hot objects compared with cold objects. This enables the estimation of a relative reconstruction quality for measurements in which geometric measurement standards cannot be used. For these, it is necessary to establish a ground truth for each measurement scenario by measuring a similar object in cold state.

In the future, the proposed and presented reconstruction quality metrics will be used to assess different compensation methods for the light deflection effect induced by an inhomogeneous refractive index field around wrought-hot objects. The combination of the presented metric and a similar geometric measurement standard can be used to calculate an absolute quality metric in reference to said standard. Also, it is intended to reduce or restructure the intrinsic reconstruction error to enable an analysis of hot objects without the need for a comparison with a cold object of similar shape.

Acknowledgments

This article was funded by Deutsche Forschungsgemeinschaft (DFG); Collaborative Research Centre 1153 (CRC) Process Chain to Produce Hybrid High-performance Components Through Tailored Forming; Project number 252662854; Subproject C5 Multiscale Geometry Inspection of Joining Zones.

References

1. 

R. Beermann et al., “Light section measurement to quantify the accuracy loss induced by laser light deflection in an inhomogeneous refractive index field,” Proc. SPIE, 10329 103292T (2017). https://doi.org/10.1117/12.2269724 PSISDG 0277-786X Google Scholar

2. 

R. Beermann et al., “Full simulation model for laser triangulation measurement in an inhomogeneous refractive index field,” Opt. Eng., 57 (11), 114107 (2018). https://doi.org/10.1117/1.OE.57.11.114107 Google Scholar

3. 

L. Quentin et al., “3D geometry measurement of hot cylindric specimen using structured light,” Proc. SPIE, 10329 103290U (2017). https://doi.org/10.1117/12.2269607 PSISDG 0277-786X Google Scholar

4. 

L. Quentin et al., “Enhanced measurement routine for optical 3d geometry measurement of hot specimen by reduction of ambient pressure,” in Eng. Changing World: Proc.; 59th IWK, Ilmenau Scientific Colloquium, 11 –15 (2017). Google Scholar

5. 

R. Beermann et al., “Background oriented schlieren measurement of the refractive index field of air induced by a hot, cylindrical measurement object,” Appl. Opt., 56 (14), 4168 –4179 (2017). https://doi.org/10.1364/AO.56.004168 APOPAI 0003-6935 Google Scholar

6. 

R. Beermann et al., “2D refractive index field measurements in air in different pressure scenarios,” Proc. SPIE, 10819 108190E (2018). https://doi.org/10.1117/12.2500315 PSISDG 0277-786X Google Scholar

7. 

R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, Cambridge (2003). Google Scholar

8. 

W. Liu et al., “An improved online dimensional measurement method of large hot cylindrical forging,” Measurement, 45 (8), 2041 –2051 (2012). https://doi.org/10.1016/j.measurement.2012.05.004 0263-2241 Google Scholar

9. 

Y.-C. Zhang et al., “Measurement and control technology of the size for large hot forgings,” Measurement, 49 52 –59 (2014). https://doi.org/10.1016/j.measurement.2013.11.028 0263-2241 Google Scholar

10. 

A. Zatočilová, D. Paloušek and J. Brandejs, “Development of a photogrammetry system for the measurement of rotationally symmetric forgings,” Proc. SPIE, 9525 952516 (2015). https://doi.org/10.1117/12.2184916 PSISDG 0277-786X Google Scholar

11. 

A. Schöch et al., “High-speed measurement of complex shaped parts at elevated temperature by laser triangulation,” Int. J. Autom. Technol., 9 (5), 558 –566 (2015). https://doi.org/10.20965/ijat.2015.p0558 Google Scholar

12. 

A. Schoech, “Quality control of freeform parts at elevated temperature,” (2016). https://books.google.de/books?id=gN8hyQEACAAJ Google Scholar

13. 

Z. Du, Z. Wu and J. Yang, “3D measuring and segmentation method for hot heavy forging,” Measurement, 85 43 –53 (2016). https://doi.org/10.1016/j.measurement.2016.02.004 0263-2241 Google Scholar

14. 

Y.-C. Zhang et al., “Automatic measurement method for the size of large forgings based on scattering on rough surface,” IET Sci. Meas. Technol., 11 (1), 118 –124 (2017). https://doi.org/10.1049/iet-smt.2016.0283 Google Scholar

15. 

M. Hawryluk, J. Ziemba and Ł. Dworzak, “Development of a method for tool wear analysis using 3D scanning,” Metrol. Meas. Syst., 24 (4), 739 –757 (2017). https://doi.org/10.1515/mms-2017-0054 Google Scholar

16. 

Ł. Dworzak, M. Hawryluk and J. Ziemba, “Wear analysis of die inserts in the hot forging process of a forked type forging using reverse scanning techniques,” Adv. Sci. Technol. Res. J., 11 225 –238 (2017). https://doi.org/10.12913/22998624/80318 Google Scholar

17. 

D. Bračun, G. Škulj and M. Kadiš, “Spectral selective and difference imaging laser triangulation measurement system for on line measurement of large hot workpieces in precision open die forging,” Int. J. Adv. Manuf. Technol., 90 (1–4), 917 –926 (2017). https://doi.org/10.1007/s00170-016-9460-0 IJATEA 1433-3051 Google Scholar

18. 

Y. Zhou, Y. Wu and C. Luo, “A fast dimensional measurement method for large hot forgings based on line reconstruction,” Int. J. Adv. Manuf. Technol., 99 (5–8), 1713 –1724 (2018). https://doi.org/10.1007/s00170-018-2551-3 IJATEA 1433-3051 Google Scholar

19. 

Y.-C. Zhang et al., “Online measuring method of radial section line for ring forgings,” Int. J. Adv. Manuf. Technol., 101 (9–12), 3035 –3046 (2019). https://doi.org/10.1007/s00170-018-3104-5 IJATEA 1433-3051 Google Scholar

20. 

G. Wiora, “Optical 3D-metrology: precise shape measurement with an extended fringe projection method,” University Heidelberg, (2001). Google Scholar

21. 

Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell., 22 (11), 1330 –1334 (2000). https://doi.org/10.1109/34.888718 ITPIDJ 0162-8828 Google Scholar

22. 

T. Peng, S. K. Gupta and K. Lau, “Algorithms for constructing 3-D point clouds using multiple digital fringe projection patterns,” Comput.-Aided Des. Appl., 2 (6), 737 –746 (2005). https://doi.org/10.1080/16864360.2005.10738337 Google Scholar

23. 

C. Bräuer-Burchardt et al., “Determining exact point correspondences in 3D measurement systems using fringe projection—concepts, algorithms, and accuracy determination,” Applied Measurement Systems, 211 –228 IntechOpen, Rijeka, Croatia/London (2012). Google Scholar

24. 

C. Reich, R. Ritter and J. Thesing, “3-D shape measurement of complex objects by combining photogrammetry and fringe projection,” Opt. Eng., 39 224 –231 (2000). https://doi.org/10.1117/1.602356 Google Scholar

Biography

Lorenz Quentin is a research associate at the Institute for Measurement and Automatic Control of the Leibniz Universität Hannover (LUH). He received his diploma in mechanical engineering from the LUH in 2016. He is part of the Collaborative Research Centre 1153 (CRC) Process Chain to Produce Hybrid High-Performance Components Through Tailored Forming and researches mainly the optical 3-D geometry measurement of hot specimens using fringe projection.

Rüdiger Beermann is a research associate at the Institute of Measurement and Automatic Control at the LUH. He received his diploma in mechanical engineering from LUH in 2013, and his state examination as a teacher for math and metal technology for vocational schools in 2015. His current research interests include the development of fringe projection systems for high-temperature workpieces and thermo-optical simulations.

Markus Kästner is the head of the Production Metrology research group at the Institute of Measurement and Automatic Control at the LUH. He received his PhD in mechanical engineering in 2008 and his postdoctoral lecturing qualifications in 2016 from the LUH. His current research interests are optical metrology from macro- to nanoscale and optical simulations.

Eduard Reithmeier is a professor at the LUH and head of the Institute of Measurement and Automatic Control. He received his diplomas in mechanical engineering and in math in 1983 and 1985, respectively, and his doctorate degree in mechanical engineering from the Technische Universität München in 1989. His research focuses on system theory and control engineering.

© The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Lorenz Quentin, Rüdiger Beermann, Markus Kästner, and Eduard Reithmeier "Development of a reconstruction quality metric for optical three-dimensional measurement systems in use for hot-state measurement object," Optical Engineering 59(6), 064103 (2 June 2020). https://doi.org/10.1117/1.OE.59.6.064103
Received: 17 March 2020; Accepted: 18 May 2020; Published: 2 June 2020
JOURNAL ARTICLE
15 PAGES


SHARE
Advertisement
Advertisement
RELATED CONTENT


Back to Top