3.1.

Geometrical Setup and Refractive Index Field

The quantification of the virtual geometry manipulation by optical inhomogeneity in air requires a reference geometry. The geometry choice is guided by numerical needs: a horizontal cylinder guarantees robust conditions for the crucial density simulations based on heat transfer, as a numerically stable convective heat and density flow is building up above the shaft. This is indispensable for the derivation of the RIF. The geometrical dimensions of the simulation setup are outlined in Fig. 1. The cylinder has a diameter of 27 mm and a length of 170 mm. It has a starting temperature of 900°C, 1100°C, or 1250°C, respectively. These parameters are similar to a Tailored Forming workpiece after forming, postulating a slight cooling effect down to 900°C, considering workpiece handling time.

Fig. 1

Geometrical dimensions of the simulation setup in mm with two cross sections of the hot steel cylinder (here: $T_{\text{steel}}=1250°\mathrm{C}$) and resulting inhomogeneous RIF. The RIF was derived from a heat transfer simulation after a simulation time of $t=15\mathrm{s}$. The virtual triangulation sensor comprises a matrix camera and a laser line generator approximated by several discrete laser locations defining a plane via ray tracing. The triangulation angle $\alpha$ is 60 deg. 3-D geometry data is gained via laser light-section method by intersecting laser plane and camera line-of-sight. In order to reveal the effect of the sensor location on the measurement result, the sensor is rotated by an angle $\beta$ (0 deg, 15 deg, 30 deg) around the cylinder axis.

The heat transfer simulation requires the specification of involved materials. As a start, a simple steel monomaterial is chosen for the cylinder geometry in order to limit the simulation complexity. The relevant material parameters are listed in Table 1, e.g., the steel cylinder’s thermal conductivity or specific heat capacity. Humid air at a pressure of 1 atm is postulated as surrounding medium. Furthermore, the expected convective flow is restricted to a laminar character. Turbulences are not reproduced in the model to save calculation costs and in order to keep the analysis of the subsequent ray tracing results as simple as possible. Further information on the used heat transfer equations goes beyond the scope of this paper and can be found in the provided software user guide for the heat transfer module.¹⁴

Table 1

Summary of simulation boundary conditions for heat transfer and ray tracing simulations.

The following simulation routine has been implemented to gain an inhomogeneous 3-D RIF: First, the heat transfer from the hot measurement object into the surrounding air is simulated in order to gain a scalar 3-D density field with locally varying density values. The simulation is stopped after a simulation time of $t=15\mathrm{s}$ since this is the planned maximum time to position the hot measurement object in front of the sensor in an experimental setup. Subsequently, the density values are used to derive a scalar 3-D RIF using the Ciddor equation.⁵ Ciddor introduced an equation for the refractive index in air dependent on wavelength, temperature, pressure, humidity, and ${\mathrm{CO}}_2$ content. By using the ideal gas law and postulating isobaric state, a relationship between density and refractive index can be deduced. This approach is only accurate for moderate temperatures, as the Ciddor equation is only valid up to 100°C. Assuming that a density of $\rho =0\frac{\mathrm{g}}{{\mathrm{cm}}^3}$ results in a refractive index $n=1$, the Ciddor equation can be linearly extrapolated for extreme density values in air that develop near the hot object. An exemplary simulation result for the RIF is displayed in Fig. 1 for a temperature of $T_{\text{steel}}=1250°\mathrm{C}$, revealing the convective density flow above the cylinder and its symmetrical shape. A summary of the hypothesized simulation boundary conditions is given in Table 1.

3.2.

Optical Triangulation in Inhomogeneous Media: Simplified Outline

A simplified outline of a 2-D triangulation measurement setup with RIF effect, illumination unit (laser), and camera sensor is given in Fig. 2. To enhance clarity, the RIF is approximated by discrete air layers with different refractive index values $n_1$, $n_2$, $n_3$, and $n_4$. The air layer directly next to the hot cylinder surface features the lowest refractive index ($n_1$).

Fig. 2

Principle outline of 2-D triangulation measurement with and without inhomogeneous RIF. To enhance clarity, the RIF is approximated by discrete air layers with $n_1$, $n_2$, $n_3$, and $n_4$. A 2-D point is represented by a bold character (e.g., $\mathit{A}$). Index $m$ indicates a measured point. $|(\mathit{B}-{\mathit{B}}_m)|$ is the Euclidean distance between the two points $\mathit{B}$ and ${\mathit{B}}_m$. Blue (solid) line: light path from illumination unit (laser) to object $(\mathit{A})$ and from object to camera with homogeneous RIF. Red (dashed) line: light path with inhomogeneous RIF resulting in laser dot location $\mathit{B}$ on the object’s surface. $|(\mathit{B}-\mathit{A})|$: distance indicating the light deflection on the object’s surface. $|({\mathit{B}}_m-{\mathit{A}}_m)|$ (dotted): distance between unaffected and affected measured laser point locations. $|(\mathit{B}-{\mathit{B}}_m)|$: distance between actual and reconstructed laser point location with inhomogeneous RIF.

For demonstration purposes, the sensor is positioned laterally to the measurement object. A 2-D point is represented by a bold character (e.g., ${\mathit{A}}_m$). Index $m$ indicates a measured point. The blue (solid) line encodes the unaffected light path assuming homogeneous optical conditions, the red (dashed) line encodes the affected path in an inhomogeneous field. The surface of the cylinder is reconstructed by intersecting the activated camera’s line-of-sight with the laser line (or in 3-D with the laser plane), leading to a measurement difference $({\mathit{B}}_m-{\mathit{A}}_m)$ when comparing affected and unaffected scenario. The difference between the actual laser point $\mathit{A}$ to the measured location by triangulation ${\mathit{A}}_m$ in a homogeneous scenario (cold cylinder) is small, if a high triangulation accuracy is assumed. This is indicated by depicting $\mathit{A}$ and ${\mathit{A}}_m$ in the same location (${}_w\mathit{A}\approx {}_w{\mathit{A}}_m$). A loss in geometry information due to the sensor discretization is not considered in the simplified outline Fig. 2.

3.3.

Virtual Triangulation Sensor

The actual simulation has been realized with a virtual 3-D triangulation sensor using the light-section method. It comprises a matrix camera and a telecentric laser line generator (see Fig. 1). The measurement results are given in world coordinate system ${}_{w}K$, if not declared differently. The laser is approximated by several discrete and equidistant laser rays, differing in the ${}_{w}y$-discharge value for ray tracing. As a telecentric laser line generator is used (fan angle is 0 deg), the start vector defining the rays’ tracing direction is assumed to be constant. Laser line generators with fan angles greater than 0 deg would require different ray tracing start vectors to reproduce the beam expansion. The virtual camera’s projection center and the laser are positioned in distance of 300 mm to the world coordinate system ${}_{w}K$. The triangulation angle $\alpha$ is 60 deg. In order to examine the effect of the sensor pose on the measurement result, a rotation angle $\beta$ is defined to adjust the sensor location relatively to the cylinder axis. The exemplary angles are $\beta =0\mathrm{deg}$, 15 deg, and 30 deg.

To keep the simulation routine as simple as possible, the virtual camera is modeled as ideal pinhole camera. This precondition leads to a set of assumptions:

The mathematical description of the mapping of an arbitrary 3-D point ${}_{\mathrm{cam}}\mathit{X}={}_{\mathrm{cam}}{(X,Y,Z)}^T$ in the camera coordinate frame ${}_{\mathrm{cam}}K$ onto the 2-D sensor frame ${}_{img}K$ in pixel position ${}_{\mathrm{img}}\mathit{u}={}_{\mathrm{img}}{(u,v)}^T$ is given in Eq. (2) (e.g., according to Ref. 16, see also Fig. 3):

Fig. 3

(a) Virtual triangulation setup comprising a pinhole camera and a telecentric laser line generator. (b) Viewing direction onto the measurement setup. The laser line generator projects a line onto the cylindrical measurement object. The line is deformed subject to the cylinder’s geometry. This line deformation is captured by the camera. To reconstruct the 3-D data of a measurement point, the line-of-sight through a specific, activated camera pixel is constructed. The resulting line $g$ is intersected with laser plane $E$. $g$ and $E$ must be formulated in the same coordinate frame.

If a 2-D point needs to be reprojected into 3-D space, the scaling factor $\lambda$ needs to be known (the length of the camera’s line-of-sight). To this end, Eq. (2) can be transformed to

The transformation between two different coordinate systems (for instance, between the world and the camera coordinate frame) can easily be realized with the help of transformation matrix $\mathit{T}$, according to the definition in Eq. (4):

The basic triangulation routine is realized by a simple plane-line-intersection, as outlined in Fig. 3(a) (e.g., according to Ref. 18). The exemplary viewing direction onto the displayed triangulation setup is indicated in Fig. 3(b) (with the cylinder cross-section, white arrow). The camera sensor is displayed in front of the camera’s projection center (unlike the depiction in Fig. 2). This is done for demonstration purposes and in order to display the camera according to the mathematical definition of the pinhole model, as given in Eqs. (2) and (3). Although physically not correct, this basic mathematical camera pinhole definition (with sensor in front of the projection center) is commonly used, as it simplifies the description of the mapping of a 3-D point onto the 2-D sensor (image is not upside down, no need for negative signs, e.g., according to Ref. 19, p. 370 ff.).

The mathematical definition of the camera’s line-of-sight in coordinate frame ${}_{\mathrm{cam}}K$ is represented by line $g$, the laser plane is given in the Hessian normal form and is represented by plane $E$. If a laser line is projected onto the measurement object, the line is deformed subject to the object’s geometry. This line deformation is captured by the camera. A specific laser line dot activates a specific camera pixel ${}_{\mathrm{img}}\mathit{u}$. If the camera’s line-of-sight $g$ through this specific pixel is constructed and intersected with the laser plane $E$, the 3-D information of the laser line point can be reconstructed. As laser plane $E$ is given in the simulation in the coordinate frame of the laser ${}_{\text{laser}}K$, it first has to be transformed into the coordinate frame of the camera ${}_{\mathrm{cam}}K$ by an appropriate transformation matrix ${\mathit{T}}_{\text{laser},\mathrm{cam}}$ to intersect $g$ and $E$.

4.1.

Theoretical Background

The following Eqs. (5)–(7) are taken from the provided ray tracing software user guide.¹⁴ A derivation of the presented equations goes beyond the scope of this paper. Nevertheless, the equations are cited to provide physical background information for inhomogeneous ray tracing. More detailed information can be found in Born et al.,²⁰ Saleh and Teich,²¹ and Krueger.²² The ray tracing algorithm in Comsol is deduced from the principles of wave optics. Basic assumptions are that the electromagnetic ray is observed at locations far from the light source and that its amplitude changes very slowly with time and place. The electromagnetic field can therefore be approximated locally by plane waves. The mathematical description of the amplitude is neglected. In this case, the rays’ phase is nearly linearly dependent on time and position according to

The equations need to be solved with respect to $\mathit{k}$ and $\mathit{r}$ to calculate ray trajectories in inhomogeneous media. Fermat’s principle can be gained from these equations, using the so-called Eikonal.²¹ Fermat’s principle is defined based on the path of light, but not on the path direction. This means that the light path can be simulated in either way, as long as it passes the same two points: from object to camera or inversely from camera to object. This so-called inverse principle¹³ is helpful for the iterative ray tracing optimization: starting point for ray tracing is the camera and not the laser light point on the measurement object.

4.2.

Measurement Simulation with Iterative Ray Tracing Optimization

An iterative approximation of the light path from laser incidence location on the cylinder surface to camera needs to be implemented in order to approximate the corresponding camera pixel location on which the laser dot is projected on. Alternatively, referring to the inverse principle, the camera can be starting point for the iterative approximation, as light takes the same way from point $G_1$ to $G_2$ as from point $G_2$ to $G_1$ [see Eq. (1)].

Provided the inhomogeneous RIF around the cylinder has been derived, the measurement simulation for a single data point can be summed up by the subsequent steps, referring to the parameter labeling in Fig. 4.

Fig. 4

Approximation of laser dot location ${}_w{\mathit{B}}_l={}_w(B_{l,x},B_{l,y},B_{l,z})$ by camera line-of-sight via multi-step ray tracing optimization. For demonstration purposes, only the $zy$-plane of the world coordinate system ${}_{w}K$ is illustrated. At the beginning of the optimization process, the directional vector for ray tracing ${}_{\mathrm{cam}}{\mathit{a}}_1$ is constructed through camera projection center $C_{\mathrm{proj}}$ and start pixel ${}_{\mathrm{img}}{\mathit{u}}_s={}_{\mathrm{img}}{\mathit{u}}_1={}_{\mathrm{img}}{(u_1,v_1)}^T$. ${}_{\mathrm{cam}}{\mathit{a}}_k$ is iteratively adapted in dependency of the distance ${}_h{d}_k=|{}_h{(B_{k,x},B_{k,y})}^T-{}_h{(B_{l,x},B_{l,y})}^T)|$ between the target location ${}_h{\mathit{B}}_l$ and the actual position on the cylinder surface ${}_h{\mathit{B}}_k$ in helper coordinate system ${}_{h}K$, parallel to the camera sensor. By comparing the corresponding $x$- and $y$-values, the new pixel location is defined. Example for $x$-direction, in case of ${}_h{d}_1<{}_h{d}_{\min }$: Determine distance of step $k=1$ via ${}_h\mathrm{\Delta }{B}_{1,x}={}_h{B}_{1,x}-{}_h{B}_{l,x}$. ${}_h\mathrm{\Delta }{B}_{1,x}\le 0$, therefore the pixel $u$-location for the next iteration step $k=2$ needs to be adapted according to ${}_{\mathrm{img}}{u}_2={}_{\mathrm{img}}{u}_1+\mathrm{\Delta }{u}_1$. Iteration step $k=2$, assuming ${}_h{d}_2<{}_h{d}_1={}_h{d}_{\min }$: Actual distance is ${}_h\mathrm{\Delta }{B}_{2,x}={}_h{B}_{2,x}-{}_h{B}_{l,x}$. As ${}_h\mathrm{\Delta }{B}_{2,x}>0$, the new pixel location is ${}_{\mathrm{img}}{u}_3={}_{\mathrm{img}}{u}_2-\mathrm{\Delta }{u}_2$. $\mathrm{\Delta }{u}_k$ is reduced with every iteration step. The optimization procedure is stopped, if the absolute value of the Euclidean norm ${}_h{d}_k$ is smaller than the maximum deviation radius ${}_h{r}_{\max }$ acceptable. Another stop criterion is the maximum number of iteration steps $k$.

The main challenge arises from step 2 in which the projected laser dot location in terms of pixel location ${}_{\mathrm{img}}{\mathit{u}}_l$ is approximated. As an idealized pinhole model is hypothesized, light mapped onto the 2-D camera sensor is forced to pass projection center $C_{\mathrm{proj}}$. In a first step, the start pixel ${}_{\mathrm{img}}{\mathit{u}}_s={}_{\mathrm{img}}{\mathit{u}}_1$ is calculated by linearly mapping ${}_w{\mathit{B}}_l$ onto the camera sensor with the help of Eqs. (2) and (4). Due to the pinhole assumption, a directional vector ${}_{\mathrm{cam}}{\mathit{a}}_k$ can be constructed through projection center $C_{\mathrm{proj}}$ and ${}_{\mathrm{img}}{\mathit{u}}_1$, leading to the light discharge direction for ray tracing in iteration step $k=1$. After the initial ray tracing simulation (step 2.3), the actual distance ${}_h{d}_k=|{}_h{(B_{k,x},B_{k,y})}^T-{}_h{(B_{l,x},B_{l,y})}^T)|$ between the actual light incidence location ${}_w{\mathit{B}}_k$ and the target location ${}_w{\mathit{B}}_l$ is calculated in helper coordinate system ${}_{h}K$ (step 2.4). There is no need to compare the $z$-values, as depth information is lost when imaging. If the condition ${}_h{d}_k<{}_h{d}_{\min }$ is fulfilled, ${}_h{d}_{\min }$ being the smallest distance value so far, both ${}_{\mathrm{img}}{\mathit{u}}_{\min }$ and ${}_h{d}_{\min }$ are updated with the actual step values. Provided the maximum number of iterations $k_{\text{stop}}$ is not reached and ${}_h{d}_{\min }$ is not smaller than the maximum deviation radius ${}_h{r}_{\max }$ allowed, the new pixel location ${}_{\mathrm{img}}{\mathit{u}}_{k+1}$ is determined according to step 2.5 with iterative pixel step size $\mathrm{\Delta }{\mathit{u}}_k$. $\mathrm{\Delta }{\mathit{u}}_k$ is adapted in dependency of the actual iteration step $k$ and the width $w_{sg}$ and height $h_{sg}$ of the pixel search grid. To prevent an erroneously mapping of undercut points onto the camera sensor, the $z$-distance is finally checked in step 3.1 by calculating the Euclidean norm $|{}_h\mathrm{\Delta }{B}_{k,z,\min }|=|{}_h{B}_{k,z,\min }-{}_h{B}_{l,z}|$. If $|{}_h\mathrm{\Delta }{B}_{k,z,\min }|$ is bigger than an initially defined threshold value ${}_h{z}_{\max }$, the corresponding camera point is not used for triangulation.

In an experimental (not simulated) triangulation measurement, the limited lateral resolution of the camera sensor restricts the exact mapping of a 3-D world point onto the sensor. Furthermore, a light section measurement depends on the accurate localization of the laser’s center line in the camera image, e.g., by fitting Gaussian distribution curves into the laser line’s intensity profiles. This approach permits subpixel accuracy. To take this discrete and virtual increase in pixel number into account, the value for ${}_{\mathrm{cam}}{\mathit{u}}_{\min }$ is rounded to a virtual pixel size of 0.25 pixel. As the lateral resolution of the camera sensor is $22\mu \mathrm{m}$ (compare to Table 1), the maximum deviation radius ${}_h{r}_{\max }$ is set to a value of $1\mu \mathrm{m}$. A stricter threshold is not necessary, as even the assumed subpixel accuracy of 0.25 pixel only allows a mapping of areas of $5.5\mu \mathrm{m}\times 5.5\mu \mathrm{m}$ onto the camera sensor.

5.1.

Cylinder Geometry Measurement by Light-Section Method

The steel cylinder temperature is set to 1250°C. The triangulation sensor is not rotated around the cylinder axis ($\beta =0\mathrm{deg}$) to realize a measurement from above under full influence of the RIF (see Fig. 1, right side). Nine discrete light paths from laser to camera are simulated—only differing in the ${}_{w}y$-discharge location (equidistantly arranged from 0 to $-10\mathrm{mm}$) but with same directional vectors (laser with telecentric lens). The parameter nomenclature is given in Fig. 5, and the corresponding simulation results are depicted in Figs. 6 and 7. The laser incidence location on the cylinder for a homogeneous RIF (cold cylinder) is given in the world coordinate frame ${}_{w}K$ as ${}_w\mathit{A}$, the laser incidence location for an inhomogeneous RIF (hot cylinder) as ${}_w\mathit{B}$. The corresponding locations on the camera sensor are ${}_{\mathrm{img}}\mathit{B}$ and ${}_{\mathrm{img}}\mathit{A}$ and given in pixel in the coordinate frame ${}_{\mathrm{img}}K$. Normally, the pixel location on the sensor is defined by the letter $\mathit{u}$. This nomenclature is deviated from in this section to ensure a clear distinction of parameters and in order to avoid the introduction of further indexes. The measured 3-D points for cold and hot cylinder scenario are ${}_w{\mathit{A}}_m$ and ${}_w{\mathit{B}}_m$ (see Fig. 5). The results in Fig. 7 are given as distances between two points [e.g., $({}_w\mathit{B}-{}_w\mathit{A})$], whereas not only the difference between the scalar entries [e.g., $\mathrm{\Delta }x$, $\mathrm{\Delta }y$, and $\mathrm{\Delta }z$] are presented but also the 2-D or 3-D Euclidean norms of the distances between two points (e.g., ${}_h{d}_{\min }$, $d_{\text{euclid},xyz}$, or $d_{\text{euclid},uv}$).

Fig. 5

Basic triangulation setup with parameter nomenclature in world coordinate frame ${}_{w}K$. (a) Lateral view. (b) Frontal view on cylinder cross-section. $\mathit{A}$ parametrizes a point for homogeneous measurement conditions and $\mathit{B}$ for inhomogeneous conditions (hot cylinder). Index $m$ indicates a measured (triangulated) point.

Fig. 6

Distance ${}_h{d}_{\min }=|{}_h\mathrm{\Delta }{\mathit{B}}_{xy,\min }|$ between real and optimized laser point locations, given in the helper coordinate frame ${}_{h}K$. Simulation result for a triangulation measurement from above ($\beta =0\mathrm{deg}$) and a cylinder temperature of $T_{\text{steel}}=1250°\mathrm{C}$ (compare to setup depicted in Fig. 1). The parameter nomenclature is given in Fig. 5.

Fig. 7

Simulation results for a triangulation measurement from above ($\beta =0\mathrm{deg}$) and a cylinder temperature of $T_{\text{steel}}=1250°\mathrm{C}$ (see setup depicted in Fig. 1). The parameter nomenclature is given in Fig. 5. $\mathit{A}$ parametrizes a point for homogeneous measurement conditions (cold cylinder), $\mathit{B}$ a point for inhomogeneous conditions (hot cylinder) in the world coordinate frame ${}_{w}K$. (a) Distance between (actual) laser points on cylinder surface (hot and cold cylinder). (b) Distance between actual and measured laser points for inhomogeneous measurement conditions (hot cylinder). (c) Distance between measured (triangulated) laser points on cylinder surface (hot and cold cylinder). (d) Laser point displacement on the camera sensor in ${}_{\mathrm{img}}K$ (hot and cold cylinder). (e) Distance between actual and measured laser points for homogeneous measurement conditions (cold cylinder).

In an experimental triangulation measurement, a camera sensor always operates as low pass filter, as information is lost due to discretization. The difference between the actual laser point location ${}_w\mathit{A}$ to the measured location by triangulation ${}_w{\mathit{A}}_m$ in a homogeneous scenario (cold cylinder) is relatively small, as a subpixel accuracy of $\mathrm{\Delta }u=\mathrm{\Delta }v=0.25\text{pixel}$ is assumed for the detection of the points and no further light deflection is induced by the surrounding medium air in a cold scenario. This is indicated in the measurement outline in Fig. 5 by depicting ${}_w\mathit{A}$ and ${}_w{\mathit{A}}_m$ in the same location (${}_w\mathit{A}\approx {}_w{\mathit{A}}_m$).

The term quasi-continuous indicates the fact that a result is given not rounded to the camera sensor’s discretization limitation of 0.25 pixel but according to the output of the ray tracing optimization routine. The iteration step $k$ defines the optimization routine’s variable pixel step size (compare to step 2.5 in Sec. 4.2). Therefore, depending on the routine’s stop criteria, the sensor pixel on which the laser dot is projected on, is determined more accurately than given by the sensor’s subpixel accuracy (0.25 pixel). As this more accurate pixel location is still not continuous—the iteration does stop at a discrete value $k$—the nonrounded results are called quasi-continuous. As the RIF induced deflection effects are easier to interpret without sensor discretization, the simulated data in Fig. 7 are given for quasi-continuous sensor conditions. Discrete results (rounded to 0.25 pixel) are discussed later.

The quality of the iterative ray tracing optimization according to Sec. 4.2 is checked by analysis of Fig. 6: The maximum deviation radius ${}_h{r}_{\max }$ has been set to $1\mu \mathrm{m}$. Therefore, the difference ${}_h{d}_{\min }=|{}_h\mathrm{\Delta }{\mathit{B}}_{xy,\min }|$ between the actual laser incidence location ${}_h\mathit{B}$ and the optimized location ${}_h{\mathit{B}}_{\min }$ in the helper coordinate frame ${}_{h}K$ has to be smaller than this threshold value. This is the case, as all values for ${}_h{d}_{\min }$ stay below $1\mu \mathrm{m}$.

The simulated curve in Fig. 7(a) depicts the displacement of the laser incidence location (${}_w\mathit{B}-{}_w\mathit{A}$) on the cylinder surface in the world coordinate frame ${}_{w}K$. The curve reveals the effect of the cylinder curvature on the measurement: With decreasing laser ${}_{w}y$-discharge values, $\mathrm{\Delta }z$ and $\mathrm{\Delta }y$ are continuously getting smaller (the absolute value is increased). This is due to the changing cylinder’s surface gradient $\frac{\mathrm{\Delta }z}{\mathrm{\Delta }y}$, when distancing from the origin of coordinate frame ${}_{w}K$ in negative ${}_{w}y$-direction [see Fig. 5(b), the cylinder’s “shoulder slope” is getting steeper]. The $\mathrm{\Delta }x$ value increases [see definition of ${}_{w}x$-axis in Fig. 5(a)]. This geometry effect due to the cylinder curvature is not or only slightly obeyed to for ${}_{w}y=0\mathrm{mm}$. In this case, the laser’s start vector (for ray tracing) is directly pointing on the origin of coordinate frame ${}_{w}K$. The surface gradient $\frac{\mathrm{\Delta }z}{\mathrm{\Delta }y}({}_{w}y=0)$ is 0. Therefore, the cylinder’s curvature does not “boost” small RIF-induced deflection values, if a light ray hits the surface in the vicinity of the origin of coordinate frame ${}_{w}K$.

The simulated data in Fig. 7(b) gives information on the difference between the actual laser incidence location on the hot cylinder surface ${}_w\mathit{B}$ and the measured point ${}_w{\mathit{B}}_m$. A thought experiment helps to reveal the significance of the data: Provided the light deflection from laser to object only happens inside the laser plane, and no further deflection occurs from object to camera, the measured point ${}_w{\mathit{B}}_m$ would not differ from the real point ${}_w\mathit{B}$. This means that in the very unlikely event of a pure laser plane-bound light deflection, and if no deflection from object to camera is induced at all, the correct 3-D point would be triangulated. Figure 7(b) proves that this is not the case for the simulated triangulation measurement. Furthermore, the great Euclidean distances up to $75\mu \mathrm{m}$ between measured 3-D point ${}_w{\mathit{B}}_m$ and real 3-D point ${}_w\mathit{B}$ demonstrate clearly that the measurement deviation due to an inhomogeneous RIF cannot be neglected. The measurement result for homogeneous conditions [compare to Fig. 7(e)] shows a maximum deviation of $\sim 2.7\mu \mathrm{m}$ for $d_{\text{euclid},xyz}$. The results are given rounded to the camera sensor’s discretization limitation of 0.25 pixel. This allows a mapping of areas of $5.5\mu \mathrm{m}\times 5.5\mu \mathrm{m}$ on a quarter of a pixel. The maximum discretization error by rounding is therefore 2.75 pixel, which matches the maximum value of $d_{\text{euclid},xyz}$ in Fig. 7(e). As no further light deflection is induced by the surrounding medium air in the simulation scenario with cold cylinder, the absolute distance between ${}_w\mathit{A}$ and ${}_w{\mathit{A}}_m$ is very small and can be explained exclusively by sensor discretization (${}_w\mathit{A}\approx {}_w{\mathit{A}}_m$).

The data in Fig. 7(b) gives no information on whether the 3-D point ${}_w{\mathit{B}}_m$ is accidentally a point of the cylinder surface. In order to verify this, the closest distance between ${}_w{\mathit{B}}_m$ and the numerical cylinder surface has to be calculated. The additional merit of such an analysis is restricted, as it only slightly helps to understand the deflection in the RIF.

Within the scope of this work, the measured points ${}_w{\mathit{A}}_m$ in cold cylinder state are used as reference data for the evaluation of the measured points ${}_w{\mathit{B}}_m$ in hot state. The difference between theses points is depicted in Fig. 7(c). The measurement results are correlated to the laser point displacement on the camera sensor [compare norms $d_{\text{euclid},xyz}$ and $d_{\text{euclid},uv}$ in Fig. 7(c) and (d)], as a specific laser point location on the sensor is used to derive the camera’s line-of-sight in order to reconstruct the point’s 3-D data via line-plane-intersection (see Sec. 3.3). Oddly enough, an increasing laser point displacement on the cylinder surface [see Fig. 7(a)] does not necessarily result in increasing differences between the measured points $({}_w{\mathit{B}}_m-{}_w{\mathit{A}}_m)$ [see Fig. 7(c)]. The values for the difference $({}_w{\mathit{B}}_m-{}_w{\mathit{A}}_m)$ show a maximum value in region (I) for the laser ray with ${}_{w}y=0\mathrm{mm}$. The difference decreases until it reaches the lowest value in region (II), only to increase again in region (III) [compare to Figs. 7(c) and 7(d)]. A detailed analysis of the simulated norms $d_{\text{euclid},xyz}$ and $d_{\text{euclid},uv}$ is given in the next subsection to explain this apparent contradiction.

5.2.

Analysis: Superimposition of Light Deflection

To allow an interpretation of the difference $({}_w{\mathit{B}}_m-{}_w{\mathit{A}}_m)$ according to Fig. 7(c), the laser point displacement on the camera sensor $({}_{\mathrm{img}}\mathit{B}-{}_{\mathrm{img}}\mathit{A})$ in Fig. 7(d) has to be analyzed. The laser light displacement is a superimposition of the deflection in two different paths: the deflection from laser to cylinder surface and the deflection from cylinder surface to camera. If both paths’ deflection effects are opposed to each other, the resulting pixel displacement on the sensor is reduced. Therefore, the decrease in region (II) in Fig. 7(d) can be explained. The basic procedure to separate the paths’ deflection effects is depicted in Figs. 8(a)–8(c). An information in advance to avoid irritation: the deflection of point ${}_w\mathit{B}$ is not necessarily restricted to the laser plane [as depicted in Fig. 8(a), compare to red, dashed line]. This is for demonstration purposes only.

Fig. 8

Approach to separate the laser light displacement on the camera sensor into two separate parts: the deflection induced in the path “laser to object,” and the deflection from “object to camera.” (a) Displacement $({}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}-{}_{\mathrm{img}}\mathit{A})$ induced by path “laser to object”: ${}_w\mathit{A}$ (blue, solid line) and ${}_w\mathit{B}$ (red, dashed line) are linearly projected onto the camera sensor, resulting in two corresponding pixel locations ${}_{\mathrm{img}}\mathit{A}$ (blue, solid line) and ${}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}$ (red, solid line) on the sensor. (b) Displacement $({}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}-{}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}})$ induced by path object to camera: The linear projection of ${}_w\mathit{B}$ onto the camera sensor in location ${}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}$ is compared to the nonlinear, RIF-affected projection in location ${}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}$ (see red, dashed line from ${}_w\mathit{B}$ to camera). (c) Displacement $({}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}-{}_{\mathrm{img}}\mathit{A})$ induced by path “laser to camera”: Both pixel displacement values are superimposed, resulting in the total displacement value.

To gain the laser light displacement for the path “laser to object,” both ${}_w\mathit{A}$ (blue, solid line) and ${}_w\mathit{B}$ (red, dashed line) are linearly projected onto the camera sensor, resulting in two corresponding pixel locations ${}_{\mathrm{img}}\mathit{A}$ (blue, solid line) and ${}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}$ (red, solid line) [compare to Fig. 8(a)]. By doing this, the deflection induced in the path “object to camera” is not taken into account. This deflection effect is derived according to image Fig. 8(b): The linear projection of ${}_w\mathit{B}$ onto the camera sensor in location ${}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}$ is compared to the nonlinear, RIF-affected projection in location ${}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}$ (see red, dashed line from ${}_w\mathit{B}$ to camera). If now both pixel displacement values are superimposed [see Fig. 8(c)], the resulting displacement is $({}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}}-{}_{\mathrm{img}}\mathit{A})+({}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}-{}_{\mathrm{img}}{\mathit{B}}_{\text{laser}\to \text{object}})=({}_{\mathrm{img}}{\mathit{B}}_{\text{object}\to \mathrm{cam}}-{}_{\mathrm{img}}\mathit{A})$. The superimposition must lead to the results depicted in Fig. 7(d). A special scenario is depicted in Fig. 8(c): the resulting pixel displacement from laser to camera can be close to zero, leading a small difference between ${}_w{\mathit{B}}_m$ and ${}_m{\mathit{A}}_m$, even though the light deflection in the two different paths is non-neglectable [see Fig. 8(c), ${}_w{\mathit{B}}_m\approx {}_w\mathit{A}\approx {}_w{\mathit{A}}_m$]. The depicted approach in Fig. 8 is nevertheless also valid for points ${}_w{\mathit{B}}_m$ and ${}_m{\mathit{A}}_m$, which are reconstructed in different locations.

The suggested routine has been realized for the pixel displacement in Fig. 7(d) and is outlined in Fig. 9. Not only the laser point displacement on the camera sensor is given for both light paths [see Figs. 9(a)–9(c)] but also a graphical interpretation of the displacement on the sensor for the laser light path corresponding to ${}_{w}y=-7.5\mathrm{mm}$ [see Figs. 9(d)–9(f)]. The displacement values [e.g., $\mathrm{\Delta }{v}_{\text{laser}\to \text{object}}$ or $\mathrm{\Delta }{u}_{\text{laser}\to \text{object}}$] are marked in Figs. 9(a)–9(c).

Fig. 9

Laser point displacement on the camera sensor for a measurement scenario from above according to results in Fig. 7(d), separated into the deflection induced in different light paths: (a) for the light path “laser to object,” (b) for the path “object to camera,” and (c) for the complete path “laser to camera.” (d)–(f) An exemplary graphical interpretation of the displacement on the sensor for the laser ray with ${}_{w}y=-7.5\mathrm{mm}$ is given. (a)–(c) The corresponding displacement values $\mathrm{\Delta }u$ and $\mathrm{\Delta }v$ are marked.

First of all, the resulting pixel displacement by superimposition in Fig. 9(c) is the same as in Fig. 7(d), the suggested approach is therefore legitimate. Moreover, especially the progression of the pixel displacement in the $u$-direction indicates that the induced light deflection values are opposite to each other: $\mathrm{\Delta }{u}_{\text{laser}\to \text{object}}$ is decreasing to a pixel value of approximately $-4$ for ${}_{w}y=-10\mathrm{mm}$ [Fig. 9(a)], whereas $\mathrm{\Delta }{u}_{\text{object}\to \mathrm{cam}}$ increases to a value of more than 2 pixel [Fig. 9(b)]. The resulting value $\mathrm{\Delta }{u}_{\text{laser}\to \mathrm{cam}}$ varies around a value of $-0.5\text{pixel}$ [Fig. 9(c)].

The value for $d_{\text{euclid},uv}({}_{w}y=-7.5\mathrm{mm})$ in Fig. 9(c) is therefore not contradictory: the consideration of both light paths results in a reduction of pixel displacement [see also graphical interpretation in Figs. 9(d)–9(f)], which again leads to a reduced difference for the gained values $d_{\text{euclid},uv}$ in region (II) in Fig. 7(d). This special scenario is exemplary depicted in Fig. 8(c): the resulting 3-D point ${}_w{\mathit{B}}_m$ (hot cylinder) is depicted in the same location as point ${}_w{\mathit{A}}_m$ (cold cylinder). As the resulting pixel displacement is again increasing for light rays with ${}_{w}y<-7.5\mathrm{mm}$, also the distance between ${}_w{\mathit{B}}_m$ and ${}_w{\mathit{A}}_m$ rises.

To gain a deeper understanding of an exemplary light refraction scenario, the rounded (discrete) interpretation of Fig. 7(c) is analyzed for the laser ray with a discharge value of ${}_{w}y=0\mathrm{mm}$. The graphical result of this analysis is given in Fig. 10(a), based on the rounded pixel displacement [to sensor subpixel accuracy of 0.25 pixel, Fig. 10(b)]. First of all, there is only a slight difference between the graphs in Figs. 7(d) and 10(b). The difference would be bigger, if the subpixel accuracy was further limited—for instance, to a value of 0.5 pixel. The laser light path for a ${}_{w}y$-discharge value of 0 mm is only marginally deflected in the ${}_{w}y$-direction due to the symmetry of the RIF to the $xz$-plane (in ${}_{w}K$). The cylinder curvature only has small influence on the resulting incidence location ${}_w\mathit{B}$ on the cylinder surface. Therefore, only the $xz$-plane is depicted in the graphical analysis in Fig. 10(a).

Fig. 10

(a) Exemplary graphical interpretation of the light refraction scenario for the laser ray with ${}_{w}y=0\mathrm{mm}$, (b) based on the rounded pixel displacement on the camera sensor.

When the laser light enters the inhomogeneous RIF from the left side, the ray is deflected downward toward more dense air layers, where greater refractive index values are present [see vertical black lines in Fig. 10(a), the lines separate areas with different refractive index value]. This leads to the dashed light path. The closer the ray moves toward the cylinder, the more predominates a horizontal expansion and variation in the refractive index (see horizontal black lines). Therefore, the ray is refracted away from the cylinder, where the surrounding air is more dense. The ray reaches the cylinder in location ${}_w\mathit{B}$. As the homogeneous RIF is basically symmetric to the $zy$-plane of the world coordinate frame ${}_{w}K$ (see Fig. 1), the path from cylinder to camera is flipped vertically in the graphical interpretation, according to Fig. 10(a). Based on this simplification, the triangulated measurement point ${}_w{\mathit{B}}_m$ is reconstructed above the real cylinder surface. This matches the gained result for $\mathrm{\Delta }z$ and $\mathrm{\Delta }x$ [see Fig. 10(b)].

The interpretation in Fig. 10(a) explains the simulation result in Fig. 10(b) for the laser ray with discharge value of ${}_{w}y=0\mathrm{mm}$. It also reveals the complexity of light refraction in an inhomogeneous RIF.

5.3.

Comparison: Different Cylinder Temperatures and Sensor Poses

The result section is closed by a comparison of different measurement scenarios. To this end, the steel cylinder temperature (900°C, 1100°C, 1250°C) and the triangulation sensor pose are varied (0 deg, 15 deg, 30 deg).

In Figs. 11(a)–11(c), different data curves for a measurement from above with $\beta =0\mathrm{deg}$ are depicted, revealing the influence of a temperature increase: The pixel displacement on the camera sensor and the measurement difference (${}_w{\mathit{B}}_m-{}_w{\mathit{A}}_m$) indicates an increase due to temperature only for the measurement points corresponding to the laser rays with ${}_{w}y$-discharge values from 0 to $-5\mathrm{mm}$. This is due to the expansion of the inhomogeneous RIF (see Fig. 1, right side with cylinder cross-section): As the RIF variation develops its full effect directly above the hot cylinder due to the convective density flow, the spatial region in which a cylinder temperature increase takes effect, is expanded widely. Smaller light ray ${}_{w}y$-discharge values do not lead to differences ($-10\mathrm{mm}<{}_{w}y<-5\mathrm{mm}$), except for ${}_{w}y=-10\mathrm{mm}$. This might be explained by the analysis in Sec. 5.2: the resulting light deflection is reduced, due to the superimposition in the path from laser to object and from object to camera. This effect, based on the symmetry of the RIF, is not affected by a temperature increase, as the symmetry of the RIF does not change.

Fig. 11

Simulated measurement results for different cylinder temperatures and sensor poses. (a–b) Measurement from above ($\beta =0\mathrm{deg}$) for a cylinder temperature of 900, 1100 and 1250°C. $d_{\text{euclid},xyz}$ parametrizes the Euclidean 3-D norm of the distances between a triangulated measurement point under homogeneous (${}_w{\mathit{A}}_m$) and inhomogeneous conditions (${}_w{\mathit{B}}_m$). (c) Corresponding laser point displacement on the camera sensor. $d_{\text{euclid},uv}$ defines the Euclidean 2-D norm of the distances between ${}_{\mathrm{img}}\mathit{A}$ and ${}_{\mathrm{img}}\mathit{B}$. (d) Laser point displacement on the camera sensor for different sensor poses ($\beta =0\mathrm{deg},15\mathrm{deg},30\mathrm{deg}$, see Fig. 1), with a cylinder temperature of 900°C.

The discretization effect can be evaluated, when comparing Figs. 10(a) and 10(b). Due to the pixel rounding to a value of 0.25 pixel (subpixel accuracy of the camera sensor), the RIF-induced deflection effect is “discretized” as well. Curve (a.2) shows more abrupt steps than curve (a.1). The triangulation sensor’s resolution therefore affects the “reproduction” of the deflection effect in an inhomogeneous RIF.

The effect of a sensor rotation around the cylinder axis (see Fig. 1, right side) on the laser point displacement is depicted in Fig. 11(d) for a cylinder temperature of 900°C. The displacement is reduced with increasing angle $\beta$. The simulated curves therefore indicate the obvious: if a measurement is not performed directly through the greatest expansion and variation of the inhomogeneous RIF, but rather sideways through less expanded regions, the pixel displacement is reduced and the corresponding measurement more trustworthy. The measurement difference (${}_w{\mathit{B}}_m-{}_w{\mathit{A}}_m$) for the rotated sensor is not depicted: The Euclidean distance is below $20\mu \mathrm{m}$ for all laser rays for $\beta =15\mathrm{deg}$ and below $7\mu \mathrm{m}$ for $\beta =30\mathrm{deg}$.