Parallax correction of texture image in fringe projection profilometry

Abstract. In fringe projection profilometry, the measurement system generally consists of a projector for casting fringes onto a measured object and a monochrome camera for capturing the deformed fringe patterns. In addition to these components, we can add a color camera for capturing the texture of the object simultaneously. For implementing texture mapping on the reconstructed three-dimensional (3-D) surface, the parallax between the views of the texture camera and the measuring camera has to be corrected. For this purpose, we analyze the geometry of the fringe projection system with a color texture camera and further suggest a system calibration method. Using this method, the corresponding relationship between the texture and the 3-D data and the mapping relationship between the depths and the fringe phases are determined simultaneously, so that the duration time for the system calibration implementation is saved. The data processing with this method is of a low computational complexity because it involves only linear minimizations. Using the calibration results, we can transform the texture image from the view of the color camera to that of the measuring camera and precisely map it on the reconstructed object surface. Experimental results demonstrate that this method is effective in correcting the parallax of the texture image.


Introduction
Fringe projection profilometry 1 has been extensively developed to meet the demands of various applications. It has advantages over other methods in being noncontacting and providing whole-field information. With it, periodic fringe patterns are projected onto the object surface, then the distorted patterns caused by the depth variations of the surface are recorded from different directions. Analyzing the distorted patterns allows us to reconstruct the depth map of the object. In addition to the depth measurement, we can capture the object texture and map it on the reconstructed threedimensional (3-D) surface in order to enhance its realism.
Combining with texture mapping, the range of applications of the fringe projection technique is broadened, especially to fields such as virtual reality, 2-4 commercial advertisement and entertainment, 5 medical diagnosis, 6-8 cosmetics, 9 anthropology, 10 and cultural heritage preservation. [11][12][13][14] Many methods have been developed for acquiring the texture of a measured object. Among them, the simplest one is to use a single camera to capture both the deformed fringe pattern and the texture image. As typical examples, Ref. 15 proposed a Gray-code fringe method and Ref. 16 proposed a marker-encoded fringe pattern method for measuring 3-D shapes with abrupt steps. In them, the fringe patterns and the textures are recorded in turn using the same camera. To decrease the time of image capturing in dynamic measurement, the texture is usually calculated from the deformed fringe patterns rather than directly captured by use of the camera. For example, in Fourier transform profilometry, 17 one can recover the depth map from the spectrum of a single fringe pattern and simultaneously retrieve the (blurred) texture image by use of a notch filter or a low-pass filter. [18][19][20] With the phase-shifting technique 21 that requires capturing a sequence of fringe patterns, simply averaging the captured fringe patterns will produce the texture image, i.e., the distribution of the background intensities with fringes removed. [22][23][24] These methods can also be used for calculating a color texture image, which is more realistic than a gray one, as long as a color camera is used and the object is illuminated with white light. 22,23 In the more complex case of a colored light source used to illuminate a colorful object, for example, when using the color-code fringe projection techniques a more sophisticated algorithm is required to retrieve its texture from the colored fringe patterns. 25 Using the methods that use a single camera, however, the Bayer color processing and chromatic balance may destroy the response linearity of the camera, thus leading to difficulties in fringe pattern analysis or degrading the measurement accuracy.
To overcome the aforementioned problems, the measurement system can equip two separate cameras, one monochrome and one color. The monochrome camera with a high linearity is used to capture the deformed fringe patterns in order to guarantee the 3-D measurement accuracy. The color camera is used for capturing the texture; its parameters are adjusted independently, thus adapting the color texture image to the observation of human eyes, which are nonlinear in response. In Ref. 26, a beam splitter is mounted on the optical axis of a black and white (B/W) camera, by which the texture can be captured with a color camera from the same view. In Refs. 27 and 28, a multiple-chip CCD camera is used to capture the infrared fringe patterns and the color texture simultaneously. This camera can be thought of as a combination of two cameras that have different channels but share the same optical axis. With these methods, the texture data can align the recovered 3-D data exactly since the texture and the fringe patterns are recorded from the same view.
In comparison with the aforementioned techniques, a more flexible method is to use two cameras with separate optical axes. 29,30 In Ref. 30 for example, an infrared camera is used to capture the deformed fringe patterns. In addition, a color camera is used for capturing the color texture. This system is much easier to fabricate. However, the texture and the 3-D shape data are obtained from different views, and therefore a parallax exists between them. A solution to this problem is to perform a registration between the texture image and the 3-D model by matching their features (e.g., points, lines, and edges). 31,32 However, this method is not always effective, especially when the features are too weak on the 3-D model. Another method is to employ a binocular geometry. 30 The mapping relationship between object depths and fringe phases is determined by calibrating the projectorcamera system. In addition, the intrinsic and extrinsic parameters of the texture camera and the measuring camera have to be calibrated, so that the corresponding relation between the texture image and the 3-D model can be determined. The mainstream camera calibration techniques 33,34 are flexible to meet the demands of various applications and can also be used to calibrate the texture camera in the fringe projection system. However, these methods usually involve complex nonlinear minimization problems, implying that they are somewhat complicated to implement and program unless existing software is available.
In this paper, we analyze the geometry of the fringe projection system with a color texture camera and suggest a simple calibration method, by which the corresponding relationship between the texture and the 3-D data and the mapping relationship between the phases and depths are determined simultaneously, so that the duration time for implementing the system calibration is saved. In addition, data processing involves only linear minimizations that have low computational complexity. Using the calibration results, we can transform the texture image from the view of the color camera to that of the measuring camera, with its parallax being removed, and then precisely map the texture on the reconstructed object surface. Experimental results demonstrate that this method is valid in correcting the parallax of texture.  Figure 1 shows the measurement system, which mainly consists of a projector, a B/W camera, a color camera, and a stage mounted on a track. The projector casts sinusoidal fringe patterns onto the object surface. The B/W camera (i.e., the measuring camera) is used to record the deformed fringe patterns. This measuring camera should have a linear response to intensities in order to guarantee the accuracy of phase measuring. In addition, the color camera (i.e., the texture camera) is used to capture the color texture of the measured object. In addition to these components, we have a standard plane that serves as the reference plane, a benchmark for depth measurement. In the calibration procedure, it is also used as the calibration board, mounted on the stage and shifted by the track. In Fig. 1, the measuring and texture cameras have different optical axes, and therefore a parallax exists between their views. In other words, the color photograph captured by the texture camera cannot be directly used for texture mapping. In this situation, parallax correction is a necessary step for texture mapping. Through it, the texture image is transformed from the view of the texture camera to that of the measuring camera, thus aligning the reconstructed 3-D data exactly.
The parallax of the texture image depends not only on the geometry of the measurement system, but also on the 3-D shape of the measured object. Parallax correction of the texture image has to be implemented concurrently with 3-D data reconstruction. Therefore, in the next two subsections, we shall introduce the principle of depth measurement, including steps of fringe analysis and the phase to depth conversion. In Sec. 2.4, the relationships between the object lateral coordinates and the camera pixel coordinates will be determined. Using these results allows us to deduce in Sec. 2.5 a method for texture parallax correction.

Phase Measuring
Phase measuring is an important step in measurement. It affects the accuracy of depth reconstruction and of texture parallax correction. There are many methods for extricating phases from fringe patterns. In this paper, we presume that the phase-shifting technique is used. With it, the deformed fringe patterns with phase shifts captured by the measuring camera are represented with E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 3 2 6 ; 2 3 2 I n ðu; vÞ ¼ Jðu; vÞ þ γðu; vÞ cos½ϕðu; vÞ þ 2πn∕N; (1) where N is the number of phase shifts and n ¼ 0; 1; : : : ; N − 1 denotes the phase-shift indexes. ðu; vÞ are the pixel coordinates on the image plane of camera, and Jðu; vÞ and γðu; vÞ denote the recorded background intensity and the modulation at the pixel ðu; vÞ, respectively. ϕðu; vÞ is the phase at ðu; vÞ. The wrapped phase is calculated using a standard phase-shifting algorithm, 21 The phase map calculated using Eq. (2) is not continuous but constrained to its principal value within the range from -π to π rad. The discontinuities in a phase map are usually removed by implementing a two-dimensional (2-D) phase unwrapping. 35 With fringe projection profilometry, however, the conventional spatial phase unwrapping algorithms are not always effective, because the abrupt steps in the measured shape may induce shadows, over-dense fringes, and fringe-order ambiguities in the fringe patterns.
In this work, we use the temporal phase unwrapping algorithm 36 to overcome the problems described above. We project several groups of phase-shifting fringe patterns onto the object. From one group to the next, the spatial frequency of fringes increases by multiplying a fixed ratio 2. In the first group, each pattern contains only one fringe, so there is no ambiguity in the fringe order. The phase map for each group is calculated using Eq. (2), so we obtain a sequence of wrapped phase maps with increased sensitivities. In principle, their unwrapped phases at each pixel should be a geometric progression with a common ratio of 2. Therefore, the phase at each pixel can be unwrapped independently of other pixels over time. Using this temporal phase-unwrapping technique, the unwrapped phase map of the last group, which has the highest sensitivity, is used for measurement purpose. In this procedure, the invalid regions induced by shadows and other factors are segmented and excluded from the fringe patterns by setting a threshold for the fringe modulations. 37 This temporal phase-unwrapping method is very stable in providing an absolute measure of surface height, but it has a drawback that a relatively large number of fringe patterns have to be captured.
A more efficient method that requires fewer fringe patterns is to use the Gray-code technique combining with phase-shifting fringe projection. 38 In this method, the Gray-code fringes are exploited to detect the surface discontinuities without any ambiguities, and the phase-shifting technique is used to achieve a high measurement resolution. The limitation of this technique is that it involves an image binarization procedure, which is sensitive to the illumination nonuniformity and noise.
For convenience, the same phase measurement method is also implemented to the reference plane. The unwrapped phase maps of the object and the reference plane are obtained and represented with ϕ obj and ϕ ref , respectively, so the phase difference is from which the depth map of the measured surface relative to the reference plane can be calculated and the 3-D shape of the object can be reconstructed.

Depth Map Measuring
Fringe projection profilometry is a triangulation-based technique. When the projector and the B/W camera in Fig. 1 are located arbitrarily, the mapping relationship between the depths and phase differences is dependent on the system parameters and becomes very complex. Even so, relationship for each pixel ðu; vÞ, according to the geometric analysis in Ref. 39, can always be formulated with where hðu; vÞ is the object depth relative to the reference plane at the pixel ðu; vÞ. Coefficients aðu; vÞ and bðu; vÞ are the functions of pixel coordinates ðu; vÞ, whose distributions are dependent on the system parameters such as the pitch of fringes, the positions and orientations of the project and camera. The coefficients aðu; vÞ and bðu; vÞ can be determined through a system calibration and saved in look-up tables for use in measurement. According to Eq. (4), the depths of the measured object are reconstructed with E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 3 2 6 ; 6 2 4 zðu; vÞ ¼ −hðu; vÞ ¼ −Δϕðu; vÞ aðu; vÞ − bðu; vÞΔϕðu; vÞ :

Lateral Coordinate Calculating
The depth map hðu; vÞ gives 2.5-dimensional data about the shape of the measured object. It is necessary to calculate the lateral coordinates at each pixel ðu; vÞ. Figure 2 shows a simple geometry of a camera in the system with a reference plane. The u − v plane is the image plane of the camera, and C is its center of lens. For convenience, we assume a fictitious plane at the distance of l away from the point C. The coordinate system O 0 x 0 y 0 z 0 is bound with this fictitious plane, with z 0 axis coinciding with the optical axis of camera, and x 0 and y 0 axes being parallel with u and v axes, respectively. A pair ðx 0 ; y 0 Þ on the x 0 O 0 y 0 plane corresponds to the image pixel ðu; vÞ, with a relation of the form E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 3 2 6 ; 4 2 1 where κ is a scale factor. World coordinate system Oxyz is established with xOy coinciding with the reference plane, so the transform relationship between Oxyz and O 0 x 0 y 0 z 0 is E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 3 2 6 ; 3 3 2 Among the 12 entries of the transform matrix, only 6 are independent. They are determined by the rotations and translations in 6 degrees of freedom.

Parallax Correction of the Texture Image
In Fig. 1, the texture image captured with the color camera cannot be used directly for texture mapping, because the texture and the measuring cameras have different optical axes. Therefore, it is necessary to correct the parallax between them. We denote the pixel coordinates on the image plane of the color camera as ðs; tÞ. Similar to Eqs. (14) and (15), we have the relations between a pixel ðs; tÞ and the 3-D coordinates of its corresponding object point: and E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 7 ; 3 2 6 ; 4 0 5 yðs; tÞ ¼ p 9 þ p 10 s þ p 11 t 1 þ p 1 s þ p 2 t zðs; tÞ þ p 12 þ p 13 s þ p 14 t 1 þ p 1 s þ p 2 t ; where p 1 ; p 2 ; : : : ; p 14 are the coefficients related to the parameters of the color camera. They are determined through a calibration procedure. As shown in Fig. 3, the pixel ðu; vÞ of the measuring camera has a corresponding pixel [sðu; vÞ, tðu; vÞ] in the image plane of the texture camera. These two pixels correspond to the same object point with the coordinates [xðu; vÞ, yðu; vÞ, zðu; vÞ]. Using Eqs. (16) and (17) Using Eqs. (18) and (19) allows us to match the corresponding pixels between the two cameras. As a result, the color texture image from the view of the measuring camera can be calculated, with its intensities of three primary colors being (20) where R, G, and B are the intensities of the primary colors (i. e., red, green, and blue) in the texture image taken by the color camera. Generally, the calculated coordinate pair [sðu; vÞ, tðu; vÞ] does not exactly locate at a pixel of the captured texture image. In this case, we calculate its values from its four closest pixels using a bilinear interpolation. Using Eq. (20), the parallax of the texture image is corrected.

System Calibration
As noted in the previous section, both the 3-D measurement and the texture parallax correction are dependent on coefficients related to the geometry of the measurement system. We implement the calibration in order to obtain these coefficients. We use a standard plane with diffused surface as the calibration board, on which we placed a 2-D array of black circular markers, as shown in Fig. 4. The lateral coordinates of the centers of these markers are previously known as ðX k ; Y k Þ, with k ¼ 1;2; : : : ; K being their indexes and K being the number of the markers. Using this calibration board, the procedure of calibration is summarized as follows.
Step 1: We position the calibration board at z ¼ 0, as shown in Fig. 5, and take the image of the circular marker array using the B/W camera. In the world coordinate system, the center of the k'th circular marker has the coordinates ðX k ; Y k ; 0Þ. In the image, the pixel coordinates of centroid of this marker are calculated as ðU k;0 ; V k;0 Þ. Step 2: At the same calibration board position, we take the image with the texture camera. In this image, the pixel coordinates of centroid of the k'th marker are calculated as ðS k;0 ; T k;0 Þ.
Step 3: According to the principle in Sec. 2.2, we project several groups of phase-shifting fringe patterns, with their frequencies varying from low to high by multiplying a fixed ratio 2, onto the object surface using the projector. We capture the distorted patterns using the B/W camera and analyze them to calculate their absolute phase distribution. In this step, since the markers are black and of low reflectivity, their phases cannot be accurately calculated.
To solve this problem, we segment these markers off from the calculated phase map by setting a threshold for the fringe modulations and then calculate the full-field phase map by fitting the segmentation result with a rational function. 40 This full-field phase map is denoted as ϕ 0 ðu; vÞ, which also serves as the reference phase distribution ϕ ref ðu; vÞ in measurement.
Step 4: As shown in Fig. 5, we shift the calibration board along the perpendicular direction (i.e., z direction), to at least two positions with difference depths h n (n ¼ 1; 2; : : : ; N). In this procedure, the circular markers have the fixed lateral coordinates ðX k ; Y k Þ, and their vertical coordinates are Z k ¼ −h n . Repeating Steps 1 through 3, we have the pixel coordinates of centroids of the markers ðU k;n ; V k;n Þ in the image plane of the measuring camera, and ðS k;n ; T k;n Þ in the image plane of the texture camera, with k ¼ 1; 2; : : : ; K and n ¼ 1; 2; : : : ; N. At the same time, the phase distributions [i.e., ϕ n ðu; vÞ for n ¼ 1; 2; : : : ; N] are also measured.
Step 5: We calculate the phase difference with  with aðu; vÞ and bðu; vÞ being unknowns. Solving it in the least squares sense, we have the distributions of aðu; vÞ and bðu; vÞ. By this step, the mapping relationship between the phase differences and the object depths is determined.
Step 6: Substituting the center coordinates of the markers ðX k ; Y k ; −h n Þ and the centroid coordinates ðU k;n ; V k;n Þ with k ¼ 1; 2; : : : ; K and n ¼ 0;1;:::;N, in Eqs. (14) and (15), produces 2KðN þ 1Þ simultaneous linear equations: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 2 3 ; 6 3 ; 6 4 5 X k U k;n q 1 þ X k V k;n q 2 þ h n q 3 þ h n U k;n q 4 þ h n V k;n q 5 − q 6 − U k;n q 7 − V k;n q 8 ¼ −X k Y k U k;n q 1 þ Y k V k;n q 2 þ h n q 9 þ h n U k;n q 10 þ h n V k;n q 11 − q 12 − U k;n q 13 − V k;n q 14 ¼ −Y k ; (23) from which the coefficients q 1 ; q 2 ; : : : ; q 14 are estimated in the least squares sense.
Step 7: Similar to Step 6, substituting ðX k ; Y k ; −h n Þ and ðS k;n ; T k;n Þ with k ¼ 1; 2; : : : ; K and n ¼ 0; 1; : : : ; N, in Eqs. (16) and (17), we have 2KðN þ 1Þ simultaneous linear equations: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 2 4 ; 6 3 ; 5 5 4 X k S k;n p 1 þ X k T k;n p 2 þ h n p 3 þ h n S k;n p 4 þ h n T k;n p 5 − p 6 − S k;n p 7 − T k;n p 8 ¼ −X k Y k S k;n p 1 þ Y k T k;n p 2 þ h n p 9 þ h n S k;n p 10 þ h n T k;n p 11 − p 12 − S k;n p 13 − T k;n p 14 ¼ −Y k ; (24) from which the coefficients p 1 ; p 1 ; : : : ; p 14 are estimated in the least squares sense.
Through this calibration procedure, the corresponding relationship between the texture and the 3-D data and the mapping relationship between the phases and depths are determined simultaneously. The data processing involves only linear minimizations that have low computational complexity. Even so, it should be noted that with this procedure we have to adjust the orientation of the calibration board carefully in order to make it exactly perpendicular to the direction of the track.

Three-Dimensional Measurement and Texture Parallax Correction
When the measurement system is calibrated, we can use it for measuring the 3-D shape and correcting the parallax of texture and mapping it on the 3-D shape. Its result will be a sixdimensional array, with [xðu; vÞ, yðu; vÞ, zðu; vÞ] describing the 3-D data and [R 0 ðu; vÞ, G 0 ðu; vÞ, B 0 ðu; vÞ] representing the color texture. The overall procedure is summarized as follows: Step 1: Capture the texture image with the color camera.
Step 2: Project several groups of phase-shifting fringe patterns, with their frequencies increasing by a fixed ratio 2, onto the object surface using the projector. Capture the distorted fringe patterns with the B/W camera.
Step 3: Analyze the captured fringe patterns using the phase-shifting method and the temporal phaseunwrapping technique introduced in Sec. 2.2, in order to retrieve the object phase map ϕ obj ðu; vÞ; calculate the phase difference Δϕðu; vÞ using Eq. (3); and, finally, calculate the z coordinates through Eq. (5) whose coefficients aðu; vÞ and bðu; vÞ were calibrated in the previous subsection.
Step 4: Using the pixel coordinates ðu; vÞ and the vertical coordinates zðu; vÞ obtained in Step 3, we calculate the lateral coordinates of object points [i.e., xðu; vÞ and yðu; vÞ] through Eqs. (14) and (15) 1024 pixels)]. The focal length of lenses (Computar M1620-MPV with F1.6) is 16 mm. The distance between the camera and the measured object is around 800 mm. In both the system calibration and the measurement, we use the phase-shifting technique and the temporal phase-unwrapping technique introduced in Sec. 2.2 for analyzing the distorted fringe patterns. Here, we use eight groups of phaseshifting fringe patterns with their spatial frequencies varying from low to high by multiplying a fixed ratio 2. Each pattern in the first group contains only one fringe with no ambiguity in its fringe order, and consequently each pattern in the last group contains 128 fringes. Only the phase map of the last group, which has the highest sensitivity, is reserved for measurement purpose. According to the sampling theory of phase-shifting technique, the N-frame algorithm is insensitive to the harmonics up to N − 2 order 41,42 induced by the luminance nonlinearity 43 of the projector. Therefore, in the last group the number of phase steps is 8, and the phase increment between consecutive frames is π∕4 rad in order to guarantee the measurement accuracy. In each group before the last, the number of phase steps is 4 in order to reduce the number of fringe patterns.
We begin the experiment by calibrating the measurement system using the procedure presented in Sec. 3.1. Figure 6 shows the calibration board with a circular marker array, on which the space between two neighboring markers is 40 mm in both horizontal and vertical directions. In the calibration procedure, the calibration board is perpendicularly shifted, within the depth of the focus of the cameras, to nine different depth positions with the depth increment between two positions being 5 mm. At each calibration board position, we take its photograph using the color camera and take the deformed fringe patterns using the B/W camera. Figures 7(a)  and 7(b), respectively, show a photograph of the color camera, and one of the fringe patterns captured using the B/W camera, when the calibration board is at the position z ¼ 0. By comparing their marker positions, the parallax between the views of the two cameras is evident. From the captured fringes patterns, we calculate the absolute phase map for each calibration board position and then estimate the coefficients aðu; vÞ and bðu; vÞ, whose distributions are illustrated in Fig. 8. These coefficients determine the mapping relationship between the depths and phase differences through Eq. (5). From the images of circular markers, we estimate their centroids and further calculate the coefficients q 1 ; q 2 ; : : : q 14 for the B/W camera and p 1 ; p 2 ; : : : ; p 14 for the color camera. They are used for determining the relationship between pixel coordinates and the lateral coordinates of the object points.
The first experiment examines the accuracy of this method by using the calibration board in Fig. 6 as the   measured object. As mentioned in Sec. 2, parallax correction of the texture image must be implemented concurrently with the depth reconstruction because the parallax is dependent on the 3-D shape of the object. We position the measured board at 0, 16, and 32 mm depths. By measuring it and correcting its texture image, we can evaluate the accuracies of depth reconstruction and the texture parallax correction.
For example, when the measured board is positioned at the depth of 16 mm, one of the fringe patterns captured with the measuring camera is shown in Fig. 9(a). From these fringe patterns, we recover the unwrapped phase map by using the method in Sec. 2.2, and its result is shown in Fig. 9(b). Subtracting the reference phases from it yields the phase differences, whose distribution is illustrated in Fig. 9(c). Furthermore, we calculate the depth map through Eq. (5), giving the result shown in Fig. 10. From Sec. 2.5, we know that the depth measurement accuracy strongly affects texture parallax correction through Eqs. (18) and (19). Table 1 quantitatively investigates the precision of the depth measurement results. The depth maps of the measured board reconstructed from the fringe patterns may deviate from its nominal depth values, say 0, 16, and 32 mm. From this table, we see that the mean values of the deviations are very small, the RMS (root-mean-square) values of the deviations are of 0.1-mm level, and the maximum deviations are around 0.5 mm. The depth measurement errors should be smaller than these values, since these deviations are caused not only by the measurement errors but also by the flatness error of the board.
Regarding texture parallax correction, Fig. 11(a) shows an image of the measured board captured using the texture camera when the measured board is positioned at the depth of 16 mm, and Fig. 11(b) is the image of the same board    Fig. 11(a) from the view of the texture camera to that of the measuring camera; the result is shown in Fig. 11(c). We observe that the circular markers in Fig. 11(c) are exactly aligned with those in Fig. 11(b). This fact implies that the parallax of the texture image has been corrected successfully. Table 1 also lists the deviations of the circular marker centroids of the texture image after parallax correction from those of the measuring camera image. The mean and RMS deviations are smaller than 0.2 pixels, and the maximum deviations are not more than 0.5 pixels (corresponding to about 0.2 mm in lateral coordinates). These results demonstrate that the  texture parallax correction method can achieve a satisfactory accuracy.
Continuing the experiment, we use the same method to measure a plastic bottle, which has a freeform curved surface. Figure 12(a) shows one of the fringe patterns captured with the B/W camera. The wrapped phase map recovered from the fringe patterns is shown in Fig. 12(b). By subtracting the reference phases from it, Fig. 12(c) gives the phase difference distribution. The reconstructed 3-D shape of the object is illustrated in Fig. 13.
The last step of this experiment is to map texture on the surface of the 3-D shape just obtained. Figure 14(a) is the texture photograph taken by the color camera. Here the calibration board is kept behind the measured object, making it easy to observe the parallax between the views of the color and B/W cameras by comparing Figs. 14(a) and 12(a). Without correcting the parallax, directly mapping the captured texture image in Fig. 14(a) onto the 3-D object surface shown in Fig. 13 leads to a faulty result. As shown in Fig. 14(b), at the handle and lid positions of the measured bottle, the displacements of texture from its real positions, induced by the pixel misalignment, are evident. To solve this problem, we transform the texture image from the view of the color camera to the view of the B/W camera. Figure 14(c) shows the texture image with the parallax having been corrected. Using it for texture mapping, the result is illustrated in Fig. 14(d), in which the misalignment faults have been removed. Figure 14(e) shows the image difference obtained by subtracting Fig. 14(a) from Fig. 14(c). We directly observe from this image difference the parallax between the two images. Figure 14(f) is the result of mapping the image difference onto the reconstructed 3-D shape. These results demonstrate that the method introduced in this paper is valid in correcting the parallax of texture when the texture camera and measuring camera have different optical axes.
Regarding the measurement efficiency, data processing with this method involves only linear minimizations with a low computational complexity, but we still took about 30 min to calibrate the system and less than 2 min to measure an object in the experiments. The overwhelming majorities of these durations were consumed just for capturing fringe patterns, because we used the temporal phase unwrapping technique, 36 which requires capturing a great number of  fringe patterns, to measure an object with abrupt steps. Using a more efficient technique, such as the Gray-code combining with phase-shifting technique, 38 may significantly reduce the time needed for image capture. In addition, manually shifting the calibration board to different depths along the track also occupied a portion of time for system calibration. Using an automatically controlled track will be helpful for improving the efficiency and accuracy of the system calibration.

Conclusion
In this paper, we have analyzed the geometry of the fringe projection system with a color texture camera and suggested a system calibration method. This method can be used for determining the corresponding relationship between the 2-D texture and the 3-D data and the mapping relationship between the object depths and phase differences simultaneously, thus decreasing the duration time for system calibration.
In it, the data processing involves only linear minimizations and has a low computational complexity. The experimental results demonstrate that using the suggested method allows us to transform the texture image from the view of the color camera to that of the measuring camera, thereby correcting the parallax of the texture image and precisely mapping the texture on the reconstructed object surface.