Three-dimensional shape sensing plays an important role in machine vision, reversed engineering, automatic manufacturing, and other industrial applications. The use of a full-field technique, such as stereovision,1, 2 fringe projection,3, 4 and structured light illumination,5, 6 has been recognized as a promising method for the measurement of a surface profile.
One of the major studies for 3-D sensing is the measurement of dynamic objects. For objects with fast-moving speeds, 3-D shape measurements always require that the observed images not be blurred by motion. High-speed cameras or stroboscopic illuminations are commonly used to obtain unblurred images. Unfortunately, when the speed of the objects still exceeds the temporal resolution of the sensor, the image is blurred. Of course, an ultra-short laser pulse can still be used to freeze the motion on the image. However, the illumination intensity might not be sufficient to perform large-scale measurements, and the cost of such light sources is generally high.
In this paper, we show that the projected fringe profilometry4 does not need to avoid the blurred images. As a typical setup, we use a fringe pattern to illuminate the dynamic object and utilize a CCD camera to record the fringe distribution. Fringes on the obtained image are deformed by the topography of the object and also blurred by motion. Theoretical analysis shows that objects moving within one period of the projected fringe can be directly described by the projected fringe profilometry. Thus, the cost of the detection system is effectively reduced.
Figure 1 shows the system configuration. The plane is located in the figure plane, and the axis is normal to the figure plane. A fringe pattern is projected onto the inspected surface. Intensity of the fringes when propagating in space is represented asis the background or dc intensity level, is the fringe contrast, and and are periods of the fringes in and axis, respectively. The depth value on a surface point is measured relative to the plane indicated in the figure. Thus, the reflected intensity on the surface is expressed as is the reflectivity of the measured object.
The projected fringes on the surface are observed by the image sensor array. The detection plane coordinate system is defined in the CCD detection plane with and axes parallel to the row and the column directions of the sensor array, respectively. The gray level on the recorded image corresponding to is described asis the background or dc gray level, is the modulation amplitude, and is the measured absolute phase. For a telecentric system, the mapping transformation between the image plane and plane is is the magnitude of the telecentric lens. A phase value sampled at an object point is assumed equal to that sampled at its image point. This assumption applies when the point spread function of the system is symmetric (coma-free). Thus, Eq. 3 can then be rewritten as identifies the linear relationship between the reflected light intensity and the image gray level .
can be extracted with the phase-shifting technique or Fourier transform method. Since both phase evaluation techniques involve the arctangent operation, the extracted phases have discontinuities with phase jumps. Unwrapping is inevitable to recover the absolute phases.7 Once the unwrapped phase is obtained, depth on the surface point can be directly found from Eq. 5, as given by
Now, consider this inspected object moving with speed in the world coordinates. Its depth profile is a function of time and is given byis the object depth function at . and are unit vectors. ∇ is a 2-D gradient operator, which is denoted as .
The image sensor array obtains a blurred image within the exposure time . The gray level of the blurred image with reference to Eq. 5 can be expressed asis the measured phase from the blurred fringes, is the measured background or dc gray level, and is the modulation amplitude.
For objects in which varies slowly with and , Eq. 8 can be represented as7 into Eq. 9, the intensity of the blurred image is then simplified as , and .
According to Eq. 7, the depth profile at can be expressed as10 is represented as8 with Eq. 12, it is found that can be fully identified from the blurred fringes, as given by can be directly retrieved from the blurred fringe. If the exposure time is close to zero, Eq. 13 becomes close to Eq. 5, and the fringes are not blurred.
A ball with moving speed , , and was chosen as the dynamic sample. Its diameter was approximately . A sinusoidal fringe pattern was illuminated by a halogen lamp and then projected onto this dynamic sample. A CCD camera with at pixel resolution was used to record the fringe distribution. Fringes were blurred by linear motion. Figure 2 shows the fringe distribution, in which the exposure time was .
Phase-extraction was performed with the Fourier transform method.3 Figure 2 shows the computed phase , which was within the interval between and . Unwrapping was a necessary procedure to eliminate the discontinuities. In our experiment, we use Goldstein’s algorithm7 to restore the absolute phases. With Eq. 13, depth profile was determined. Figure 3 shows the retrieved profile. Its 1-D profile is shown in Fig. 3.
A comparison when the sample was static was performed as well. Appearance of the projected fringes on the static sample is shown as Fig. 4. Equation 6 was employed to retrieve the 3-D shape. Figures 4 and 4 show the retrieved 3-D shape and its 1-D profile, respectively. Systematic accuracy for a static object was approximately . The errors were mainly from the spatially sampling density of the CCD camera and phase-extraction. The sampling resolution was approximately , which was determined by the field of view and the pixel numbers of the CCD camera.
The difference between one two profiles (the dynamic case and the static case) is depicted in Fig. 5, in which the shifting displacement has been compensated. It was found that accuracy at the center area of the sample could be achieved with the same order of the static case, implying that our theoretical analysis was correct. However, enormous errors occurred at the edge of the dynamic object.
There were two sources causing such errors. They were (1) variation of effective exposure time at the boundary area, and (2) ambiguity of phase-extraction for surfaces with large depth variation. The exposure time for image pixels that inspected the boundary area was unfortunately not a constant. It was highly dependent on the moving direction and the shape of the boundary. For example, a surface point on the boundary is observed by the image sensor array. Since this object is dynamic, the observed point is moving from point to point on the detection plane during the exposure time . As shown in Fig. 6, for a sensor pixel that is located within the interval between and , its effective exposure time is . The effective exposure time on the boundary area was therefore not . Equation 9 was not applicable for the boundary area. The example shown in Fig. 6 also indicates that the effective exposure time was corresponding to the shape of the boundary on the detection plane. Effective exposure time for pixel in Fig. 6 is different from that for pixel in Fig. 6.
Errors from ambiguity of phase-extraction occur when the shifting amount of the projected fringes is larger than their periods. A displacement of the dynamic object directly causes the projected fringes to shift from one surface point to another. If the shifting amount is equal to the fringe period, the fringe contrast becomes zero. This phenomenon can be mathematically described when the sinc function in Eq. 10 is equal to zero, or say, is equal to . In such a situation, phase cannot be identified on that area. Moreover, aliasing occur when the shifting amount is larger than the fringe period, i.e., . This directly causes a phase offset when performing the phase unwrapping. Equation 10 for the aliasing area should be modified as13 is replaced as
Since aliasing occurs with7. Figures 7 and 7 are appearances of the inspected sample moving along the and axis, respectively. The moving speed was , and the exposure time of the CCD camera was . The phase extracted by the Fourier transform method is depicted as Fig. 8. Aliasing areas are enclosed with dotted lines. Ideally, the phase offset on the aliasing area could be compensated by adding or reducing a phase value. Unfortunately, the signal-to-noise ratio (SNR) was too low around the zero fringe-contrast area, and therefore phase information was lost. This directly made performing the phase extraction uncertain when using the Fourier transform method. The phase distribution became continuous on the boundary between the aliasing area and the unaliasing area. If the phase on the aliasing area was compensated by adding or reducing a phase value, discontinuity with a phase jump appeared on the zero fringe-contrast area. Thus, it is impractical to robotically recover phases on the aliasing area by simply adding or reducing a phase value.
Figure 9 shows the recorded blurred image when the sample was moving along the axis. The moving speed of the sample was , and the exposure time was . The retrieved phase is shown in Fig. 9. Since and were zero, the value was only a function of . Fringe contrast over the whole image was therefore varied only with , not with or . Phase extraction did not encounter any ambiguity. Sources of errors corresponding to various moving directions are reported in Table 1.
Sources of errors caused by the moving direction.
|Distributionof the zerofringecontrast|
|Area withphaseuncertainty||Aliasing area andarea with zerofringe contrast||Aliasing area andarea with zerofringe contrast||Area withzero fringecontrast||Aliasing area and area with zero fringecontrast|
|Area withenormousmeasurementerrors||Edge area, aliasingarea, and area withzero fringe contrast||Edge area, aliasingarea, and area withzero fringe contrast||Edge areaand areawith zerofringecontrast||Edge area, aliasing area, and area withzero fringe contrast|
Systematic accuracy for a dynamic object can be illustrated as Fig. 10, in which a plate moving along the axis was inspected. The roughness of this plate was approximately . The moving speed was , and the exposure time was . A comparison was evaluated when this plate was static, as shown in Fig. 10. The retrieved 3-D shape for the dynamic case and the static case is depicted as Figs. 11 and 11, respectively. Even though the fringe contrast on the dynamic object was relatively low, its profile can be retrieved with accuracy as high as the static one.
Compared with other methods using deblurred algorithms to restore the observed information, the proposed method saves the computation time. For approaches using a high-speed camera or stroboscopic illuminations to freeze the object’s motion, the cost of the proposed system is relatively low. However, limitations are that the inspected object should be a rigid body, and this object should move linearly within one period of the projected fringes. If the displacement of the projected fringes shifts larger than one period of the fringes, aliasing will occur. In addition, errors also occur when the inspected object is rotating. Equation 9 is not applicable when the moving vector is time-dependent.
We have presented a discussion on how to retrieve the 3-D shape from an image blurred by motion. With the fringe projection method, objects moving within one period of the projected fringes can be fully described. Thus, it is not necessary to avoid blurred images. Accuracy can be achieved that is as high as with the static image. This makes it desirable to reduce the cost of the detection system. We believe that applications to microelectromechanical systems (MEMS) and biomedical inspections can be realized.
Wei-Hung Su is an assistant professor in the Department of Material Science and Optoelectronic Engineering at the National Sun Yat-Sen University, Taiwan. He earned a PhD degree and an MS degree in electrical engineering from Pennsylvania State University in 2002 and 1999, respectively. His professional interests are optical metrology, digital image processing, and optical information processing.
Chao-Kuei Lee received his PhD in electro-optical engineering from National Chiao Tung University, Taiwan, in 2003. He is currently an assistant professor who directs the Laboratory of Femtosecond and Quantum Modulation with the Institute of Electro-Optical Engineering in National Sun Yat-sen University. His research interests include femotosecond light sources, ultrafat optoelectronics, and coherent quantum control.