1.

Introduction

Fingerprinting is a classical topic in biometric and computer vision domains and has gained wide application in our daily life. With any fingerprinting technique, how to capture the high-quality fingerprint image is always the principle concern. Current fingerprint acquisition instruments usually consist of an image sensor and a touch panel. With such a touchable operation manner, the captured fingerprint images are usually degraded by improper finger placement, skin deformation or slippage, smearing of fingers, sensor noise, etc.¹ To overcome these disadvantages, a technique named touch-less fingerprinting is emerging recently.² By employing a three-dimensional (3-D) scanning procedure, 3-D models of fingerprints can be obtained with a contactless operation. In comparison with traditional two-dimensional (2-D) fingerprint images, plentiful fingerprint information can be retrieved from the 3-D fingerprint models, thus making the subsequent fingerprint recognition more reliable.³

Compared with the traditional 2-D image-based fingerprinting techniques, 3-D fingerprinting is still a new research domain that appears in recent years. The major technical challenge involved in 3-D fingerprinting is how to capture the 3-D model of live fingerprint precisely and efficiently. Various computer vision techniques have been applied for this purpose, such as shape from silhouette, structured light systems (SLS), stereo vision, etc.^4–11 In Ref. 6, a multiple-view system that consists of five cameras and a set of 16 green light-emitting diode (LED) arrays is proposed for the 3-D fingerprint scanning. Multiple cameras are used to capture the fingerprint images under different viewpoints and LED lightings. And the corresponding silhouettes are extracted for the 3-D modeling of the fingerprints via shape from silhouette method. Based on the 3-D fingerprint acquisition system proposed in Ref. 6, to make the 3-D fingerprint images compatible with current 2-D fingerprint systems, an unwrapping algorithm is presented in Ref. 7. The equidistance unwrapping approach is utilized to minimize the distortion while preserving the ground-truth of the fingerprint. By unfolding the 3-D fingerprint in such a way, it resembles the effect of virtually rolling the 3-D finger on a 2-D plane. In Ref. 8, an SLS is developed for the 3-D fingerprint acquisition. While the fringe projector operates at a blue wavelength, a green illumination is used to image the papillary lines. The fringe pattern analysis technique is applied for the 3-D depth recovery from phase information.⁹ The nonparametric unwrap approach is applied to preserve the distances between surface points. A 3-D fingertip scanning system that consists of one projector and two industrial cameras is presented in Ref. 10. 3-D fingerprint models are achieved via SLS and stereo-vision means, and the 3D models are unwrapped to compare with 2-D counterparts. In Ref. 11, a high-speed SLS is established for the 3-D scanning of fingerprints. The system is configured with a digital light processing projector and a camera. A shifting sine wave pattern is projected onto the finger surface and the images with pattern illuminations are captured synchronously. Via the proposed decoding algorithm, high-density 3-D points of the finger surface can be obtained. However, such 3-D scanning-based approaches usually suffer from the translucent finger skin and thus degrade the 3-D reconstruction accuracy. Moreover, the complicated structure and high cost of the hardware also make these techniques unaffordable to end users and prevent the touchless 3-D fingerprinting technology for wide applications.

In this paper, an efficient 3-D fingerprint acquisition method based on a simple hardware setup is presented. The system only consists of one camera and some LED lights. A shiny ball with known radius is used to calibrate the lighting directions of different LEDs. Considering nonuniform lighting conditions by the LEDs, a correction procedure is introduced to calibrate each LED light. Compared with traditional photometric stereo (PS) methods, which are usually implemented with Lambert reflectance assumption, a two-layer Hanrahan–Krueger (HK) model¹² is introduced to model the finger surface’s reflectance property more accurately. An objective function is derived from HK irradiance equation based on PS principle, and the Levenberg–Marquardt method¹³ is used to solve the equation for accurate estimation of surface normal. Finally, a linear surface normal transformation is adopted to enhance the reconstructed 3-D fingerprint model. Experiments on real fingerprints and palm prints are experimented to demonstrate its feasibility and 3-D reconstruction accuracy.

The rest of the paper is organized as follows. Reflectance modeling algorithm of the finger surface and how to solve the surface normal from the model are introduced in Sec. 2. Section 3 describes the adopted methods for lighting direction calibration, nonuniform lighting correction, and 3-D fingerprint model enhancement. Section 4 presents the experimental results on real fingerprints and the comparisons with traditional method. Conclusions and future work are offered in Sec. 5.

2.

Reflectance Property Modeling of Finger Surface

PS is an important approach in computer vision that is usually used to estimate the surface normal by observing the target surface under various illuminations. This technique was first introduced by Woodham.¹⁴ Various PS-based methods have been proposed in the past decades, and their major concerns are usually focused on the following issues: calibration of the lighting directions, modeling of the surface reflectance property, 3-D reconstruction under nonuniform lighting conditions, etc.^15–20 The modeling of surface reflectance property is a crucial issue in the PS algorithms, since it directly determines how the incident lights be modulated by the target surface as well as the final surface normal estimation. The most popular surface reflectance description is the Lambert model, which assumes the target surfaces with ideal diffuse reflection property. Given three or more known illumination conditions, surface normal can be efficiently calculated by solving some linear irradiance equations. ²¹

Skin surface like fingerprint is a kind of translucent material that contains certain multiple scattering and specular reflections. It cannot be well represented by a traditional linear reflection model like Lambert, especially in the case where high reconstructing precision is demanded. To model the human skin more precisely, a nonlinear reflection descriptor named Torrance and Sparrow (TS) model is introduced in Ref. 22. As a physical-based model, the TS model assumes the skin reflectance consists of Lambertian and purely surface scattering components. Incorporating an uncalibrated PS method, reflectance parameters of skin surface can be well estimated and the negative effects from generalized bas-relief ambiguity can be reduced.²³ In comparison, the HK model considers the skin as a layered surface based on one-dimensional linear transport theory.¹² The underlying principle is that the amount of light reflected by a material that exhibits subsurface scattering can be calculated by summing the amount of lights reflected by each layer times the percentage of light that actually reaches that layer. Hence, it is a more reasonable description for translucent surfaces like human skin and fingers.

With reference to the HK model, in this paper, the finger surface is modeled as a two-layer material that consists of the epidermis and the dermis. The layers have different reflectance parameters that determine how the incident lights are reflected as illustrated by Fig. 1.

Fig. 1

The finger skin is represented by a two-layer Hanrahan–Krueger model. The incident light $L_i$ is reflected twice by the epidermis and dermis layers; the reflected lights $L_r$ are captured by the camera.

According to Ref. 12, the irradiance equation of a two-layer HK model can be written as

Define surface normal vector as $\mathbf{n}=(n_x,n_y,n_z)$, light direction vector as $\mathbf{l}=(l_x,l_y,l_z)$, reflection light direction as $\mathbf{r}=(r_x,r_y,r_z)$, and view direction as $\mathbf{z}=(z_x,z_y,z_z)$. All these vectors are normalized. Then, $\cos {\theta }_i$, $\cos \varphi$, and $\cos {\theta }_r$ in Eq. (1) can be rewritten in the form of inner product between two of these vectors as

Suppose all illumination conditions are known, and $g$, $T^{12}$ and $T^{21}$ are constant and known over the whole surface. Then there are total seven unknown parameters involved in Eq. (1), i.e., $n_x$, $n_y$, $n_z$, $\rho$, $d$, ${\sigma }_s$, and ${\sigma }_a$. Define $\mathbf{x}$ for the unknowns as $\mathbf{x}=(n_x^j,n_y^j,n_z^j,{\rho }^j,d^j,{\sigma }_s^j,{\sigma }_a^j)$, where $j$ indicates the $j$’th surface point. Given $K$ images with each one taken under different illumination conditions, the objective function can be formulated as

3.

System Calibration

3.1.

Calibration of Lighting Directions

In Ref. 24, two cameras and a shiny ball with unknown position and radius are used for the lighting direction calibration. Since only one camera is adopted in our system, a shiny ball with known radius $r$ is used instead. With Zhang’s camera calibration method,²⁵ focal length $f$ and the camera center $C$ can be estimated. With reference to the camera coordinate frame, lighting direction of the $i$’th light source can be represented as ${\mathbf{l}}_i=(l{x}_i,l{y}_i,l{z}_i)$ as shown in Fig. 2(a).

Fig. 2

(a) Illustration of lighting direction calibration. (b) Relation of normal vector $N$, reflection direction $R$, and the incident light direction $l$ on the specular point $S_i$.

According to the following two observations in the perspective image of a sphere—(1) the line passing through the camera center and any boundary point of the sphere in the image plane is a tangent line to the sphere and (2) the perpendicular distance from the center of the sphere to a tangent line of the sphere is the radius of the sphere—we can get the following equation:

Eq. (5)

$$\{\begin{array}{l}{d}_i^2={|CS|}^2-{\cos }^2\theta =h_i(s_x,s_y,s_z)\\ \cos \theta =\frac{(CS·C{B}_i)}{|CS||C{B}_i|}=\frac{\sqrt{{|CS|}^2-d_i^2}}{|CS|}\end{array},$$

where $B_i$ is a boundary point on the image plane with a checked position $(i_i,j_i,f)$, $|CS|$ is the distance between point $C$ and point $S$, $CS$ is the vector that connects the two points. Note that Eq. (5) contains only three unknowns, $S=(s_x,s_y,s_z)$, i.e., three components of the sphere center. Suppose $m$ boundary points $B_i$, $i\in (1,\dots ,m)$ can be obtained via edge detection algorithm,²⁶ then an error function can be defined as

Eq. (6)

$$\mathrm{EOF}=\sum _{i=1}^m{(h_i-r^2)}^2.$$

By minimizing this error function EOF, we have three equations about $S$, and the optimal values of the sphere location can be solved. The specular point $P(i_1,j_1)$ on the image plane can be detected by finding the brightest image point. Then we can get its corresponding surface point $S_1$ as well as surface normal vector $\mathbf{N}$ as illustrated by Fig. 2(b). With $\mathbf{N}$ and $\mathbf{R}$, the lighting direction $\mathbf{l}$ can be calculated as

Eq. (7)

$$\mathbf{l}=2(\mathbf{N}·\mathbf{R})\mathbf{N}-\mathbf{R}.$$

3.2.

Correction for Nonuniform Lighting Conditions

Notice that the distribution of the illumination (DOI) for each LED lamp is usually different and nonuniform. Figure 3(a) shows a homogeneous white paper surface under the illumination of one LED. Figure 3(b) shows the distribution of the image intensities within a selected image area. The nonuniform distribution of LED lights causes direct effect to the surface normal estimation. Therefore, a procedure is required to correct these deviations. Notice that when the light position is fixed, the brightest spot has a fixed area. That means under the illumination of a fixed light source, for a point on the image plane $(i,j)$, the portion $p(i,j)$ of its pixel value $I(i,j)$ to the brightest pixel value $I_{\max }$ is constant to the pixel position and independent of the surface shape, i.e.,

Eq. (8)

$$p(i,j)=\frac{I(i,j)}{I_{\max }}.$$

Fig. 3

(a) A white paper under the illumination of one light-emitting diode (LED) light; the red circle is the brightest spot, which indicates the illumination distribution not uniform. (b) The diagram of values of $p$ calculated from (a).

Suppose $l^k$ is the $k$’th LED light whose DOI needs to be calibrated; a white planar board with Lambertian surface is used for its correction. The procedure can be implemented by the following steps:

• a. Take an image of the planar board under the illumination of $l^k$, and define it as $I^k(i,j)$, $i\in \text{image}\text{Height}$, $j\in \text{image Width}$;

• b. Search the biggest values in $I^k$, i.e., $I_{\max }^k=\underset{i,j}{\max }\{I^k\}$;

• c. Substitute $I_{\max }^k$ into Eq. (8) to calculate $p^k(i,j)$ for all pixels.

With the above correction, image intensity to each light can be generally corrected to a uniform distribution. Figure 4 gives a comparison between original and corrected fingerprint images.

Fig. 4

(a) Original fingerprint image. (b) Intensity of the corrected image looks more homogenous.

4.

Experiments and Discussion

The experimental setup consists of one camera (Prosilica GC650, 90FPS) with a resolution of $659\times 493\text{pixels}$ and seven LED lamps (white color, 0.2 W, emitting angle 120 deg) as shown in Fig. 5(a). Focal length of the lens (Navitar NMV-25M23) is 25 mm, and the operating distance (from the finger place to the front of lens) is $\sim 90\mathrm{mm}$. The LED lamps are evenly mounted on a lampshade with radius of 75 mm. The lampshade has a distance $\sim 65\mathrm{mm}$ to the finger place. The LED positions are finely adjusted to point to the finger place region. An external I/O board is developed to synchronize the camera and LEDs to control the whole capturing time within 0.1 s. The algorithm is implemented under MATLAB2010a on a PC with 2.7 GHz CPU and 2 G RAM.

Fig. 5

(a) The system consists of one camera and seven LEDs. (b) Plot of the calibrated lighting directions of seven LEDs, where the camera optical center is origin and its principal optical axis is $Z$ direction.

Via the proposed lighting direction calibration method, directions of LED lamps are calculated as the following normalized vectors: $l_1\{0.4571,0.1702,-0.8729\}$, $l_2\{0.7714,-0.1672,-0.6139\}$, $l_3\{0.6645,-0.6205,-0.4163\}$, $l_4\{0.1975,-0.9110,-0.3618\}$, $l_5\{-0.3447,-0.8340,-0.4307\}$, $l_6\{-0.6193,-0.4498,-0.6434\}$, and $l_7\{-0.4701,0.1366,-0.8719\}$, as illustrated in Fig. 5(b).

Real fingerprints and palm prints are used for the experiments, and the results are compared with traditional PS methods where the Lambert reflectance model is adopted. The experiments are implemented with the following steps:

4.1.

Comparison with Traditional PS Method

In our experiment, the parameters of $g$, $T^{12}$ and $T^{21}$ in Eq. (1) are set to {0.8, 1.25, 0.8}. The unknown parameters of $d^j$, ${\sigma }_s^j$ and ${\sigma }_a^j$ in Eq. (1) are initialized to $\{0.085\mathrm{mm},50{\mathrm{mm}}^{-1},3.8{\mathrm{mm}}^{-1}\}$ for the fingerprint experiments and $\{0.12\mathrm{mm},30{\mathrm{mm}}^{-1},4.5{\mathrm{mm}}^{-1}\}$ for the palm print experiment. To enhance the recovered 3-D model, $w=5$, $k=1.5$ is set to Eq. (9).

There are still technical challenges for current instruments to get the 3-D ground truth of elastic surface like skin. That means it is difficult to evaluate the precision of reconstructed 3-D fingerprint models directly. As an alternative approach, a homogeneous white paper surface is used for experiment as shown by Fig. 6. Surface normal at each image pixel is calculated, and a standard deviation of only 1.65 deg is obtained. It shows high reconstruction accuracy can be guaranteed by the proposed calibration and modeling methods, except for some distinct errors that appeared in the marginal image regions affected by inhomogeneous lighting conditions. Similar accuracy evaluations are also performed on some synthetic data with known depth map as shown by Figs. 7 and 8. The data can be downloaded from Ref. 27. Seven synthetic images with virtual illuminations are generated as shown by Figs. 7(a) and 8(a). The reconstructed 3-D models under various viewpoints are displayed by Figs. 7(b) to 7(d) and 8(b) to 8(d). The reconstruction errors (in the unit of pixel) with respect to the ground truth models are presented by Figs. 7(e) and 8(e). It shows an average reconstruction deviation far less than one pixel and demonstrates the accuracy of the proposed algorithm.

Fig. 6

Accuracy evaluation of three-dimensional (3-D) reconstruction results. (a) A flat paper surface is used for the experiment. (b) The reconstructed planar surface. (c) 3-D plane under various viewpoints. (d) Distribution of the surface normal deviations.

Fig. 7

Accuracy evaluation with synthetic data: saddle-shape. (a) Seven saddle images with virtual illuminations. (b) to (d) The reconstructed 3-D model under various viewpoints. (e) Distribution of the reconstruction error in pixel unit.

Fig. 8

Accuracy evaluation with synthetic data: cone. (a) Seven cone images with virtual illuminations. (b) to (d) The reconstructed 3-D model under various viewpoints. (e) Distribution of the reconstruction error in pixel unit.

In the experiment with fingerprints, the original four images with different lighting conditions are as shown in Fig. 9(a). The reconstructed 3-D model by traditional PS method, which adopts Lambert reflectance model, is as shown in Fig. 9(b). The reconstructed 3-D model by the proposed method is given by Fig. 9(c). Figure 9(d) shows the obtained albedo map by the proposed method. Another experiment is also conducted on the palm print with same parameter setting as shown by Fig. 10. From the results we can see that skin features like the finger and palm ridges can be clearly retrieved in 3-D space. To solve the nonlinear model for each image pixel, our algorithm takes more time than the traditional linear Lambert model. The whole computation time from images input to final 3-D model is given in Table 1. The efficiency can be further improved by more efficient developing tools like Visual C++ and parallel processing devices like graphics processing units in the future.

Fig. 9

Experimental results on fingerprint. (a) Four images under illumination direction of $\{-0.41,0.45,0.80\}$, $\{-0.39,0.47,-0.80\}$, $\{0.12,0.58,-0.81\}$, and $\{0.52,0.24,-0.82\}$. (b) Reconstruction result via traditional photometric stereo (PS) method based on Lambert reflectance model. (c) Reconstruction result by the proposed method. (d) The albedo map obtained by the proposed method.

Fig. 10

3-D reconstruction of a palm print. (a) Four images of palm print under illumination directions of $\{0.77,-0.17,-0.61\}$, $\{0.20,-0.91,-0.36\}$, $\{-0.34,-0.83,-0.43\}$, and $\{-0.47,0.14,-0.87\}$. (b) The albedo map obtained by the proposed method. (c) 3-D reconstruction result via traditional PS method based on Lambert reflectance model with $w=5$, $k=1.5$. (d) 3-D reconstruction result by the proposed method with $w=5$, $k=1.5$. (e) to (f) The cropped 3-D images for close observation of (c) and (d).

Table 1

Calculation time by the nonlinear model and linear Lambert model (in seconds).

Dataset	Image size	Image number	Our method	Lambert model
Fingerprint	$300\times 200$	7	183.57	23.21
Palm print	$659\times 493$	7	726.66	45.83

4.2.

Enhancement of 3-D Models Under Different Parameters

The parameters of $w$ and $k$ in Eq. (9) play an important role in the rendering of 3-D fingerprint model. In this experiment, different $w$ and $k$ are tested as shown in Fig. 11. Figure 11(a) shows the original fingerprint image. The reconstructed 3-D fingerprint without surface normal enhancement is as shown in Fig. 11(b). Figures 11(c) to 11(e) show the 3-D models with surface normal enhancing parameters of $w=5$, $k=1.5$, $w=7$, $k=2.0$ and $w=11$, $k=4.0$, respectively. From the results we can see that with the increase of enhancing parameters, the fingerprint ridges can be clearly emphasized. However, the ridges are also usually broken subject to the overenhancement as shown in Fig. 11(f).

Fig. 11

3-D fingerprint models under different surface enhancing parameters $w$ and $k$. (a) Original fingerprint image under one LED illumination. (b) 3-D fingerprint model without surface normal transformation. (c) Result with parameters of $w=5$, $k=1.5$. (d) Result with parameters of $w=7$, $k=2.0$. (e) Result with parameters of $w=11$, $k=4.0$. (f) Cropped area to show overenhancement.

4.3.

Reconstruction Results with Different Image Numbers

In this experiment, three, five, and seven images are used for the fingerprint 3-D reconstruction, and the results by the proposed method and traditional Lambert method are also compared respectively. The captured seven fingerprint images under different LED illuminations are as shown in Fig. 12. To make the comparison unbiased, all the 3-D models are presented without enhancement processing. Figure 13 shows the 3-D reconstruction result with seven images. Figures 13(a) and 13(b) show the results by our method and traditional Lambert model, respectively. Figures 13(c) and 13(d) give close-up views. In comparison, without the enhancement process, Fig. 13(c) demonstrates more finger ridge details with sharp contrast than Fig. 13(d). It means higher-quality 3-D model can be achieved with the proposed method. Experimental results with five and three images are given in Figs. 14 and 15, respectively. From these results, we can see that once the number of images declined to three, as illustrated in Fig. 15, there is no big difference between our method and the traditional method. It is because that there are seven unknowns in Eq. (1) and it needs more images to achieve a stable solution.

Fig. 12

Seven fingerprint images captured under different LED illuminations.

Fig. 13

3-D reconstruction results of finger print with seven images. (a) Result by the proposed method. (b) Result by the traditional Lambert model. (c) and (d) The cropped images for close observation.

Fig. 14

3-D reconstruction results of finger print with five images. (a) Result by the proposed method. (b) Result by the traditional Lambert model. (c) and (d) The cropped images for close observation.

Fig. 15

3-D reconstruction results of finger print with three images. (a) Result by the proposed method. (b) Result by the traditional Lambert model. (c) and (d) The cropped images for close observation.

5.

Conclusions and Future Work

This article presents a novel method for 3-D fingerprint acquisition based on the principle of PS. Compared with previous 3-D scanning-based methods, the proposed system is rather simple and only contains a camera and some LED lamps. To calibrate the lighting direction, a shiny sphere with known radius is used. To model the reflectance property of finger surface, a two-layer HK reflection model is proposed instead of traditional Lambert reflectance model. Considering the nonuniform lighting property of LEDs, a simple correction procedure is also introduced. Finally, the reconstructed 3-D model is enhanced via a linear transformation process. To verify feasibility of the proposed algorithm and system, the experiments are implemented with real finger and palm prints. The results by traditional Lambert model are also provided for comparison to demonstrate the improvements in 3-D reconstruction accuracy.

Future work can address how to improve the correction method for nonuniform lighting conditions, especially to the marginal image regions. Moreover, how to apply the obtained 3-D fingerprint models to the recognition phase and to establish an extensive 3-D fingerprint database will be an urgent task in the future.