1.

Introduction

Human skin gradually changes as time flies by, it is very fresh and elastic in the young days. The tension of skin is lost as one grows older. Use of cosmetics that is suitable for the human skin surface condition of a respective human is effective to prevent skin aging. Recently, there are many choices of cosmetic products. It is very desirable to develop an automatic evaluation system for the human skin surface condition to show the effect of cosmetics.

The human skin surface has the pattern called grid texture. This pattern is composed of the valleys that spread vertically, horizontally, and obliquely and the hills are separated by valleys. Changes of the grid are closely linked to the condition of the human skin surface. They can serve as a good indicator for the skin condition. By measuring the skin grid using digital image processing technologies, we can evaluate the human skin surface about its aging, its health, and its alimentary status.

Besides the grid, there are also a lot of pits and bulges textured among the lines and some impurities like the sebum scattered on the skin surface. This complexity of dermal structure makes it difficult to extract the centerline. What is more, some external disadvantages caused in the acquisition of the skin image by clinical photographing, such as illuminating conditions, noise level, spherical effect, etc., make it more difficult. To our knowledge, no published work has done well in the extraction.

In this paper, the centerlines is detected first and a subsequent postprocessing is performed to form connected, closed regions. Through measuring and analyzing some parameters of the grid. the condition of the skin can be monitored. The paper is arranged as follows. In Sec. 2, we start with some characters of human skin. and on this basis, feature space transformation is described. In Sec. 3, an edge detection scheme is introduced. It is based on some features of maximal surface principal curvature. The postprocessing steps used to form closed regions are discussed in Sec. 4. In Sec. 5, an experimental result is presented, followed by the discussion about the efficiency and the advantages of the method. In Sec. 6, main conclusions are drawn at the end the paper.

2.

Feature Space Transformation

This operation is to map the original image from grayscale space to energy space. Figure 1(a) shows one original image of human skin, Fig. 1(b) illustrates the grayscale distribution curve of a typical row in Fig. 1(a), with the x coordinate denoting the pixels in a row, and the y coordinate denoting the corresponding gray values. Note from Fig. 1(b) that the skin grids are of triangle shape, like the shape of the roof-edge. This makes it impossible to use any traditional step-edge detection methods to extract the roof-edged grids in human skin. From these two images, we can also observe that the values are not of a uniform type. They include convex ridges [shown in Fig. 2(a)] concave valleys [shown in Fig. 2(b)], and saddle shape [shown in Fig. 2(c)]. It is the effect of human vision to contrastness that combines these mussy valleys and ridges into a network. This feature is typical in the clinical images, which is in fact more like a texture image than a grid one. Nevertheless, whatever they might be, they do form a grid in relatively constant width and they do own the same feature—the graylevels of grid pixels change more abruptly than that of nongrid pixels, and the curvatures at the grid pixels are greater than that in nongrid pixels. Therefore, a scheme is needed to trim the original images before extraction.

Figure 1

The grayscale distribution curve (b) corresponding to the pixels in the 40th row of the skin image (a).

Figure 2

The one-dimensional roof-edge signals occur in the human skin image: (a) ridge edge, (b) valley edge, (c) saddle edge.

First of all, we define pixel energy as follows:

It is a linear compounding of two components, The first term denotes kinetic energy, and the second term potential energy. The kinetic energy E _kinetic reflects local gradient information. That is, E _kinetic is directly proportional to the grayscale change in a certain domain of the pixel. The more rapidly the graylevel changes, the greater the value of E _kinetic is.

The potential energy E _potential indicates the global spatial distribution of the gradient. It is directly proportional to the grayscale variation of the pixels in the whole image.

In Eq. (1), the coefficients α and β give, respectively, the strength of kinetic and potential forces. The choice of those two coefficients is very important. A tradeoff is needed to get a rather brilliant result. Usually, we choose parameter α to be proportional to the kinetic energy E _kinetic , this is to guarantee a good ridge in energy space. In the meantime, we select parameter β to be proportional to the curvature at this point to avoid the influence caused by illumination fluctuations on transformation result, because the potential energy E _potential is salient at the places where there is a big illumination fluctuation, and the import of curvature to the transformation can be used to correct this error.

The kinetic energy E _kinetic and the potential energy E _potential are defined, respectively, by the following forms.

Let I(s)=f(x,y), s(x,y)∈D be one curved surface of image grayscale defined on the domain D, I¯ be the average grayscale of the whole surface, for each point s(x,y)∈D, its energy components can be defined as follows:

According to the definitions earlier, the feature image E _pixel (x,y) corresponding to the original image f(x,y) can be calculated by

When digitized, Eqs. (2) and (3) yield, respectively, the following forms:

It denotes the maximal magnitude of the grayscale variation of the pixels in the window δ:

3.

Edge Detection

After feature space transformation, the original image is transformed into an energy image f [Fig. 3(a)], and the grid in original image is turned into ridges with textures among it. To avoid overconnection, a filtering is mandatory to smooth the inside texture while preserving the ridges. Here, we use a morphological filter based on reconstructions. The amount of filtering is an important parameter which determines the experiment results. The amount of filtering can be manually tuned to extract different lines. A severe filtering may produce a net of deep lines, and a slight filtering can produce a net of shallow lines, for more information about this filter, refer to Refs. 1 2 3 4 5 6.

Figure 3

A sequence of extraction (a) Energy image f of original image [Fig. 1(a)]. After feature space transformation and erosion-reconstruction morphological filtering. The net of ridge appears. (b) Edge detection using the normal curvature method.

The filtered image f can be considered as a gray-level surface S. The points located on the ridge centerline have a maximum normal curvature on the perpendicular direction of the ridge and a minimum normal curvature along the ridge, so the centerline can be identified by analyzing the normal curvatures of the surface S.

Now we put the surface S into a fixed coordinate system O_xyz, this surface has continuous third order partial derivatives. The function of the surface is: r=r(u,v). Let P be one point on the surface S, that is, P∈S. ∂r/∂u,∂r/∂v and n are, respectively, tangent vectors and unit normal vector of S at point P. From the theorem of the Weingarten map, we know that, at point P, there exist two orthonormal eigenvectors of the Weingarten map {e ₁,e ₂} called principal directions at P, and two eigenvalues corresponding to these directions called the principal curvatures which is the extremum of the normal curvature at P, denoted by k₁ (the maximum normal curvature) and k₂ (the minimum normal curvature).

Point P falls on ridge centerline, if it satisfies the following conditions, k₁≠0, and e ₁ ⋅z=0, that is, the principal direction e ₁ corresponding to the maximum normal curvature k₁ is always perpendicular to the Z axis. So we can detect the ridge centerline by finding k₁ and e ₁.

From the theories of differential geometry, we can individually define the two fundamental forms of surface S at P as

the first fundamental form:

the second fundamental form:

Tangent vectors ∂r/∂u,∂r/∂v span a plane [named tangent plan of S at P, denoted by T_p(S)] which comprises all tangent vectors of S at the point P. For each of the tangent directions at point P, there exists a curvature named normal curvature which represents the curvature of S in this direction. The directions to which the normal curvature take an extremum are named the principal directions, and the values of these extrema are named the principal curvatures. They are the roots of the following equations:

From Eq. (11), we can get the two principal directions corresponding respectively to the principal curvatures λ_1,2=k_1,2:

After the principal curvature k₁ and the corresponding principal direction e ₁ are calculated, if it satisfies k₁≠0 and e ₁ ⋅z=0, we think it is on the centerline. Figure 3(b) shows the centerlines superimposed on the original image.

4.

Postprocessing

Figure 3(b) is the result of edge detection. To show the effect more clearly, the background is eliminated in Fig. 4(a). Many fragmentary lines appear in this image, and there exist many discontinuous centerlines. Postprocessing is needed to remove those fragmentary lines and connect discontinuous lines into a closed grid. The cleaning process can be thought of as a tracking process. In the tracking process, the length of the lines are recorded, so that we can remove those lines whose length is shorter than a threshold.

Figure 4

The effect of postprocessing: (a) before and (b) after.

Finally, disconnected centerlines are connected to form closed contours. The basic strategy for connecting is summarized as follows.

The result of this postprocessing is shown in Fig. 4(b). A connected, closed grid of skin is obtained. This grid can be used to measure skin parameters.

5.

Experimental Results

To illustrate the validity of the method proposed in this paper, we use, respectively, traditional step-edge detectors and ridge detectors to extract central lines of the human skin grid, and make a comparison with the result of the proposed method. The example image is taken by a skin microscope from human arms by magnifying 30 times of the real skin, which is digitized to 256 gray-levels, 320×240 pixels. The comparison indicates that this method is more suitable for the skin analysis purpose.

There are a lot of methods on step-edge detection. Most of them are based on the idea that the grayscale gradient on the edge is more greater than that anywhere else. The Sobel operator is a representative one. Figure 5(a) shows the result of the Sobel operator performed on the human skin image. Note that this method can only get some discontinuous lines of the grid. These lines are not the centerlines of human creases, and cannot be used to characterize the human skin mathematically.

Figure 5

The results obtained, respectively, by using (a) Sobel operator, (b) zerocrossing method, (c) watershed transformation, (d) drainage pattern, and (e) the method proposed in this paper.

Ridge detection is of high interest in image analysis. Researchers have tried to describe it in various ways. Haralick⁷ and Koenderink⁸ identified ridges as loci of extremal height in two-dimensions, and is extended to d-D in Ref. 9 as the height condition. Gauch¹⁰ and Thirion¹¹ identified ridges as positive maxima of the curvature of the relief’s level curves. The height and vertex of ridges are of local characterizations. In implementation, ridges can be obtained by looking for sign changes in zerocrossing functions, and labeling pixels which have a change of sign, afterwards, a thinning algorithm is needed to get 1-pixel wide line. Figure 5(b) shows the effect of the zerocrossing method. Notice that the result is unsatisfactory, it yields many spurious branches, and the obtained centerline has many discontinuities.

More recent works^{12 13 14} have brought global characterizations of ridges. These works think of ridges as a separatrix of objects. Watershed in Refs. 2 and 15 absorbed more attention. Figure 5(c) shows the performance of the Watershed transformation method. Although this method can yield closed, continuous regions, the result is not exciting yet. Arbitrary lines appear, and some ridges disappear. When used in skin monitoring they will cause a measure error.

There are also many other approaches on ridge detection, for example, Refs. 16 17 18 extract drainage patterns by simulating the flow of water over the Earth’s surface. Used in human skin segmentation, this method cannot divide the skin into closed regions, and yields many fine branches, Figure 5(d) shows the result. It is also unsatisfactory in measuring skin parameters.

In the experiment using the method proposed in this paper, we choose the parameters as follows: N=5 and a constant α=1, β=k₁. Another parameter required in filtering is scale h which denotes the texture magnitude being filtered away. Different values of h will cause a different result. Figure 5(e) shows the result when h=7, and after postprocessing such as cleaning, connection to form some closed regions. We notice that this method can divide the skin surface into closed regions, and the centerlines are continuous, very little overdetection and underdetection appear, but some centerlines did not run through the center of the creases. This needs improving in the following work. Compared with other step-edge and ridge-edge approaches, this approach is effective in extracting human grid centerlines.

6.

Conclusions

In this paper, we have proposed a method for preprocessing the sophisticated human skin image. The characteristics of the original image have been described, and on the basis of these features, Feature space transformation is presented. The aim of the feature space transformation is to map the disordered grids into a neat net of ridges which will facilitate the subsequent operations. The morphological filter can remove the texture and spurious minima of the image to avoid overconnection. the amount of filtering is manually tuned for each application. Normal curvatures are calculated, and the centerlines are detected by analyzing the normal curvatures.

Some advantages and drawbacks have been reached through analyzing the experimental results. The method can divide the skin into continuous, closed regions, and the lines have a good localization, but sometimes it will yield arbitrary lines and lose genuine lines. Nevertheless, compared with other methods, this one is much suitable for the purpose of human skin monitoring.