Development of a three-dimensional surface imaging system for melanocytic skin lesion evaluation

Abstract. Even though surface morphology is always taken into account when assessing clinically pigmented skin lesions, it is not captured by most modern imaging systems using digital imaging. Our aim is to develop a novel three-dimensional (3D) imaging technique to record detailed information of the surface anatomy of melanocytic lesions that will enable improved classification through digital imaging. The apparatus consists of three high-resolution cameras, a light source, and accompanying software. Volume measurements of specific phantoms using volumetric tubes render slightly lower values than those obtained by our 3D imaging system (mean%±SD, 3.8%±0.98, P<0.05). To examine the reproducibility of the method, sequential imaging of melanocytic lesions is carried out. The mean%±SD differences of area, major axis length, volume, and maximum height are 2.1%±1.1, 0.9%±0.8, 3.8%±2.9, and 2.5%±3.5, respectively. Thirty melanocytic lesions are assessed, including common and dysplastic nevi and melanomas. There is a significant difference between nevi and melanomas in terms of variance in height and boundary asymmetry (P<0.001). Moreover, dysplastic nevi have significantly higher variances in pigment density values than common nevi (P<0.001). Preliminary data suggest that our instrument has great potential in the evaluation of the melanocytic lesions. However, these findings should be confirmed in larger-scale studies.


Introduction
2][3][4] This increase might be associated with changing sun exposure patterns, although many other melanoma risk factors might be implicated.][7][8][9][10][11] Although malignant melanoma is one of the most aggressive and life-threatening skin cancers, an early diagnosis could contribute to a successful treatment. 2,12ysplastic nevi are moles whose appearance is different from that of common nevi.They are clinically atypical with irregular borders and a variegated appearance.Their histological examination reveals architectural disorder, atypical patterns of growth, and cytological dysplasia.These nevi are very important, since they might be considered stimulants of melanoma and central risk factors and precursor lesions for cutaneous melanoma.][15][16] Correct and differential diagnosis of cutaneous melanoma is one of the major issues of clinical dermatology and is associated with the capability of differentiation from dysplastic melanocytic nevi, which are considered to be precursor lesions for melanoma.According to clinical studies, diagnosis by simple inspection has a sensitivity of 65% to 80%, depending on the dermatologist's experience.8][19][20] For nonspecialists, the aforementioned percentages are significantly lower.That fact has led to the development of imaging methods for the classification of pigmented lesions, such as the MelaFind, 21,22 SolarScan, 23 Multispectral, and Ultra spectral imaging systems 24,25 and MEDPHOS, 26 but none of them has been established in the diagnosis of melanoma so far.
MelaFind is an imaging system using multispectral illumination of both the clinical and dermoscopic images.This instrument acquires information on the absorption and scattering properties at different depths of a lesion, using 10 varying wavelengths from near infrared through the visible light spectrum.
Measurements of the size, border, and surface morphology of the lesion are obtained.The digital images are created with the use of a specialized imaging probe, which detects illumination in each spectral group.These images are then stored on a PC for further processing.With the use of accompanying software, an automatic differentiation between cutaneous melanomas and common nevi is being attempted.MelaFind is a well-known imaging system, but it does not examine three dimensions in detail, and it has not established in a dailybasis practice as a diagnostic tool for skin tumors. 21,22olarScan is an automated system for primary melanoma diagnosis that uses image analysis of dermoscopy (surface microscopy) features of melanocytic lesions of the skin.The obtained images of the lesions are evaluated by a computer according to different features, including color segmentation, lesion patterns, and geometry.The features of the lesions are evaluated against a database of statistical data on melanoma lesions.Finally, the results are presented on a PC for further clinical evaluation.This diagnostic instrument has a sensitivity of 85% and a specificity of 65% for melanoma.Preliminary data of studies for SolarScan suggested that its performance was comparable to that of physicians.However, further studies and clinical trials were necessary for confirmation of the results and improvement of the instrument. 23he common denominator of these methods is their inability to capture detailed information regarding surface anatomy.In clinical practice, apart from inspection, palpation is usually used for the evaluation of melanocytic lesions.As a result, information about the surface morphology, namely the third dimension of the skin lesion, is being incorporated in the diagnostic process.This information in combination with results from the ordinary two-dimensional (2D) measurements, such as perimeter, major and minor length, area, and color, may lead to a better, more precise surveillance of the melanocytic lesions.Herein, we present our initial results of a novel, noninvasive diagnostic technique for three-dimensional (3D) imaging of melanocytic lesions, using a new trinocular imaging system that enables the visualization and inspection of the obtained photorealistic 3D reconstruction, as well as the performance of metric measurements on surface and volume.If this new 3D evaluation system can measure changes in the skin surface more precisely than the 2D systems, it would undoubtedly be a very valuable diagnostic tool.

Computational Vision
A trinocular stereo system is employed to reconstruct the surfaces of interest (pigmented skin lesions).The apparatus is comprised of a trinocular imaging system and an illumination light source.The imaging system consists of three high-resolution cameras in approximately parallel arrangement, sharing a common baseline (Fig. 1).The acquired images are stereoscopically processed to obtain a metric and photorealistic 3D reconstruction of the lesion.Accompanying software developed in-house enables the visualization and inspection of this reconstruction, as well as the performance of metric measurements upon its surface and volume.In addition, this software facilitates the process of image acquisition both for lesion reconstruction and for system calibration.

Description of Apparatus
As mentioned previously, the three cameras of the stereoscopic system are configured upon a common baseline.All three cameras are of type "Flea-2" by Point Gray Research, have a resolution of 1; 280 × 960 pixels, and feature a Sony ICX445, 1∕3-inch charge-coupled device (CCD) sensor.The middle camera is a color camera, and the others are monochromatic.The vergence posture is such that the cameras aim toward the target location, which is assumed to be at a distance of ∼15 cm from the cameras; that is the axis of the lateral cameras, exhibiting an angle of ∼5 deg with respect to the central one.Correspondingly, all cameras are focused at a distance of 15 cm.Upon the baseline, neighboring cameras have a distance of ∼3.5 cm.In this way, the reconstruction target (lesion) is centered in the images, and the visual overlap between the cameras is maximal, thereby maximizing the reconstructed area.All cameras are configured to have a narrow field of view (FOV) lens (12.5 mm).In this way, most of the image is occupied by the lesion to be reconstructed, rather than its surrounding skin, which is not of interest in this study.
Monochromatic (grayscale) cameras exhibit relatively increased definition when compared with single CCD color ones of the same type and resolution.This is due to the effective blurring that is induced when interpolating Bayer-tiled images to generate the corresponding color images.Thus, monochromatic cameras typically are preferred over color ones when performing stereo reconstruction, as the increased definition of image structure facilitates better correspondence establishment across images.On the other hand, color is a valuable source of diagnostic information in dermatology.To balance between the two contradicting requirements, a color camera is used within the trinocular triplet, and the other two cameras are monochromatic.Since skin texture information in the reconstruction will originate from the color camera, it is placed in the middle of the trinocular triplet.In this way, the textured surface within the reconstruction is maximized, because the middle camera images relatively more common points with respect to the two lateral ones.
A light-emitting diode (LED) light source is mounted above the trinocular pair, illuminating the lesion from approximately the same viewpoint of the central camera.As can be observed in Fig. 2, the scene is well illuminated, and, in this way, potential shadows induced by the illumination are minimized.Adding additional light sources would decrease the small or nonexisting pronunciation of shadows but would increase the size of the apparatus and, in turn, decrease its level of ergonomy.Using the current setup, and given the typical structure of the reconstructed scenes, such shadows do not occur, or their spatial extent is very minute, and thus the reconstruction quality is not hindered on their behalf.
Moreover, a diffuser is placed in front of the LED light source in order to provide a homogeneous illumination and make the occurrence of unwanted specular light reflections, which could cause imperfections in the 3D reconstructions, improbable.
Furthermore, an appropriately modulated construction from Plexiglas is made on which cameras are positioned at specific locations.The distance between the cameras and the melanocytic lesions is set specific to the tri-camera's focal plane.Therefore, the Plexiglas construction, which is shown in Fig. 1, aids in rapid and accurate imaging of each lesion.During the imaging procedure, the melanocytic lesion is positioned at the aperture of the Plexiglas, which has a diameter of 5 cm and is placed at a distance of 15 cm from the cameras, as mentioned above.This Plexiglas construction contributes to a rapid, reproducible, and precise imaging with optimal stability without losing time in searching for the appropriate point for imaging.This also reduces movement of the imaging system.In addition, the whole apparatus is positioned on an ergonomic bracket in order to offer maximal stability, as well as an easy-to-handle technique for imaging of the melanocytic lesions.

Image Acquisition and Calibration Software
Camera calibration is a prerequisite for metric stereo reconstruction of surfaces.The three cameras are calibrated employing conventional techniques.More specifically, a checkerboard calibration target is utilized and detected in the images acquired during calibration following Vezhnevets et al. 27 After checkerboard calibration, an initial calibration of the intrinsic and extrinsic camera parameters is obtained using OpenCV.Finally, the estimates of these parameters are refined using the bundle adjustment procedure shown in Manolis et al. 28 The output calibration parameters are stored in an internal system file and are available upon system operation.
Image acquisition takes place through a user interface, which is shown in Fig. 2 and displays the images acquired by the three cameras in real-time.In addition, camera parameters (shutter time, gain, brightness, contrast, etc.) can be controlled through this software, which also controls the illumination source.During calibration, stereo triplets of the calibration target are acquired through this software, and during normal system operation, the lesion is aimed through the display.
Once the three images are acquired, a copy of the color image is converted to grayscale.Thus, the system retains two versions of this image: the original and a monochromatic one.This conversion is conventionally performed so that the luminance of the monochromatic image matches the luminance of the color image, which is achieved by weighting the red, green, and blue (R, G, and B) channels as follows: All four images (the three originals and the converted one) are stored in BMP format in eight-bit color (or intensity for the converted image), which is the maximum color depth of the camera.The three monochromatic images are subsequently utilized in computing the 3D reconstruction.The original color image is utilized to create the color texture of this reconstruction.

Reconstruction Algorithm
The stereo reconstruction approach that is adopted is based upon the widely known plane sweeping technique, adapted for a trinocular pair. 29We employ directly a trinocular reconstruction technique in order to improve accuracy, as opposed to, for example, using only a binocular pair.Furthermore, we prefer to use an intrinsically trinocular approach, rather than performing two (or even three) independent binocular reconstructions and fusing or merging the results, since typically the latter approach results in reconstruction artifacts or surface points that, due to noise, are reconstructed multiple times in slightly different 3D coordinates.
As a photo consistency metric for the correspondence establishment in the plane sweeping method, we use modified normalized cross correlation (MNCC). 30We combined the MNCC responses for every depth evaluated by plane sweeping through averaging.As observed by Vogiatzis et al., if the surface being imaged is devoid of occlusions, the correlation score may exhibit a local maximum near the correct depth, rather than a global one. 31Similarly to this work, we also employ a Parzen window to combine multiple views (the three possible binocular pairs, in our case), in order to consider both the actual scores of the local maxima and reinforcing local maxima that occur at similar depths. 32In this way, three correlation curves are obtained, with the reconstructed depth being determined at the point of maximum response.
The resulting depth map obtained from the above trinocular reconstruction is finally transformed into a mesh of triangles by taking into account the neighborhood relationships of (depth) pixels in this map.The depth map is then median filtered for noise, interpolating the missing surface points using a linear algorithm.In this way, texture from the middle color camera is mapped upon the mesh, producing a continuous photorealistic surface, instead of a point cloud.The result is stored in virtual reality modeling language (VRML) format for user inspection and measurements.

Data Analysis
Typically, the identification of lesions and diagnosis is based on visual examination by the physician, as well as on dermoscopy.In the field of machine vision, the aim is to develop an automatic inspection system, based mainly on the visual features perceived by the human eye and even the features beyond human perception that modern imaging technology permits.In this frame, we designed several metrics that would help the classification of the lesions using the 3D reconstruction from our camera system.Since the apparatus we developed visualizes and records in three dimensions, we considered extending the conventional 2D skin features [asymmetry, border, color, and diameter (ABCD)] with new 3D measurements.More specifically, we added the information of height, as maximum height and variance in height (V Hm ) along axis from the center of the lesion to the boundary, along with general curvature changes on the surface of the lesion.

Segmentation
The digital image acquired by the 3D camera includes information for the RGB color pigment densities (Pd R , Pd G , Pd B ) and height (H) for a large area, including the regions considered of interest for this investigation.The first step toward the extraction of features is the identification and segmentation of the region of interest.In order to allow for complicated cases to be analyzed by our algorithm, where the boundaries of the lesion could be faintly defined, we decided for the segmentation to be performed in a semi-automatic way under the guidance of the user.
Using the RGB color information, we constructed a Pd, with a bias toward the red (Pd R ) and green (Pd G ) channels, as most lesions were nearly brown in color (combination of red and green), as shown in This Pd was then plotted as height to create a pseudosurface, as it can be seen in Fig. 3(a), with the original RGB image used as texture.By moving a slider bar, the user chooses an iso-curve on the pseudo-surface to match the boundary of the lesion.We found that this method requires minimal interaction from the user while providing a good segmentation, especially with the cases where an automatic segmentation based on edge detection would not satisfy the correctness of the boundaries.For the case where this semi-automatic method does not produce totally accurate results, the user is allowed to provide amendments to the boundaries selected, as shown in Fig. 3(b).

Rotation
Once the boundary of the region of interest is defined from the segmentation, a great density 3D triangular surface mesh P T;S of the lesion is created with triangular elements T and nodes S. Let S B define the nodes on the boundary found from the segmentation.We realized that, for the 2D methods, the viewing angle of the camera in respect to the plane of the lesion is of paramount importance to the accurate extraction of information.Unfortunately for the 2D systems, the accuracy in the positioning of the camera is left to the discretion of the camera user.On the contrary, for the case of 3D imaging, we have the advantage of knowing the exact position of the lesion in space.We can therefore use this information to align the surface of the lesion orthogonal to the viewing axis of the camera.In this manner, a rotation matrix is computed using the nodes on the boundary S B of the 3D mesh and the axis of the camera.The rotation is applied to the nodes S before the collection of the features, as shown in Fig. 3(c).

Area
The area is one of the features that is traditionally used to classify a lesion.In our case, we use to calculate the discretized sum of the areas of all the triangles in the triangulation P T;S .

Major axis length
The major diagonal of the shape is the maximum distance between any two nodes on the boundary of the lesion, as shown in where i and j indicate to different points on the boundary, as shown in Fig. 3(d).

Perimeter
The length of the boundaries of the lesion is measured from the mesh shown in where b is the number of boundary points S B .

Peak to center distance
A feature showing the lack of symmetry in a lesion is the distance between the barycenter of the lesion B P ¼ EðS B Þ, where E stands from mean, and the point h of maximum height Hmax, as shown in Fig. 3(d).This can be expressed as (5)

Variance in pigment density and height
In order to examine the V Hm and the variance in the pigment density (V Pdm ) on the surface of the lesion, and most significantly along the direction from the center of the lesion to its boundary, we defined m axial lines L m , as shown in Fig. 3(e).The H Lm and the Pd Lm are sampled in equal distributed points along the axial lines starting from the center to the boundary.Then an estimator of the V Hm along the line L m is calculated by where μ Hm ¼ EðH Lm Þ is the mean value of height along the axial line L m .And, respectively for the V Pdm , where μ Pdm ¼ EðPd Lm Þ is the mean value of density along the axial line L m .Then a sum of the separate variances along the m axis L m is calculated by

Volume
Since we have a triangulation P T;S of the surface of the lesion, it is trivial to calculate the volume between the surface of the lesion and the z ¼ 0 plane that we take as a reference for the base of the lesion's mesh.More specifically, we construct the prism G i defined by the triangle T i of the triangulation and its respective projection to the z ¼ 0 plane the triangle T iz .
Multiplying the area AREAðT iz Þ of the projected triangle to the average height of the original triangle T i provides a good estimation of the overall volume when a dense triangulation is used [Fig.3(f)], as shown in 2.5.9Boundary's asymmetry The empirical knowledge of the clinicians pointed out that the similarity of the lesion's boundary to a circle is of great significance for the evaluation of the melanocytic lesions.Therefore, we defined a boundary's asymmetry feature to acquire this information.A circle C with center in the barycenter B P of P T;S and radius r the mean of the distances from the barycenter B P the nodes S B on the boundary of the lesion is created, as shown in The boundary of the lesion and the circle C are both transformed to polar coordinates with center in B P , and the distances between the equivalent points on the boundary of the lesion and those on the circumference of the circle are summed up to give the measure of boundary's asymmetry (Fig. 4), where r SBi is the radial coordinate for the boundary node S Bi .Boundary's asymmetry goes to zero when the boundary fits perfectly to a circle, and therefore the symmetry in respect to the center is maximum and increases the more asymmetric the lesion becomes.

Max density curvature and max height curvature
Finally, the correctly oriented mesh of the lesion is used to calculate the curvature of the density and the height for the whole area.The maximum value for these second derivatives for height and for density is collected.

Sample Preparation
In our study, patients presented at the Pigmented Skin Lesion Clinic of the University Hospital of Heraklion, Crete, were involved.The same two experienced dermatologists interviewed and proceeded with complete skin examination of the patients.Based on the ABCDE rule of dermoscopy for the early diagnosis of melanoma (A: asymmetry, B: border, C: color, D: diameter > 5 mm, E: evolution), both benign (common and Fig. 4 At the top of the graph, we can see the lesion's boundary superimposed with the ideal circle in green and the sampled radial axis in red.The offset between the radius of the lesion and the radius of the ideal circle is noted in blue.At the bottom of the graph, we present the offset in the y-axis for each angle (in degrees) sampled radial line along the circumference of the circle in the x-axis (also shown in Fig. 10, in color).
atypical nevi) and malignant melanocytic lesions (melanomas) were included in the study.The lesions were prepared for imaging in the Pigmented Skin Lesion Clinic in full accordance with the rules and guidelines of the Ethics Committee of the University Hospital of Heraklion.The clinical investigations were conducted according to the Declaration of Helsinki Principles.Institutional approval and patient informed consent were obtained prior to imaging of the melanocytic lesions, after clinical and dermoscopic examination.In order to achieve the best possible quality of 3D reconstruction, the hairs around a lesion were shaved when necessary, since they can affect the reconstruction.In addition, we used sterile splenium impregnated with alcohol for purification and cleaning of the skin oiliness, decreasing the light reflections and specularities.This aided in decreasing imperfection in the 3D reconstruction in combination with the use of the diffuser mentioned before, because the surfaces better approximated the Lambertian reflection behavior, which is assumed by the employed stereo reconstruction algorithm.
At this point, it must be noted that there are some restrictions in our study.The specifications and the settings of our cameras, and the specialized positioning on the Plexiglas construction in specific distances in order to achieve the best possible accuracy of the 3D reconstructions, have as a result a limiting size of skin lesions which can be imaged, measuring up to 2.5 × 2.5 cm.Larger lesions needed to be excluded from the study, because the imaging of the whole sample from every camera is not possible at the same time.In addition, the areas of the body where the lesion we want to image is located need to be flat or approximately flat.As a consequence, from our imaging process, some areas of the body (such as nose, ear, fingers, and lips) have to be excluded, due to the technical difficulties in positioning the Plexiglas construction in a reproducible and stable manner on the corresponding area for imaging the melanocytic lesion.

Statistical analysis
Comparisons of variable distributions between groups were performed using analysis of variance (ANOVA) with Bonferroni's correction for multiple comparisons.All P values calculated were two-sided, and the significance level was chosen to be 0.05.All calculations were performed with the Statistical Package for Social Sciences (SPSS) version 17.0.

Results
The study included 30 samples, divided as follows: six samples were identified as malignant melanomas of the skin, 10 as dysplastic melanocytic nevi, and 14 were histologically characterized as nondysplastic melanocytic nevi, though clinically and dermoscopically atypical.Normal skin was imaged and used as control.Differences in the obtained sequential 3D reconstructions were compared with clinical and histological measures in order to provide an accurate and valuable method for the evaluation of the melanocytic lesions.
First of all, experimental procedures including six specific phantoms of known volume, measured with the use of volumetric tubes, were conducted to confirm the accuracy and the correctness of our 3D system measurements.Every phantom was measured three times using three different volumetric tubes.In addition, every phantom was photographed by the 3D system from three different optical sides in order to gather three measurements of the volume.Our aim was to examine the accuracy, as well as the repeatability, of our imaging system.In five out of six phantoms, the mean values obtained by volumetry were slightly lower than those obtained by our 3D imaging system.The mean AE SD percent difference was 3.8% (AE0.98).The results of these tests are presented in Fig. 5.
In addition, tests with five melanocytic lesions, which were imaged from three different sides, were completed in order to examine the repeatability of our results and the reproducibility of our system.Three different imaging sets from three different sides (therefore, three different 3D reconstructions for each melanocytic lesion) took place.Measurements from each reconstruction of the same lesion were compared with each other in order to examine the reliability of our method and to evaluate the repeatability of our results.The mean (AESD) percent differences in the measurements obtained for area, major axis length, volume, and max height were 2.1% AE 1.1, 0.9% AE 0.8, 3.8% AE 2.9, and 2.5% AE 3.5, respectively.The results of these tests are presented in Fig. 6.
In the multiple comparisons of the results of all the parameters that were studied, we noticed that V Hm is a significant factor that could contribute to the evaluation of the melanocytic skin lesions.As shown in Fig. 7, there was a statistically significant difference in V Hm values between malignant and benign melanocytic lesions (common and dysplastic nevi) (P < 0.001).The V Hm values for melanomas was observed to be significantly higher [mean ðAESDÞ ¼ 13.54 AE 8.29] than those for dysplastic nevi [mean ðAESDÞ ¼ 0.46 AE 0.44] and common nevi [mean ðAESDÞ ¼ 0.51 AE 0.288].Furthermore, as presented in Fig. 8, in malignant melanomas [Fig.8(c)], the axial lines from the center of the lesion to its boundary [see Fig. 3(e)] appear to have greater abnormality compared with the corresponding axial lines in benign skin lesions [common and dysplastic nevi-Fig.8(a) and 8(b), respectively].Greater abnormality of the axial lines means more extensive irregularity in the height alteration on the surface of the lesion, which contributes to higher values in V Hm .These results suggest that our 3D imaging device could aid in classification between malignant and benign melanocytic skin lesions.with the use of volumetric tubes, along with the volume measured by our imaging device.Every phantom was measured three times using three different volumetric tubes.In addition, every phantom was photographed by the 3D system from three different optical sides in order to gather three measurements of volume.Volume measurements of the phantoms using volumetric tubes rendered slightly lower values than those obtained by our 3D imaging system (error bars: standard deviation).
In addition, another parameter which could contribute to the differentiation between malignant and benign melanocytic skin lesions, in conjunction with the V Hm , is the boundary's asymmetry of the lesion.This is a parameter which reflects the atypia and the asymmetry of the boundaries of a melanocytic lesion, since the similarity of the lesion's boundary to a circle is of great significance for the evaluation of the melanocytic lesions, which we defined through the boundary's symmetry.The sum of the polar coordinates with center in B P and the distances between the equivalent points on the boundary of the lesion and those on the circumference of the circle define the boundary's asymmetry value, as described in Sec.2.5.9 (Fig. 4).As presented in Fig. 9, there was a statistically significant difference in boundary's asymmetry values between malignant and benign melanocytic lesions (common and dysplastic nevi) (P < 0.001).The boundary's asymmetry values for melanomas were detected to be higher [mean ðAESDÞ ¼ 525.08 AE 267.136] than those for dysplastic nevi [mean ðAESDÞ ¼ 103.9 AE 48.176] and common nevi [mean ðAESDÞ ¼ 89.01 AE 41.141].As confirmed in Fig. 10, where the boundary's asymmetry of a common nevus [Fig.10(a)], a dysplastic nevus [Fig.10(b)] and a melanoma [Fig.10(c)] are presented, the corresponding values are 65.352, 111.759, and 280.158, respectively.Higher values of boundary's asymmetry are noticed in melanoma compared with benign melanocytic skin lesions.The boundary's asymmetry results, in combination with the V Hm values, suggest that this new instrument might be a valuable tool for imaging and evaluation of melanocytic skin lesions.
Finally, as shown in Fig. 11, the V Pdm could be the parameter which would automatically differentiate common nevi from dysplastic nevi and malignant melanomas.A statistically significant difference in V Pdm values in common nevi compared with those in dysplastic nevi and malignant melanomas was noticed (P < 0.001).The V Pdm values for melanomas was detected to be higher [mean ðAESDÞ    Greater abnormality of the axial lines means larger irregularity in the color distribution and its alteration on the surface of a lesion, which contributes to higher values of V Pdm .These results suggest that our novel imaging system could be employed for the evaluation and classification of pigmented skin lesions and might be a valuable diagnostic tool for physicians.The other features that were exported automatically using our imaging system, such as perimeter and diameter, are more descriptive of the lesions than significant for their discrimination.They are statistically significant (the data are not presented in the manuscript) but not as medically, clinically, or diagnostically significant as V Hm , boundary's asymmetry, and V Pdm , on which we decided to base our study for the current manuscript.Our interpretation was that they are parameters which are exported automatically, accurately, and rapidly using our system, and they are more significant for the description and the presentation of the skin melanocytic lesions.

Discussion and Conclusion
Up to now, many computer-based methods for the evaluation of melanocytic lesions have been developed in an attempt to supplement clinical examination and the ABCDE system.In the present study, we have developed a novel, stereoscopical 3D surface imaging system with the aim of classifying melanocytic skin lesions.Our measurements have demonstrated that this new instrument has great potential for the imaging and evaluation of melanocytic skin lesions.Our study provides preliminary evidence that the metric and photorealistic 3D reconstructions, as provided by our imaging device and the accompanying software, could be potentially very useful for the differential diagnosis of common nevi, dysplastic nevi, and cutaneous melanoma.It is an objective, repeatable, and automated method which could aid in rapid and sure discrimination of skin melanocytic lesions.It is also a system which can provide automatic, precise, and recordable 2D and 3D measurements with the benefit of creating files with the final results for each patient, with the potentiality of using them in follow-ups, supplementing and improving the clinical examination.
The imaging accuracy of the melanocytic lesions and their rapid and precise evaluation is an important issue for the diagnosis and the prognosis of skin tumors.The efforts for the development of photographic systems which could improve the imaging accuracy of the melanocytic lesions, as well as their evaluation, led to the emergence of imaging techniques such as the MelaFind, 21,22 SolarScan, 23 Multispectral, and Ultra spectral imaging systems 24,25 and MEDPHOS. 26However, up to date, none of them is used widely on a daily basis as a diagnostic tool.
In this study, we have chosen to examine the potential of a new 3D stereoscopic surface imaging system which enables us to obtain metric and photorealistic 3D reconstructions of the melanocytic lesions and to proceed to their visualization and inspection.The initial goal of this study was to assess the performance of metric measurements upon surface and volume and their utility in melanocytic nevi evaluation and in melanoma diagnosis.Our initial measurements were accomplished in vivo in order to evaluate different types of pigmented skin lesions and study the surface morphology of a lesion and its changes.The discrimination was performed independently on each feature, after multiple comparisons of the results of all the parameters that were studied and exportation and presentation of the statistically significant ones, as well as the medically, clinically, and diagnostically significant ones.
Specifically, V Pdm in conjunction with V Hm and boundary's asymmetry were proven to be the most significant parameters to differentiate among common nevi, dysplastic nevi, and malignant melanomas, with the drawback that the sample is rather small and should be confirmed by larger-scale studies.The other features were considered as more descriptive of the lesions and are more important for their presentation during the clinical examination than for the skin lesions' discrimination.This   in vivo study is an adjunct and a complement to the previous ex vivo one of optical computed tomography for discrimination between benign and malignant pigmented skin lesions. 33Our team has a continuous interest in developing novel ex vivo and in vivo computerized imaging systems for the early diagnosis and prognosis of the behavior of malignant melanoma.
The performance of measurements upon the photorealistic 3D reconstructions, as provided by our imaging device, as well as comparisons between results of different types of lesions, lead to significant parameters which might aid in the differential diagnosis of common nevi, dysplastic nevi, and cutaneous melanomas.An important difference between malignant and benign melanocytic lesions is the abnormality in the height alteration on the surface of the lesion, which can influence the V Hm .Hence, higher values of V Hm because of a greater height abnormality can be expected from malignant lesions, which is what we have also noticed in our measurements.The corresponding Fig. 8 confirms this suggestion, as in malignant lesions, the axial lines from the center of the lesion to its boundary present greater abnormality, thus showing higher values of V Hm [Fig.8(c)], compared with the corresponding axial lines in benign melanocytic lesions [Fig.8(a) and 8(b)].In addition, another parameter which was examined and could show the atypia in the boundaries of a skin lesion is boundary's asymmetry.The similarity or not of a lesion to a circle could be of great significance in the evaluation of melanocytic skin lesions (Fig. 4).The greater the asymmetry of a melanocytic lesion is, the higher value the boundary's asymmetry has.Therefore, another significant difference between malignant and benign melanocytic lesions is the asymmetry of the boundaries, which can increase our parameter (boundary's asymmetry) in our measurements.As a consequence, evaluation of the V Hm and boundary's asymmetry could lead to differentiation between benign and malignant melanocytic skin lesions.In conjunction with the above parameters, we can take under consideration the V Pdm , a parameter which could "measure" the abnormality in the color distribution and its alteration on the surface of a lesion.Greater color abnormality, which means higher values of V Pdm , can be expected from more atypical lesions, as is confirmed by our measurements.As a consequence, common nevi could be discriminated from dysplastic nevi and malignant melanomas, as presented in the corresponding Fig. 12, confirming that, in malignant melanomas [Fig.12(c)] and dysplastic nevi [Fig.12(b)], the axial lines from the center of the lesion to its boundary present greater abnormality (and thus higher values of V Pdm ) compared with the corresponding axial lines in common nevi [Fig.12(a)].
Currently, studies for imaging and evaluation of other skin tumors such as basal cell carcinoma (BCC) and squamous cell carcinoma (SCC), apart from melanocytic lesions presented in this work, are under way.
Our future aim is to have a follow-up of a large number of melanocytic lesions from patients who are under regular monitoring in the Pigmented Skin Lesion Clinic of the University Hospital of Heraklion, Crete, and notice alterations in their characteristics on the surface.Provided that the results of the current study will be confirmed in a formal clinical trial, the ultimate goal would be the creation of an "atypia index" of the melanocytic skin lesions, taking into account all the 2D and 3D measurements for a rapid and accurate evaluation of the pigmented skin lesions.Heraklion, Crete, for help, expertise, and advice, as well as P. Panteleris, I. Karatzanis, M. Bouloukakis, and T. Sarmis from the Foundation for Research and Technology-Hellas (FORTH) for their contribution to the development of the 3D imaging system and the corresponding software.

Fig. 2
Fig.2The user interface which displays the images acquired by the three cameras in real-time.

Fig. 3
Fig.3(a) The pseudo-surface created with the original RGB image used as texture, after pigment density was plotted as height.By moving a slider bar, the user chooses an iso-curve on the pseudo-surface to match the boundary of the lesion.(b) Where the semi-automatic method does not produce totally accurate results, the user is allowed to provide amendments to the boundaries selected, pushing the redraw button (arrow).(c) A rotation matrix is computed using the nodes on the boundary S B (the nodes on the boundary found from the segmentation) of the 3D mesh and the axis of the camera.The rotation is applied to the nodes S before the collection of the features, in order to align the surface of the lesion orthogonal to the viewing axis of the camera.(d) The major diagonal of the shape is the maximum distance between any two nodes on the boundary of the lesion.(e) The axial lines starting from the center of the lesion to the boundary and the corresponding alterations in height and pigment density along these axial lines on the surface of the lesion.(f) We construct the prism G i defined by the triangle T i of the triangulation and its respective projection to the z ¼ 0 plane the triangle T iz .Multiplying the area AREAðT iz Þ of the projected triangle to the average height of the original triangle T i provides a good estimation of the overall volume when a dense triangulation is used.

Fig. 5
Fig.5Diagrams of six specific phantoms of known volume, measured with the use of volumetric tubes, along with the volume measured by our imaging device.Every phantom was measured three times using three different volumetric tubes.In addition, every phantom was photographed by the 3D system from three different optical sides in order to gather three measurements of volume.Volume measurements of the phantoms using volumetric tubes rendered slightly lower values than those obtained by our 3D imaging system (error bars: standard deviation).

Fig. 6
Fig. 6 Diagrams of four different parameters (volume, height, maximum diameter, and area) that were studied during tests in five melanocytic lesions, which were imaged from three different sides, in order to examine the reproducibility of our method (error bars: standard deviation).

Fig. 7
Fig. 7 Boxplot of values of variance in height (V Hm ) for each group of lesions (asterisks: P < 0.001, error bars: standard deviation).

Fig. 12 ,
Fig. 12, in malignant melanomas [Fig.12(c)] and dysplastic nevi [Fig.12(b)], the axial lines from the center of the lesion to its boundary [see Fig. 3(e)] present greater abnormality compared with the corresponding axial lines in common nevi [Fig.12(a)].Greater abnormality of the axial lines means larger irregularity in the color distribution and its alteration on the surface of a lesion, which contributes to higher values of V Pdm .These results suggest that our novel imaging system could be employed for the evaluation and classification of pigmented skin lesions and might be a valuable diagnostic tool for physicians.The other features that were exported automatically using our imaging system, such as perimeter and diameter, are more descriptive of the

Fig. 8
Fig.8The variation in height (V Hm ) along the axial lines from the center of the lesion to its boundary in (a) a common nevus, (b) a dysplastic nevus, and (c) a melanoma.

Fig. 9
Fig. 9 Boxplot of values of boundary's asymmetry for each group of lesions (asterisks: P < 0.001, error bars: standard deviation).

Fig. 10
Fig. 10 At the top of each of the graphs (a, b, c), we can see the lesion's boundary superimposed with the ideal circle in green color and the sampled radial axis in red.The offset between the radius of the lesion and the radius of ideal circle is noted in blue.At the bottom of each graph, we present the offset in the y-axis for each angle (in degrees) sampled radial line along the circumference of the circle in the x-axis.(a) Common nevus, (b) dysplastic nevus, (c) malignant melanoma.

Fig. 11
Fig. 11 Boxplot of values of variance in pigment density (V Pdm ) for each group of lesions (asterisks: P < 0.001, error bars: standard deviation).

Fig. 12
Fig.12The variation in pigment density (V Pdm ) along the axial lines from the center of the lesion to its boundary in (a) a common nevus, (b) a dysplastic nevus, and (c) a melanoma.