2.1.

Spectrometer System

The instrument used to record the spectra is shown schematically in Fig. 1 . Diffuse reflectance spectra in the wavelength range $550\text{to}1000\mathrm{nm}$ were obtained using a pocket spectrometer (CVI Laser Systems, Albuquerque, NM) with a $2048\text{-}\text{pixel}$ linear charge-coupled device detector and spectral resolution of $2\mathrm{nm}$ . The light source is a lamp utilizing a regulated 2900-K tungsten-halogen bulb. The fiber probe contains seven $400\text{-}\mu \mathrm{m}$ core step-index fibers optimized for low loss in the UV/VIS region in a close-packed six-illumination-around-one-collection arrangement with center-center separations of $470\mu \mathrm{m}$ and an outer diameter of $1.3\mathrm{mm}$ (CVI Laser Systems). Data were acquired from the spectrometer via a digital acquisition card (National Instruments). The acquisition software was written in LabView and produced data files suitable for further processing and analysis in Matlab.

Fig. 1

Schematic diagram of the fiber probe-based skin spectroscopy instrument.

For calibration, reference spectra from a white diffusing reflectance standard (CVI Laser Systems) and a dark scan were recorded using the same integration time as the measurement. The fiber probe end was held at a fixed angle and distance to the reflectance standard in a blackened, light-tight enclosure designed to eliminate secondary reflections and stray light. This distance and angle were designed to yield a reflectance slightly greater than that of pale skin in contact with the probe. A standard used in a contact geometry with optical properties closer to that of skin would be preferable, but we know of none with sufficiently well characterized properties. The lamp produced stable output after less than five minutes of operation. Warm-up times of at least five minutes were used for all calibration and measurement. Spectra were recorded by taking ten spectral measurements over a period of five seconds, each with an integration time of $30\mathrm{ms}$ . These measurements were averaged to obtain the final spectrum. Varying probe pressure did affect the spectra, but spectra measured with this system from a small area of normal skin were found to vary by less than 5%. A trained operator was able to achieve repeatability on the order of 2%.

2.2.

Measurement Protocol and Histopathology

In this study, we sought (for the first time to our knowledge) to more closely couple the process of histopathological diagnosis with the spatial localization provided by the fiber probe. This was achieved in three steps: 1. record a localized spectrum; 2. transfer the location of the probe from the tissue in vivo to the microscopic histological section; and 3. record a histopathological diagnosis of a type and scale relevant to the recorded spectrum. This last represents a major departure from routine histopathology practice.

To determine the tissue volume over which the probe was most sensitive, we assumed the sensitivity to tissue changes to be proportional to the photon fluence.¹⁷ Light transmission and collection through skin was simulated with a layered model (to be described elsewhere) and Monte Carlo modeling based on a modified version of the multi-layered Monte Carlo software package (MCML).¹⁸ We concluded that the area of highest sensitivity was confined to a $400\text{-}\mu \mathrm{m}$ -diam cylinder immediately below the central collection fiber. This cylinder had a wavelength-dependent depth range of $200\text{to}400\mu \mathrm{m}$ , extending into the papillary dermis.

2.2.1.

Record a localized spectrum

To make multiple spectral measurements from different areas of the same lesion, the skin surrounding the lesion was marked with a $2\text{-}\mathrm{mm}$ reference grid using ink that remained visible throughout the excision and subsequent tissue processing steps. These markings were used to guide the placement of the probe via a matched template attached to it. At each location, the probe was placed in light contact with the skin (using index-matching glycerol) and a spectrum was recorded. Figure 2 shows schematic diagrams of the grid, the matching template, the scan locations on pathology tissue blocks, and a photograph of a marked lesion. A spectrum was also taken from the normal tissue adjacent to the lesion.

Fig. 2

Measurement procedure: (a) lesion with marked measurement grid, shrinkage references, ellipse, and scan positions; (b) template used to position probe; (c) excised tissue blocks showing offset scan positions; and (d) photograph of a lesion.

2.2.2.

Transfer probe locations to histopathological sections

Immediately following the recording of the spectra, the lesion was excised following an inscribed ellipse and placed in formalin. An incision was made along one side of the grid as a position reference. When the size of the ellipse permitted, a second double incision was made along the opposite grid edge to permit shrinkage measurement. The ellipse and incisions are shown schematically in Fig. 2(a). The incisions appeared on the sections as notches, as illustrated in Fig. 3 . The excised tissue was cut along the grid lines into $2\text{-}\mathrm{mm}$ -wide blocks, noting the side adjacent to the probe location [Fig. 2(c)]. Standard histological procedures for cutaneous lesion diagnosis¹⁹ were then followed. The blocks of tissue were dehydrated, clarified, embedded in wax, and sliced with a microtome, initially to cut back the uneven tissue surface until continuous sections were obtained, and then to yield $5\text{-}\mu \mathrm{m}$ -thick sections. These sections were stained using hematoxylin and eosin solution and mounted on a microscope slide.

Fig. 3

Example section showing shrinkage and position reference incisions, and a magnified location showing relative size of diagnostic grid.

The initial cutting back of the tissue with the microtome was determined to remove approximately $0.5\mathrm{mm}$ of tissue, but this varied between samples, leading to some uncertainty in position. The measurement grid was offset by $0.5\mathrm{mm}$ from the marked grid lines to compensate for this. Tissue shrinkage also caused uncertainty in position. It may occur during the excision, fixation, and histopathology processing steps. Thin sections may exhibit nonuniform shrinkage, resulting in uneven curvature of the skin’s surface, which impedes the accurate measurement of length. Lateral shrinkage was determined from a comparison of the measured distance between the reference marks on the mounted section and the known grid spacing. Measured values were in the range 0 to 25%. These values were used to adjust the locations for localized histopathology.

Shrinkage could only be measured when the excision area was large enough to permit both reference incisions to be made. Cosmetic considerations frequently prevented this, resulting in shrinkage data being available for fewer than half of the recorded lesions.

2.3.

Description of Clinical Data

Table 1 lists the number of lesions and spectra collected during this study in each of the four pathology groups. The 120 lesions were obtained from 115 patients. The number of spectra for each lesion varied with the number of grid intersections that lay within the lesion. Lesion sizes recorded in our study varied with type: dysplastic nevi ranged in size from $2\text{to}15\mathrm{mm}$ in diameter and melanomas from $2\text{to}20\mathrm{mm}$ . The average number of spectra taken for each lesion type ranged from 3.2 in nevus, to 5.6 in invasive melanoma and 5.0 in dysplastic nevus.

Table 1

Lesions and localized spectra recorded during this study.

A new feature of this study is the collection of multiple spectra from each lesion. Our measurement grid ensures that these spectra are from nonoverlapping areas spaced over the whole lesion. Indications of the variability of localized histological diagnosis within our measured lesions are provided in Table 2 . Each of the abnormal pathology subgroups have a localized diagnosis of common nevus in around 10% or more of instances, and invasive melanoma has a localized diagnosis of in situ melanoma in around 10% of instances. The whole-lesion histopathological diagnosis (the gold standard) is as severe as the worst local diagnosis. For example, an in situ melanoma may be associated with areas of common and dysplastic nevus, but will not contain any regions of invasive melanoma.

Table 2

Local diagnoses (columns) as a percentage for each category of overall histopathological diagnosis (rows).

Average spectra recorded by the system for normal skin, dysplastic nevus, in situ melanoma, and invasive melanoma are shown in Fig. 4 . The shaded areas represent the regions within one standard deviation above and below the average curves shown. The figures illustrate the variability between and within the groups of spectra, and provide an indication of the difficulty of the classification problem.

Fig. 4

Measured spectra showing average curves and $\pm 1$ SD boundaries for skin and invasive melanoma (upper) and dysplastic nevus and in situ melanoma (lower).

2.4.

Data Analysis

Little information currently exists on the relationship between specific types of pigmented lesions and their reflectance spectra. In the absence of such information, we approach the problem of lesion classification with statistical techniques and seek classifiers that can separate lesions into pathology groups. Next, we describe the extraction of numeric features from the spectra, the construction and evaluation of classifiers, and the evaluation of the importance of features.

3.1.

Classifiers

Three- and five-feature classifiers were constructed using feature sets A and B for a range of groupings reported with sensitivity/specificity in Table 3 . Classifiers were constructed to distinguish between benign (N and DN) and malignant (ISM and IM) pigmented lesions. The best classifier had sensitivity/specificity of $64∕69\%$ and used five features from the feature set B. Under internal analysis, the same classifier had sensitivity/specificity of $72∕78\%$ . For comparison, classifiers were constructed between N and IM. Although not clinically useful for early melanoma, this classification facilitates the comparison with other studies in Sec. 4. The best classifier had sensitivity/specificity of $73∕73\%$ and used five features from feature set B. Under internal analysis, the same classifier had sensitivity/specificity of $74∕83\%$ .

Table 3

Five-fold cross-validated classifier performance for three- and five-feature classifiers expressed as percentage sensitivity/specificity for the feature sets A and B.

Given the modest performance of these classifiers, three additional classifier types were generated to test the hypothesis that the different stages of melanoma development produce distinct changes to the recorded spectra that are not easily distinguished by a single benign (B)/malignant (M) classifier. The classifiers were set up between N and DN, DN and ISM, and ISM and IM, and constructed from five-feature classifiers from feature set B. Figure 5 displays representative ROC curves for each classifier as well as for the B/M classifier. Ideally, the B/M classifier should exhibit significantly better performance than the others, as it is based on a larger dataset. In fact, the performance of the N/DN classifier exceeds that of the B/M classifier, and the DN/ISM and ISM/IM classifiers display better performance over much of the threshold range. These results suggest that an approach utilizing multiple classifiers may perform better. This hypothesis is explored further in Sec. 4. We cannot test multiple classifier performance directly, as the subdivided datasets are too small for such an analysis. We instead describe a method for evaluating and comparing the significant spectral features in each classifier.

Fig. 5

Receiver-operator characteristic curves for selected diagnosis-based groups of melanoma and nevus. The straight curve represents random classification.

3.2.

Quantifying Feature Importance

An obvious approach for determining the most effective classifier features is to declare those features in the best classifier to be the most important. This approach is flawed, as the single best classifier may have arisen through a coincidental fit and may not perform well on new data. Our feature importance metric is the number of occurrences of each feature within the top 1% of classifier feature sets, which is similar to the approach suggested by Huberty.²⁰ We are able to confidently report feature significance, because we evaluate the performance of all classifiers for each of the feature sets. Figure 6 displays histograms of the frequency of occurrence of these features in the top-ranked three- and five-feature classifiers based on feature set A for the N/DN, DN/ISM, and ISM/IM classifiers. Figure 7 shows the same feature frequency histograms for the top classifiers obtained using feature set B. These results are discussed in the next section.

Fig. 6

Feature significance counts for feature set A for each classification group for three- and five-feature classifiers. The $x$ axis enumerates features.

Fig. 7

Feature significance counts for feature set B for each classification group for three- and five-feature classifiers. The $x$ axis represents the spectral sampling point.

4.1.

Issues Arising from this Study

4.2.

Wider Implications

Ours is the only reported study based on a single illumination-collection fiber spacing. Wallace ’s probe had 18 illumination and 12 collection fibers arranged symmetrically with several different spacings and a total diameter of $1.5\mathrm{mm}$ ,¹³ similar to our own $\left(1.3\mathrm{mm}\right)$ . McIntosh, in contrast, used a much larger probe with a $7\mathrm{mm}$ diameter held slightly above the surface of the skin, causing the returned signal to be averaged over the lesion.¹⁵ Neither study quantified the volume of sensitivity of the probe. Garcia-Uribe employed a probe with varying fiber spacings across the lesion and recorded the spectra returned from each fiber.¹⁶ No correction was reported for the different sampling depths toward one side of the lesion, the effect of the probe’s asymmetry, or the contribution of lesion extent to these readings. While these studies provide a useful starting point, the use of varying fiber spacings and unquantified volumes of probe sensitivity prevent the studies from being extended to make use of localized histopathology. Furthermore, sampling of a whole lesion could prevent the identification of small morphological changes that are important in the histological detection of early melanoma.

Wallace ¹³ reported a good classification performance of $100∕84\%$ with a set of 15 melanoma and 32 compound nevi (i.e., common nevi containing melanocytes in the epidermis and dermis). An extension to this study¹⁴ reported a reduced performance of $83∕88\%$ using a neural network classifier on a larger dataset with 26 melanoma and 49 common nevi. This classification task differed from the one we report by the exclusion of the intermediate cases of dysplastic nevus and in situ melanoma. McIntosh and Garcia-Uribe both reported high classification accuracies (97 and 100%, respectively) in distinguishing nevus from dysplastic nevus. Our N/DN classifier had the highest performance, suggesting it is the most tractable of the clinically useful classifiers.

Most other studies of the diagnosis of pigmented lesions have employed a camera-based system. This has been coupled with image processing to replicate the procedures followed by physicians, or has included images taken at multiple wavelengths and their analysis. Camera-based systems are further developed than other technologies for lesion classification and have been the subject of larger clinical trials. The B/M classification performance (sensitivity/specificity) of these systems include the Spectrophotometric Intracutaneous Analysis system SIAscope’s $80.1∕82.7\%$ on a set of 384 pigmented lesions with 52 melanomas with six in situ;²³ Electro-Optical Sciences (Irvington, NY) MelaFind’s $100∕85\%$ on a set of 246 lesions with 63 melanomas comprising 33 invasive and 30 in situ;²⁴ and the diagnostic and neural analysis of skin cancer (DANAOS) study that yielded $82∕85\%$ with a dataset containing 2218 lesions with 187 melanoma.²⁵ Several of these studies relied on leave-one-out analysis to verify their classification accuracies. This technique has greater variance than five-fold cross-validation and may yield overestimates for individual classifiers.²⁰

The presence and handling of the intermediate lesion pathologies (DN and ISM) in these studies varied considerably. While the SIAscope and MelaFind studies did distinguish between in situ and invasive melanoma, the classifier was constructed to distinguish the whole melanoma set. The small proportion of in situ melanoma in the SIAscope study is not representative of early melanoma. The DANAOS study did not distinguish between melanoma development stages; instead, melanoma were grouped by pathology type with the majority being superficial spreading melanoma. The proportion of dysplastic nevus also determines the relevance of the study to early melanoma. The DANAOS and SIAscope studies presented 11 and 3% of their total nevus as dysplastic. The MelaFind study reported 60%, which was similar to our clinically observed population. Our own results (Table 3) suggest that populations lacking the intermediate cases may yield artificially high classification performances. In our view, most lesion classification studies reported to date should be considered as preliminary, and require larger and more refined clinical trials to further evaluate and develop their potential.