2.1.

Candidates Detection-Based Morphological Operation

Mathematical morphology operation is a way of nonlinear filtering used for image processing.⁵ The primary operations of mathematical morphology are dilation and erosion, denoted by ⊕ and Θ, on which other operations are based, such as opening and closing operations. In this paper, we use a black-top-hat transform. It can extract dark objects and structures in gray images. In the RGB images, the green channel exhibits the best contrast. We work on the gray image from the green channel, denoted by f. In the gray image f, MAs appear as dark patterns, small, isolated, and of circular shape.

Black-top-hat transform is the first closing operation on the image f and then subtracting the original image, defined as

The blood vessels in the retinal images are usually considered as piecewise linear structures at different orientations. A total of 12 rotated linear structures are used with a radial resolution of 15 deg. The length of a linear structuring element should be such that it is larger than MAs in retinal images. MAs appear as dark patterns within s pixels, which is <125 μm. Thus, the length of the linear-structuring element must be longer than s pixels and s is variable for different sizes of original images. For each pixel, record the minimum response as the closed result applied with those 12 rotated linear-structuring elements. Then, we obtain the image f _b by subtracting the original gray image from the closed image. The image f _b contains MAs, which appear as the local bright regions in Fig. 2(a).

Fig. 2

(a) Image after black-top-hat transform and (b) binary image after matched filter.

Only a certain number of candidates are reasonable (for example, several dozens for each image). A matched filter is used to extract regions of interest (ROI) from f _b. The matched filter is a 2-D Gaussian function with σ = 1 and has a size of s × s pixels. Gaussian difference [TeX:] $d_{\rm G}$ $d_{\mathrm{G}}$ is an index image to evaluate the difference between the local region in f _b and Gaussian function, calculated as

The ROI are the local bright regions in f _b with lower Gaussian difference value [d _G(i, j)]. They are extracted by a global threshold of Gaussian difference. Each ROI has a size of s × s pixels and is recorded by its center coordinates for facilitating to extract corresponding ROI from different source. The binary candidate region of each ROI is determined by thresholding, which minimizes the intraclass variance of candidate and surrounding [shown in Fig. 2(b)].

2.2.

Feature Extraction for Candidates

Because MAs are mainly characterized by their shape, size, and color, we use three types of features as follows:

Shape is one of the essential characteristics of the object, which provides meaningful information. The Shape feature can be divided into two categories: one is based on the boundary of the candidate and the other is based on the region. We use the relative size and compactness to describe the shape of candidates. Each ROI is a region of s×s pixels, where s depends on the size of the original image to ensure the length of s pixels is not <125 μm. The binary candidate of each ROI is obtained by thresholding the ROI, making the surrounding black and the candidate white. In the binary ROI, area A and perimeter P of the candidate is calculated. Relative size, s ₁, is defined as the ratio of area of candidates and ROI. Compactness, s ₂, is used to describe circularity of the shape of candidate, which is defined as

Texture is a significant property in digital images. It has an important role in human visual perception and offers information for recognition and interpretation. The GLCM is a powerful statistical tool that has proved its usefulness in a variety of image-analysis applications. It captures second-order gray-level information, which is mostly related to human perception and the discrimination of textures. It is common practice to utilize the 14 well-known Haralick's coefficients as the GLCM-based features.⁶ The coefficients are usually calculated from the average GLCM obtained by averaging the matrices calculated for 0-, 45-, 90-, and 135-deg directions. In this work, we first convert the gray scale of each ROI to 11, then calculate the average GLCM, [TeX:] $\bar C$ $\overline{C}$ . Last, we select four coefficients based on GLCM. They are energy t ₁, contrast t ₂, local homogeneity t ₃, and entropy t ₄.

Intensity is the only available information in the gray image, but color images provide more abundant color information, except intensity. The color contrast is a useful feature. It is defined as the sum of squared differences between the candidate region and its surroundings.⁴ We extract color features in RGB color space first, defined as

The selection of these color features is a complicated task due to the variety of color models. The RGB space of the original image is transformed to hue, saturation, and value space (HSV) because HSV color space is more appropriate since it allows the value component to be separated from the other two color components. Then we extract two kinds of color feature from the hue and saturation components as

In order to select the optimal feature vector, we construct several feature vectors, denoted by S, T, C1, C2, A1, and A2. Here, S = [s ₁,s ₂], T = [t ₁,t ₂,t ₃,t ₄], C1 = [c ₁,c ₂,c ₃], C2 = [c ₄,c ₅], A1 = {S,T,C1}, and A2 = {S, T,C2}. We test these feature vectors by SVM in Section 2.3.

2.3.

Validation of MA Based on SVM

The SVM, first introduced by Vapnik, is a learning algorithm for two-class classification. It is widely used in pattern recognition applications. It is based on strong foundations from the broad area of statistical learning theory according to structural risk minimization. A SVM is known for its good performance. The basic principle behind a binary SVM is to find the hyperplane that best separates vectors from both classes in feature space while maximizing the distance from each class to the hyperplane.⁷ There are both linear and nonlinear approaches to a SVM. If the two classes are linearly separable, then the SVM attempts to find the optimal separating hyperplane by maximizing the margin between both classes. The margin is [TeX:] $2/\!\left\| w \right\|^2$ $2/{‖w‖}^2$ . For a linear SVM, to find the optimal hyperplane is equal to min [TeX:] $\frac{1}{2}\left\| w \right\|^2$ $\frac{1}{2}{‖w‖}^2$ . When the two classes are nonlinearly separable, the SVM computes the optimal separating hyperplane by minimizing the following equation:

Different kernel functions determine the classification performance of the SVM. Thus, the selection of the kernel function is important to the SVM. We compare the performances of commonly used kernel functions based on the receiver operation characteristic (ROC) curve. The commonly used kernels are defined as

4.1.

Sample Sets

MAs can easily be confounded with other dark patterns. One of the major problems in detection of a MA is to establish a “gold standard,” namely, a set of annotated samples for learning and testing. In this paper, we used the 50 annotated retinal images from the Retinopathy Online Challenge database.⁹ The 50 images were from patients with diabetes without known diabetic retinopathy at the moment of photography; they represent a random sample of unique patients with “red lesions” from a large (>10,000 patients) diabetic retinopathy screening program. These images were taken with Topcon NW 100, NW 200, or Canon CR5-45NM “nonmydriatic” cameras at their native resolution and compression settings. The retinal specialist annotations were obtained from a combination of three ophthalmologists with retinal fellowship training. The first 20 images are for training, whereas the remaining 30 are for testing in this paper.

We extract all annotated MA regions as true positive samples and ∼30 spurious objects as true negative samples from each training image and construct a training sample set, size of 727. And we obtain a test sample set, size of 1112, in the same manner.

4.2.

Experimental Results

In order to identify the most favorable feature vector to distinguish real MAs from spurious objects, the feature vector shape features S, texture features T, and color features C1, C2, and the different combinations of them, A1 and A2, are respectively used as the input of the SVM. ROC curves for different feature vectors are shown in Fig. 3.

Fig. 3

ROC curves for different feature vectors.

The diagonal in Fig. 3 is the ROC of the SVM using C1 or C2 or S as feature vector, separately. They are laid over each other and have no diagnostic value. The ROC for T, A1, and A2 have higher AUC^ROC. Table 1 shows that the ROC for A2 has the highest AUC^ROC and the optimal classification performance among these feature vectors.

Table 1

Sensitivity and specificity of different feature vectors for SVM.

An SVM with different kernel functions has a significant difference of classification performance. In this paper, the commonly used kernel functions, such as linear function, Gaussian rbf, and polynomial kernel, are discussed. Figure 4 shows these ROC curves of SVM using these different kernel functions, and the best result is obtained by using the polynomial function kernel with p = 2. Table 2 shows that the ROCs for linear and polynomial functions have higher areas than the one for rbf, and the ROC for the polynomial function with p = 2 has the largest AUC^ROC value (0.7574), which is slightly larger than the one for the linear function. This indicates that the SVM using the polynomial function with p = 2 has the highest classification performance among the above-mentioned kernels.

Fig. 4

ROC curves for different kernels of SVM.

Table 2

Sensitivity and specificity of different kernels of SVM.

Different thresholds can bring different classification results. Thus, the threshold is very important for classification performance of the classifier. Here, sensitivity × specificity is used as a performance measure to determinate which threshold has the better diagnostic value.⁹ When threshold is set at −0.9, sensitivity × specificity reached its maximum, as shown in Fig. 5. Because the threshold of classification is −0.9, the sensitivity is 64% and the specificity is 80%.

Fig. 5

Threshold for SVM.