Translator Disclaimer
1 November 2011 Decision support system for the detection and grading of hard exudates from color fundus photographs
Author Affiliations +
Diabetic retinopathy is a major cause of blindness, and its earliest signs include damage to the blood vessels and the formation of lesions in the retina. Automated detection and grading of hard exudates from the color fundus image is a critical step in the automated screening system for diabetic retinopathy. We propose novel methods for the detection and grading of hard exudates and the main retinal structures. For exudate detection, a novel approach based on coarse-to-fine strategy and a new image-splitting method are proposed with overall sensitivity of 93.2% and positive predictive value of 83.7% at the pixel level. The average sensitivity of the blood vessel detection is 85%, and the success rate of fovea localization is 100%. For exudate grading, a polar fovea coordinate system is adopted in accordance with medical criteria. Because of its competitive performance and ability to deal efficiently with images of variable quality, the proposed technique offers promising and efficient performance as part of an automated screening system for diabetic retinopathy.



The most serious eye condition associated with diabetes affects, among other things, the retinal structures. This condition is called diabetic retinopathy (DR) and is the main cause of visual impairment and usually affects both eyes. Early detection of DR is essential for the prevention of visual loss. In some people with DR, retinal blood vessels may swell and leak fluid while in others, abnormal new blood vessels grow on the surface of the retina. Hard exudates (HEs), which are lipid leaks from blood vessels of abnormal retinas, are one of the most commonly occurring lesions in the early stages of DR.1, 2 Figure 1 shows a retinal fundus image with the main structures and HEs.

Fig. 1

Fundus image with the main structures of the retina and HEs.


Such lesions are normally detected and graded manually from retinal fundus images in intensive and time-consuming processes by clinicians. Computer-aided HE detection and grading from the retinal fundus images could facilitate immediate and accurate diagnosis. Thus, the main aim of this work is to develop a computer-aided system as a part of a medical screening scheme for evaluating the condition of the retina from the color fundus image.

Several researchers have proposed different techniques for HE detection, such as Osareh 3 who used fuzzy C-means for region segmentation followed by an artificial neural network technique to classify the segmented regions in terms of lesion-based classification. This method works well in Luv color space, but the accuracy in the case of nonuniform illumination is low. Walter 4 proposed an approach based on morphological reconstruction techniques and texture analysis to detect exudates. The results of this technique are encouraging, but the distinction between hard and soft exudates is not possible with this technique. Sánchez 5 proposed a method based on mixture models to threshold images in order to separate exudates from the background. A limitation of this method is that it sometimes misses faint exudates. Garcia 6 employed a combination of local and global thresholding for exudate segmentation and investigated three neural network classifiers to classify HEs, whereas Sopharak 7 employed naive Bayes and support vector machine classifiers for feature selection and exudate classification, but both classifiers occasionally fail to detect faint exudates. A method based on mathematical morphology for exudate detection was proposed by Welfer 8 with encouraging performance in terms of specificity but with low positive predictive value (PPV). Kose 9 proposed a relatively simple adaptive region growing method with background correction for bright-lesion detection and a background-based method for dark-lesion detection.

In this paper, we present an automated approach for the detection of retinal structures and HEs from color fundus images. Grading is achieved by first establishing a coordinate system centered at the fovea. This necessitates the prior detection of such fundus landmarks as the blood vessels, optic disk, and fovea. A morphological multiscale technique is proposed for the detection of the blood vessels, and the Hough transform is employed to localize the optic disk. The fovea is localized based on its features and geometric relation with the other retinal structures. For HE detection, a novel coarse-to-fine strategy is proposed.

This paper is organized as follows: image preprocessing, detection of retinal structures, and HE detection and grading are presented in Sec. 2. Section 3 introduces the databases used, experimental results, performance evaluation, and discussion. Conclusions are presented in Sec. 4.




Image Preprocessing

Like most camera-acquired images, retinal fundus images suffer from nonuniform illumination and variable visual contrast. Accordingly, preprocessing operations aim to prepare the image with better properties using shade correction and contrast enhancement. The RGB and HSI color spaces were compared, and the RGB space was found to be more suitable for HE detection. A color fundus image is provided as the input; it includes color information for each pixel in RGB color space. In this work, the image is initially resized to 640×480 pixels so that the proposed method could be applied to a variety of databases. The green channel component (G) is used for the detection of retinal structures and HEs because their contrast is greatest in this channel.

To correct for the nonuniform illumination of the image, a morphological top hat operation was used. This is based on producing a reasonable estimate of the background across G using morphological opening and closing operations with a large-enough structuring element to avoid entirely fitting within small candidate regions. To avoid unexpected bright areas at the borders and around the optic disk, alternating sequential opening and closing is used to calculate the approximation of the background E(G) as follows:4

Eq. 1

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} E(G) = \gamma ^{n\varsigma _1 } [ {\psi ^{n\varsigma _1 } ({\ldots\{ \gamma ^{1\varsigma _1 } [\psi ^{1\varsigma _1 } (G)]\} })}] ,\end{equation}\end{document} E(G)=γnς1[ψnς1(...{γ1ς1[ψ1ς1(G)]})],
where γ refers to opening operator, ψ refers to closing operator, ς1 refers to structuring element, and n is the number of repetitions. Based on many experimental tests, we found that selecting ς1 as a disk-shaped structuring element with fixed radius of 3 pixels and n = 8 can give good shade correction results. E(G) is then subtracted from G to produce a new image (G ) with a reasonably uniform background.

To enhance the contrast of image features, we employed a fuzzy enhancement algorithm, called minimum of fuzziness.10 The selection of this algorithm is based on computational efficiency expense. For clarity and immediate relevance, some aspects are summarized below. The gray levels of G are “fuzzified” by the membership function as follows:

Eq. 2

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} \mu = F(G^{\prime}) = \left[ 1 + \frac{G_{\max} ^{\prime} - G^{\prime} } {F_d}\right]^{ - F_{\rm e} } ,\end{equation}\end{document} μ=F(G)=1+GmaxGFdFe,
where [TeX:] $G_{\max} ^{\prime}$ Gmax is the maximum gray level value and parameters F e and F d denote the exponential and denominational fuzzifiers, respectively. F e is commonly taken up to 2 and F d is calculated with respect to the transition point of the membership function as follows:

Eq. 3

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} F_{\rm d} = \frac{{G' - X_{\rm c} }}{{2^{1/F_{\rm e} } - 1}} ,\end{equation}\end{document} Fd=GXc21/Fe1,
where X c is the crossover point, suitably selected from the image plane, where the brightness distribution is used as a measure of image quality. To modify the membership values, the intensification operator is applied to the membership function as follows:

Eq. 4

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} T_{\rm 1} (\mu) = \mu ' = \left\{ {\begin{array}{l@{\quad}l} {2[\mu]^2 } & {0 \le \mu \le 0.5} \\[3pt] {1 - 2[1 - \mu]^2 } & {0.5 \le \mu \le 1} \\ \end{array}} \right. .\end{equation}\end{document} T1(μ)=μ=2[μ]20μ0.512[1μ]20.5μ1.
Successive application of the nonlinear transformation (T r) is used to enhance the membership function as follows:

Eq. 5

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} T_r (\mu) = T_1 \{ T_{r - 1} (\mu)\} \,\,\,\,\,\,\,\,r = 1,2,\ldots. \end{equation}\end{document} Tr(μ)=T1{Tr1(μ)}r=1,2,....
The parameter r refers to the number of iterations and allows the user to set an appropriate level of enhancement. A new gray level, as a preprocessed image (G p), can be generated from the modified membership values using the inverse membership function as follows:

Eq. 6

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{eqnarray} {G}_ {p}= F^{-1}(\mu^{\prime}) = G_{\max} ^{ \prime} - F_ {d} \left({(\mu^{\prime})}^{\frac{-1}{F_{e}}} -1\right), \end{eqnarray}\end{document} Gp=F1(μ)=GmaxFd(μ)1Fe1,
Figure 2a shows the green channel component image, and Fig. 2b shows the image after the enhancement of luminosity and contrast between the background and image features.

Fig. 2

Preprocessing and retinal structure results: (a) Green channel image, (b) result of preprocessing, (c) initial blood vessel image, (d) blood vessel image, (e) initial boundary of the optic disk, (f) localization result of the optic disk, (g) region of interest for fovea localization, and (h) localization result of the fovea.



Blood Vessel Detection

Information provided by blood vessels can serve to localize the optic disk and fovea as well as provide vital diagnosis and follow-up information. Retinal blood vessels appear as dark line structures with different ranges of diameter, length, and orientation. Consequently, the multiscale approach is suitable to isolate features of blood vessels from the background.

Detection of linear structures using multiscale techniques have been attempted by some researchers, such as Gelman 11 Osareh and Shadgar,12 and Mendonca and Campilho.13 These systems work well on predefined image specifications, but they are not able to provide accurate outcomes when image quality is variable. Consequently, we propose a computationally fast method by applying a morphological closing operation to the preprocessed image, twice, with two different scales of disk-shaped structuring elements ς2 and ς3. On the basis of many experiments on different sizes and shapes of structural elements, we found (for the image resolution adopted) that the most suitable radii for the larger and smaller disk-shaped structure elements are 8 and 2 pixels, respectively. Then, the closed image with the smaller structuring element is subtracted from the larger one followed by thresholding to obtain an initial blood vessel image (B v), shown in Fig. 2c, as follows:

Eq. 7

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} B_{\rm v} = {\rm TH}_{\alpha_ {1}} \{ {\psi ^{\varsigma _2 } (G_{\rm p}) - \psi ^{\varsigma _3 } (G_{\rm p})}\}\, ,\end{equation}\end{document} Bv=THα1{ψς2(Gp)ψς3(Gp)},
where TH is a thresholding operator, α1 is a threshold value, and ψ is a closing operator. Empirically, it was found that the appropriate threshold is ∼90% of the maximal intensity. The idea behind this approach is that the dilation process expands bright regions and shrinks small dark regions, and the subsequent erosion operator will shrink the dilated bright regions to their original sizes, while the shrunk dark regions do not respond to the erosion operation. Thus, a subtraction process between the two resulting images will highlight dark regions, including the blood vessels.

Because of the variability of the condition and quality of images, the initial blood vessel image may include other types of dark regions. Discrimination of the true blood vessels from these other dark regions was achieved using a classification technique based on regional features, such as major axis length, minor axis length, aspect ratio (ratio of major axis length to minor axis length), area, perimeter, circularity, and eccentricity. A series of experiments on feature selection and vessel classification were carried out using a rule-based system, based on a number of quantities and logical rules. A rule-based system consists of if-then rules, which are used to formulate the conditional statements that comprise the complete knowledge base. In our work, these rules are empirically derived from the training data by a series of comparisons between many pairs of features from the feature vector and looking for functions of every two features. A number of such rules based on empirical constraint criteria were incorporated into the image-analysis and quantification program. These operations lead to an effective classifier. The final binary blood vessel image is shown in Fig. 2d, clearly showing a reduction in the number of false-positive blood vessel regions.


Optic Disk Localization

The location and size of the optic disk are important because they can aid in the location of the fovea, which itself is important for grading the severity of retinopathy. Moreover, the bright optic disk normally exhibits similar visual features to HEs, often resulting in misclassification. Thus, optic-disk localization is important to prevent it being detected as an exudate during grading.

Several researchers have attempted the localization of the optic disk, such as can be found in Li and Chutatate,14 who used a technique of principal component analysis, which has the advantage of the top-down strategy to extract the common characteristics among training images. Niemeijer 15 used a KNN regressor to predict the distance in pixels in the image to the object of interest at any given location based on a set of features measured at that location. Because it is easy to implement and computationally fast, the method described by Sekhar 16 was followed in this work. A circular region of interest is used to find the contour of the optic disk. The region of interest is computed using the magnitude gradient of the image by morphological closing and opening operations. The image is reconstructed using morphological reconstruction, and then the gradient is calculated by subtracting the eroded image from the dilated one. The boundary of the optic disk and its center are determined by applying the circular Hough transform to the gradient image as shown in Fig. 2e. The result of optic disk localization is illustrated in Fig. 2f.


Fovea Localization

Because the fovea contains the largest concentration of photoreceptor cells, it is the most specialized part of the retina. Determining the location of the fovea is therefore essential to grade the severity of lesions. In other words, the detection and diagnosis of lesions can provide a more precise and meaningful evaluation of risk when their spatial locations are described with reference to the location of the fovea. Several techniques have been proposed to localize the fovea using variable strategies, such as can be found in Sinthanayothin,17 who proposed a method to identify the fovea location using matching correlation together with characteristics typical of a fovea. In many retinal images, the fovea may be partially obscured by lesions or artifacts, or subject to nonuniform illumination or other distortions. Li and Chutatape18 and Welfer 19 attempt to exploit the known anatomy of the relative locations of the other retinal structures to localize the fovea center.

In this paper, the fovea is located by defining a candidate region of interest with reference to the established retinal landmarks, followed by a shape and intensity search. The two main retinal blood vessels (known as the arcade), together, can be approximated as a parabola, and in most retinal images, the fovea is located within this arcade. The position and shape of this parabola is estimated using a combination of the Hough transform and linear least-squares fitting.18

On the basis of the parabola information, the candidate region for the fovea is defined as a circle with a diameter of twice that of the optic disk (DD) along the main axis of fitted parabola and centered at a distance of 2DD from the vertex. Although the fovea is approximately equal in size to the optic disk,20 we select the region of interest four times as large to ensure that all fovea pixels are within this region. The threshold value is calculated within this region in such a way that the segmented area has the same area as the optic disk. In the case of obscured foveal features due to any of the aforementioned reasons, the method may fail in finding a suitable threshold value, in which case the fovea is approximated as a circle of diameter DD at the center of the candidate region, as illustrated in Fig. 2g. The result of fovea localization is shown in Fig. 2h.


Detection of Hard Exudates

Variations in contrast and brightness inside most retinal images make it difficult to distinguish HEs from other bright features in the image. Fortunately, most bright regions due to HEs are characterized by having distinct borders in different degrees, depending on the degree of retinopathy, while bright regions due to light reflection do not.4 Based on this, a novel two-step algorithm for exudate detection is proposed as follows:

  1. Coarse exudate segmentation to outline bright candidates with distinct borders

  2. Fine exudate segmentation to fine-tune the result of coarse exudate segmentation


Coarse hard exudate segmentation

To delimit HE candidates as regions of interest, the distinctness of their borders is exploited. To achieve this, the standard deviation around each pixel is calculated to get the local variation image. To avoid detecting the high contrast blood vessels and the optic disk as HEs, they must be eliminated before applying the local variation operator. To eliminate the optic disk, its location is masked with a color equal to the average background intensity, as shown in Fig. 3a.

Fig. 3

Steps of HE detection: (a) Preprocessed image after optic disk removal, (b) result of blood vessel removal, (c) result after applying local variation, thresholding and morphological operations, (d) result after classification, (e) combination of images from (a) and (d), and (f) final fine HE segmentation.


The blood vessels are eliminated by applying a morphological closing operator with a structuring element larger than the maximal width of the blood vessels so that all vessels get closed. Experimentation with different sizes and shapes of the structural element has shown that a disk (ς4) of radius 8 pixels is most suitable for images of the resolution adopted. The resulting image (G 1) is shown in Fig. 3b,

Eq. 8

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} G_{\rm 1} = \psi ^{\varsigma _4 } (G_{\rm p})\, .\end{equation}\end{document} G1=ψς4(Gp).
The local variation operator was applied on G 1 and the resulting image is denoted by G 2 as follows:

Eq. 9

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} G_{\rm 2} (x) = \frac{1}{{N - 1}}\sum\limits_{i \in w(x)} {\left[ {G_1 (i) - \mu (x)} \right]^2 } \, ,\end{equation}\end{document} G2(x)=1N1iw(x)G1(i)μ(x)2,
where x is a set of all pixels in a subwindow w(x) of N pixels and μ(x) is the mean value of G 1(i). The selection of window size is based on the necessary balance between the most informative performance measures, namely, the sensitivity and PPV, as discussed in Sec. 4. A window size of 9×9 was found most appropriate for the images studied here. In order to refine G 1 from artifacts and objects that have low local variation (soft exudates), an automatic Otsu thresholding was applied. Then, a morphological dilation operator with a disk-shaped structuring element (ς5) of radius 3 pixels was applied to the thresholded image to ensure that the majority of neighboring pixels are included in the candidate regions,

Eq. 10

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} G_{\rm 3} = D^{\varsigma _5 } [{\rm TH}_{\alpha _2 } (G_{\rm 2})],\end{equation}\end{document} G3=Dς5[THα2(G2)],
where TH is a thresholding operator with automatic level α2 and D is a dilation operator.

Figure 3c illustrates the result of the first phase of coarse segmentation, and although it has been successful at isolating the HEs, some artifacts remain due to residual contrasted vessels and bright regions around the retina and masked optic disk. Coarse HEs can be distinguished by many features, such as color, shape, size, and texture. We have tried to keep a reasonable and adequate number of features because misclassification probability and classifier complexity tend to increase with the number of features. A set of eight binary and gray-scale representation features are extracted for every candidate to be used as input to a rule-based classifier. The binary features are area (A), perimeter (P), circularity (C) (measure of roundness from perimeter and area), length (L), width (W), and aspect ration (R) (ratio of length to width). The gray-scale features are mean intensity (I m) and standard deviation (S td). Empirically, the rules listed in Table 1 were found most suitable for coarse HE discrimination. The result of coarse HE detection after applying the rule-based classifier is shown in Fig. 3d.

Table 1

Rules for the rule-based classifier to discriminate true coarse HEs. “rows” refers to the horizontal size of the image (in pixels), I avg is the average intensity of the whole image and σG is the standard deviation of the whole image.

FeaturesAC = 4πA/p2R = L/WLImStd


Fine hard exudate segmentation

Fine image segmentation aims to precisely extract or threshold objects from the previously delimited areas in the coarsely segmented image. In this stage, an algorithm based on region-based segmentation is applied only to the coarsely detected areas to fine-tune them. The idea behind using two stages of HE segmentation is to achieve a reasonable balance between the performance measures, namely, the sensitivity and PPV, and to trade off easily between them on the basis of medical requirements.

Algorithm motivation. A pure splitting algorithm is proposed here that achieves efficient region-based segmentation in a manner superior to pure merging and split-and-merge approaches. In the literature, a pure splitting algorithm is normally carried out by considering the entire image as an initial segmentation followed by successive splitting into quarters, and then a homogeneity test is performed to decide further splitting if the segment is not homogeneous enough. Although it is efficient and accurate, the pure splitting algorithm has a practical drawback, namely, the assumption that the image information is equally distributed throughout the image, and this may result in oversegmentation.21 To remedy this drawback, a novel pure splitting method based on successively pure splitting the image was proposed by Jaafar 22 In this paper, a newly developed technique is proposed on the basis of successive pure image splitting and introducing a novel procedure [referred to as partitioning regions of interest (PROI)]. This is based on assigning regions of interest inside the image in advance to be used later as constraints in the decision of splitting limit.

Proposed partitioning regions of interest procedure.

The coarse HE image G 3 is applied to the preprocessed image G p by means of a morphological AND operation, reducing all background pixels to zero. Regions of the other pixels in the new image (G 4) are called region-of-interest (ROI) zones. The new image is shown in Fig. 3e.

The PROI procedure is presented as follows:

  1. The image G 4 is viewed as the initial segmentation.

  2. The image G 4 is partitioned into four subregions called nodes.

  3. All nodes are tested to appoint two types of nodes: information nodes represented by resulting nodes, which contain element(s) from the ROI zone(s), and empty nodes represented by resulting nodes, which do not contain any element from ROI zone(s).

  4. Every information node is evaluated to determine whether or not it needs to be divided in accordance with predefined homogeneity criteria on the basis of standard deviation and mean intensity.

  5. Steps 3 and 4 are successively iterated only for the new nodes until achieving the predefined criteria (explained in the next paragraph) or a size threshold (ɛ).

The homogeneity test is an essential step to any region-based segmentation to avoid over or under segmentation. For this, the method described by Chen 23 was followed on the basis of feature analysis. In this method, histogram analysis is used for analyzing the characteristics of regions and mapping the frequencies of the desired features, such as gray-level distribution and local texture measures. If the histogram consists of a number of distinct modes, then the region is nonhomogeneous and needs to be segmented, and if the histogram is single mode, then the region is homogeneous. The threshold ɛ in the proposed PROI procedure is specified in such a way that the smallest node should not be less than 1/64 of the whole image area.


Thresholding is widely used to segment distinct modes in a histogram by determining a set of threshold values THk ∈ TH, k = 0,1, … , m – 1, where m is the number of distinct modes in the histogram. In our work, the image is divided into homogeneous subimages, thus being with the single mode in the histogram. Accordingly, a global thresholding can now be applied to each individual node to successfully segment the required objects. For this, a histogram-based thresholding was applied to all nodes separately. Let a partition Q of the image be defined as a subset of G 4 with respect to uniform illumination criterion. Hence running a histogram-based thresholding throughout all the nodes with automatic threshold value α3 will produce a new binary image (fine HE image) represented by G 5 as follows:

Eq. 11

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} G_{\rm 5} = \sum\limits_{l \in k} {{\rm TH}\alpha _3 (Q_l)} \, ,\end{equation}\end{document} G5=lkTHα3(Ql),
where k is the number of the nodes, l is the number of information nodes. Fine HE segmentation as a final detection result is shown in Fig. 3f.


Grading of Hard Exudates

In addition to size and number, a description of spatial locations of detected HEs can provide a more precise evaluation of clinical risk. Ophthalmologists usually use a polar coordinate system, centered either at the optic disk or the fovea, to estimate the grade of severity of HEs with a laborious and time-consuming process. To safely reduce the burden of manual grading, we adopt a foveal coordinate system (FCS) centered at the fovea, the center of vision, to assess the severity of HEs on the patient's vision. According to the brief description presented by Li and Chutatape18 and information provided by a medical information source,20 the retinal image is divided into ten fields. as illustrated in Fig. 4. The center of the fovea is used as the center of three circles of radii 0.33DD, 1DD, and 2DD. Four coordinates are used to divide each of the two bigger circles into four fields.

Fig. 4

Fovea coordinate system for a right eye.


After detection of HEs, their spatial locations are calculated throughout the fields of FCS to enable clinicians to assess the degree of harm to the vision, where this increases with HE proximity to the inner fields and center. The result of this stage is represented by a table of the ten fields with numbers of HE pixels in each field to be then evaluated by the clinician.


Results and Discussion



A database of color retinal images, known as DIARETDB0, has been made publicly available by Kauppi 24 This consists of 130 color images of which 110 contain different signs of DR according to the evaluation of clinical experts. These were captured at a 50-deg field of view and a resolution of 1500×1152 pixels. A set of ground truths for the different types of lesions, such as the microaneurysms, hemorrhages, and HEs that are reported by experts, are provided as a description of visual appearance of the DR findings with this database. Another database by Kauppi,25 known as the DIARETDB1 database, is also publicly available. This consists of 89 further images of which 38 contain HEs. These images have the same specifications of those in DIARETDB0. Images of HEs manually annotated by human experts are provided with this database, which can be used in testing the proposed method.

A set of 30 images, of which 17 images contain HEs, from the Messidor database,4 were also used in this paper. These images are captured at a 45-deg field of view with a size of 640×480 pixels. We have selected these 30 images among all images of the Messidor database because they are available with their ground truth. Thus, it is possible to use them for validation purposes.


Performance Calculation

The proposed methods for blood vessel detection, fovea localization, and HE detection were trained using a set of 130 images from the DIARETDB0. Satisfactory evaluation can be achieved when the testing images are from another source than the training source. Thus, a set of 119 images (89 from the DIARETDB1 database and 30 from the Messidor database) with their ground truth images were used to test the performance of the proposed method using two criteria: pixel-based calculation and image-based classification. In the pixel-based calculation, the performance is assessed based on pixel number of exudates correctly detected.26 While in the image-based classification, the algorithm is assessed based on its ability to classify an image without HEs as normal image or with HEs as a pathological image.26 In pixel-based calculation, four types of pixels are considered as follows: true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). These quantities were computed for each individual image to measure the following performance measures:

Eq. 12

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm Sensitivity} = \frac{{{\rm TP}}}{{{\rm TP} + {\rm FN}}} ,\end{equation}\end{document} Sensitivity=TPTP+FN,

Eq. 13

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm Specificity} = \frac{{{\rm TN}}}{{{\rm TN} + {\rm FP}}} ,\end{equation}\end{document} Specificity=TNTN+FP,

Eq. 14

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm Accuracy} = \frac{{{\rm TP} + {\rm TN}}}{{{\rm TP} + {\rm FP} + {\rm TN} + {\rm FN}}} ,\end{equation}\end{document} Accuracy=TP+TNTP+FP+TN+FN,

Eq. 15

[TeX:] \documentclass[12pt]{minimal}\begin{document}\begin{equation} {\rm Positive\; Preductive\; Value} = \frac{{{\rm TP}}}{{{\rm TP} + {\rm FP}}}.\end{equation}\end{document} PositivePreductiveValue=TPTP+FP.
In terms of pixel basis, we have achieved, for HE detection, an average sensitivity of 93.2%, specificity of 99.3%, accuracy of 99.4%, and PPV of 83.7%. To assess the ability of automated “HEs/no HEs” grading, the set of 119 images were used to evaluate the proposed method at image-based classification and the average sensitivity and specificity were 98.9 and 91%, respectively. In image-based classification, the sensitivity refers to the ratio of images correctly detected as abnormal from the whole abnormal images, while specificity refers to the ratio of images correctly detected as normal from the whole set of normal images. With respect to the detection of retinal structure, we achieved an average sensitivity of 85% and specificity of 90% for the detection of blood vessels and an average success rate of 100% for the fovea localization.


Influence of Parameters

In practical application, some error in the full automatic thresholding is inevitable. Hence, the disparity between the actual thresholding results and the ideal results (ground truth) can be used to enhance the performance of the algorithms. To do so, an intervention on some parameters used in our work has been tried to study their influence on the performance measures.

The number of true negatives that are correctly identified as non-HEs by both the grader and proposed method is the major number of image pixels; thus, the specificity and accuracy in pixel-based calculation are always near 99%. Hence, they are not very meaningful for the purposes of evaluation or comparison. Because the sensitivity and PPV are the most informative measures in the performance evaluation in the pixel-based calculation, these were used as the basis of assessment of parameter influence on performance. Several experiments have been carried out on parameters such as the window size w and the coarse and fine thresholds α2 and α3. Experimental results show that, as the window size is increased, PPV is increased but at the expense of the sensitivity, because the smaller HEs are more likely to be missed.

From these experimental results, we conclude that α2 and α3 have noticeable influence on the performance measures. Figure 5a shows the receiver operating characteristics (ROC) curves for the influence of each α2 and α3 separately on the overall measures. From these curves, it can be seen that, for the same rate of change, the threshold of the fine segmentation α3 has more positive and less negative influence on the sensitivity and PPV than those of the coarse segmentation α2. The reason for this is that a change in the coarse segmentation may significantly increase undesired false positives or decrease wanted true positives more than that in fine segmentation, and because variation of fine segmentation is limited within the delimited regions of interest (candidate regions). In other words, as the coarse segmentation delimits regions of interest, a lack in these regions can dramatically affect the number of true positives or an extra increase in these regions may cause an extra increase in false positives.

Fig. 5

ROC curves about influence of threshold parameters on performance: (a) Influence of the threshold α2 and α3, separately, and (b) influence of the threshold α2 and α3 together.


The influence of change in α2 and α3 together is also investigated, and the ROC performance curves are shown in Fig. 5b. The optimal percentage changes in the threshold α2 and/or α3 depend on the requested balance between the sensitivity and PPV, and the decision for that is up to clinicians and decided based on the diagnostic requirements.


Importance of Hard Exudate Grading

Grading of HEs is of great importance to help doctors in making their treatment decisions. To demonstrate the importance of the grading operation, in Fig. 6 we present two examples of binary HE detection results of our proposed method and the ground truth for two retinal images superimposed on the green channel of their corresponding images, where the FCS are overlaid.

Fig. 6

Comparison between two examples in terms of HE grading: (a,b) HE distribution for example 1 by the proposed method and a clinician, respectively, and (c,d) same as for (a) and (b) but with the images of example 2.


The severity of lesions in a retina depends not only on their size and number, but also on their spatial distribution throughout the fields of the FCS. Hence, the visual inspection and the pixel-based calculation, shown in Table 2 and illustrated in Figs. 6a, 6b for example 1 and Table 3, and Figs. 6c, 6d for example 2, indicate that, although the number and size of HEs in example 1 exceed those in example 2, those in example 2 are more harmful to vision and clinically in more urgent need of treatment because of their spatial distribution and proximity to the fovea.2, 20

Table 2

Distribution of HEs in example 1 on the fields of the FCS.

FCS fieldsInner fields (area in pixels)
Area fromCenterSuperiorNasalInferiorTemporal
FCS fieldsOuter fields (area in pixels)
Area fromFar TemporalSuperiorNasalInferiorTemporal

Table 3

Distribution of HEs in example 2 on the fields of the FCS.

FCS fieldsInner fields (area in pixels)
Area fromCenterSuperiorNasalInferiorTemporal
FCS fieldsOuter fields (area in pixels)
Area fromFar TemporalSuperiorNasalInferiorTemporal


Performance Evaluation

A comparison between meaningful performance measures, namely, the sensitivity and PPV (in pixel-based calculation) and sensitivity and specificity (in image-based classification) for the proposed and some related recent works is summarized in Table 4, which shows that the proposed method detects HEs with equal sensitivity in pixel-level calculation, and competitive sensitivity and specificity in image-based classification. The main drawback of the proposed method is that the PPV falls short of that reported by some researchers. The reason for this is that our method detects more false positives than the other methods used in comparison. In spite of this shortcoming in terms of PPV, a more meaningful comparison should include the full specifications of the proposed method, including the computational efficiency and the ability to deal with images of variable quality. Although detection of faint HEs is not urgent for the purpose of treatment, it is important for early tracking of patients; however, the proposed method can detect faint HEs with better performance than the other works cited as demonstrated by sensitivity performance numbers presented in Table 4. For more information about the proposed method's performance, five samples of green channel images, their proposed method results, and their ground truths are illustrated in Fig. 7.

Fig. 7

Visual comparison of proposed method results and their ground truths for five retinal images. (a–e) Five samples for the green channel component of the original images, their results produced by the proposed method, and the clinician hand-labeled images.


Table 4

Comparison of performance measures to the proposed method and previous related works for HE detection (SE = sensitivity, SP = specificity).

MethodTest setSE%PPV%SE%SP%
Osareh 367939588.9
Walter 41592.892.410086.7
Sanchez 58090.296.810090
Garcia 66787.683.510090
Sopharak 73992.353.1
Our method11993.283.798.991



This paper has presented a novel approach to the identification of retinal structures and the detection of HEs from color fundus images. A multiscale technique is presented for the detection of blood vessels, and a morphological approach combined with a circular Hough transform was used for localization of the optic disk. The fovea was localized using its own unique features in addition to its geometric relations with the other retinal structures. These are necessary steps for the HE detection and grading technique proposed.

In HE detection, a method based on coarse-to-fine segmentation is proposed where local variation, classification, and refining operations were used. In the fine HE segmentation, a new technique was developed using region-based segmentation. The contributions of the proposed method are attributed to the utilization of mapping and quantifying HEs with respect to the fovea, the center of vision. Moreover, the novel PROI-constraint image-splitting technique implemented in coarse-to-fine HE segmentation has enabled our method to detect HEs with competitive performance and superior computational efficiency compared to related works. Future work will address developments to this system by investigating other types of lesions including soft exudates to achieve a comprehensive study of bright lesions associated with DR.


The authors thank the Center of Mathematical Morphology, Mines Paris Tech,4 the University of Technology-Finland,24 and the University of Kuopio-Finland25 for their cooperation in providing retinal images. Hussain F. Jaafar acknowledges the financial support of the Iraqi government for this research.



D. E. Singer, D. M. Nathan, H. A. Fogel, and A. P. Schachat, “Screening for diabetic retinopathy,” Ann. Intern. Med., 116 (8), 660 –671 (1992). Google Scholar


“Facts about diabetic retinopathy,” (2011) Google Scholar


A. Osareh, M. Mirmehdi, B. Thomas, and R. Markham, “Automated identification of diabetic retinal exudates in digital colour images,” Br. J. Ophthalmol., 87 (10), 1220 –1223 (2003). Google Scholar


T. Walter, J. C. Klein, P. Massin, and A. Erginay, “A contribution of image processing to the diagnosis of diabetic retinopathy–detection of exudates in color fundus images of the human retina,” IEEE Trans. Med. Imaging, 21 (10), 1236 –1243 (2002). Google Scholar


C. I. Sanchez, M. Garcia, A. Mayo, M. I. Lopez, and R. Hornero, “Retinal image analysis based on mixture models to detect hard exudates,” Med. Image Anal., 13 (4), 650 –658 (2009). Google Scholar


M. Garcia, C. I. Sanchez, M. I. Lopez, D. Abasolo, and R. Hornero, “Neural network based detection of hard exudates in retinal images,” Comput. Methods Programs Biomed., 93 (1), 9 –19 (2009). Google Scholar


A. Sopharak, M. N. Dailey, B. Uyyanonvara, S. Barman, T. Williamson, K. T. Nwe, and Y. A. Moe, “Machine learning approach to automatic exudate detection in retinal images from diabetic retinopathy,” J. Mod. Opt., 57 (2), 1 –12 (2009). Google Scholar


D. Welfer, J. Scharcanski, and D. R. Marinho, “A coarse-to-fine strategy for automatically detecting exudates in color eye fundus images,” Comput Med Imaging Graph, 34 (3), 228 –235 (2009). Google Scholar


C. Kose, U. Sevik, C. Ikibas, and H. Erdol, “Simple methods for segmentation and measurement of diabetic retinopathy lesions in retinal fundus images,” Comput Methods Programs Biomed, (2011). Google Scholar


S. K. Pal and R. A. King, “Image enhancement using smoothing with fuzzy sets,” IEEE Trans. Syst., 11 (7), 494 –501 (1981). Google Scholar


R. Gelman, M. E. Martinez-Perez, D. K. Vanderveen, A. Moskowitz, and A. B. Fulton, “Diagnosis of plus disease in retinopathy of prematurity using retinal image multiscale analysis,” Invest. Ophthalmol. Vis. Sci., 46 (12), 4734 –4738 (2005). Google Scholar


A. Osareh and B. Shadgar, “An automated tracking approach for extraction of retinal vasculature in fundus images,” J. Ophthalm. Vis. Res., 5 (1), 20 –26 (2010). Google Scholar


A. M. Mendonca and A. Campilho, “Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction,” IEEE Trans. Med. Imaging, 25 (9), 1200 –1213 (2006). Google Scholar


H. Li and O. Chutatape, “Automatic detection and boundary estimation of the optic disk in retinal images using a model-based approach,” J. Electron. Imaging, 12 (1), 97 –105 (2003). Google Scholar


M. Niemeijer, M. D. Abramoff, and B. van Ginneken, “Fast detection of the optic disc and fovea in color fundus photographs,” Med. Image Anal., 13 (6), 859 –870 (2009). Google Scholar


S. Sekhar, W. Al-Nuaimy, A. Nandi, and F. Abdel-Sami, “Automated localization of retinal features,” Appl. Opt., 50 3064 –3075 (2011). Google Scholar


C. Sinthanayothin, J. F. Boyce, H. L. Cook, and T. H. Williamson, “Automated localisation of the optic disc, fovea, and retinal blood vessels from digital colour fundus images,” Br. J. Ophthalmol., 83 (8), 902 –910 (1999). Google Scholar


H. Li and O. Chutatape, “Automated feature extraction in color retinal images by a model based approach,” IEEE Trans. Biomed. Eng., 51 (2), 246 –254 (2004). Google Scholar


D. Welfer, J. Scharcanski, and D. R. Marinho, “Fovea center detection based on the retina anatomy and mathematical morphology,” Comput. Methods Programs Biomed., Google Scholar


Early Treatment Diabetic Retinopathy Study Research Group, “Grading diabetic retinopathy from stereoscopic color fundus photographs—an extension of the modified Airlie house classification,” Ophthalmology, 786 –806 (1991) Google Scholar


C.-H. Lee, “Recursive region splitting at hierarchical scope views,” Comput. Vis. Graphics Image Process., 33 (2), 237 –258 (1986). Google Scholar


H. F. Jaafar, A. K. Nandi, and W. Al-Nuaimy, “Detection of exudates in retinal images using a pure splitting technique,” 6745 –6748 (2010). Google Scholar


S.-Y. Chen, W.-C. Lin, and C.-T. Chen, “Split-and-merge image segmentation based on localized feature analysis and statistical tests,” CVGIP: Graphical Models Image Process., 53 (5), 457 –475 (1991). Google Scholar


T. Kauppi, V. Kalesnykiene, J. K. Kamarainen, L. Lensu, I. Sorri, J. Pietila, H. Kalviainen, and H. Unsitalo, “DIARETDB0: Evaluation database and morphology for diabetic retinopathy algorithm,” (2006) Google Scholar


T. Kauppi, V. Kalesnykiene, J. K. Kamarainen, L. Lensu, I. Sorri, A. Raninen, R. Voutilainen, H. Uusitalo, H. Kalviainen, and J. Pietila, “DIARETDB1: diabetic retinopathy database and evaluation protocol,” (2007) Google Scholar


A. Osareh, “Automated identification of diabetic retinal exudates and the optic disk,” Bristol, (2004). Google Scholar
© 2011 Society of Photo-Optical Instrumentation Engineers (SPIE) 1083-3668/2011/16(11)/116001/10/$25.00
Hussain F. Jaafar, Asoke K. Nandi, and Waleed Al-Nuaimy "Decision support system for the detection and grading of hard exudates from color fundus photographs," Journal of Biomedical Optics 16(11), 116001 (1 November 2011).
Published: 1 November 2011

Back to Top