Automated microaneurysm detection in diabetic retinopathy using curvelet transform

Abstract. Microaneurysms (MAs) are known to be the early signs of diabetic retinopathy (DR). An automated MA detection system based on curvelet transform is proposed for color fundus image analysis. Candidates of MA were extracted in two parallel steps. In step one, blood vessels were removed from preprocessed green band image and preliminary MA candidates were selected by local thresholding technique. In step two, based on statistical features, the image background was estimated. The results from the two steps allowed us to identify preliminary MA candidates which were also present in the image foreground. A collection set of features was fed to a rule-based classifier to divide the candidates into MAs and non-MAs. The proposed system was tested with Retinopathy Online Challenge database. The automated system detected 162 MAs out of 336, thus achieved a sensitivity of 48.21% with 65 false positives per image. Counting MA is a means to measure the progression of DR. Hence, the proposed system may be deployed to monitor the progression of DR at early stage in population studies.


Introduction
A majority of the people suffering from diabetes mellitus will eventually develop diabetic retinopathy (DR).At the final stage of DR, sufferers may lose their eyesight.DR is one of the leading causes of blindness and can be controlled if detected early.But for population-based assessment, the task of grading each retinal fundus image is both time consuming and labor intensive.With the advent of digital fundus photograph technology and the availability of fast computers, systems are being designed to detect DR automatically.Microaneurysms (MAs), which are basically the saccular enlargement of the venous ends of retinal capillaries, are said to be the first sign of DR.MAs appear and disappear during the early course of retinopathy. 1,2The MA count and turnover in digital color fundus images are important measures of DR progression. 3,4Therefore, accurately detecting MA is not only important for DR detection, but also it may assist in monitoring DR progression.
The MAs have diameters between 10 and 100 μm, are round in shape, and their color is similar to blood vessels (BVs) (red), 5,6 as shown in Fig. 1.Detection of MAs is challenging due to the variation in MA size, low and varying contrast, uneven illumination, and variation in fundus image background.MA detection is not a new topic; many researchers have worked on it since the early 1980s. 7,8Though the performance of the automatic system in digitized fluorescein angiograms is somewhat on par with a human grader, 9 it is considered to be an invasive method, as fluorescein sodium dye is injected into the eye.The associated risk of complication or adverse reaction can include transient nausea, occasional vomiting, and so on, and in very rare cases can cause death.Thus most of the current research is moving toward color fundus photography, which is a noninvasive imaging method.In the absence of contrast enhancing agent, color fundus images inevitably suffer from low contrast.The performance of the fundus image-based system is, as expected, limited and remains as an open issue in retinal image analysis.1][12][13][14][15][16] In this work, we propose to use curvelet transform (CT) for MA detection as it can detect curve singularities.Our preliminary work indicated promising results (Shah et al. 17 IOVS 2015;56: ARVO E-Abstract 5266).

Data Description
We tested our system using a publicly available Retinopathy Online Challenge (ROC) dataset. 18The dataset consists of 50 images with ground truth at different resolutions mimicking the real-world scenario.The MAs were annotated by four eye specialists at the Department of Ophthalmology, University of Iowa.The ROC dataset is challenging due to the presence of noise, compression artifacts, and the general image quality.These are in common to the image quality found in mass screening projects.The images were acquired using different types of camera and at different resolutions, which makes it more difficult to detect MAs in such images.The fundus cameras used were Topcon NW 100, a Topcon NW200, and a Canon CR5-45NM.Niemeijer selected them from 150,000 photographs collected in a DR screening program to form the ROC dataset. 19able 1 describes the different image types. 18

Candidate classification
Since the dataset consists of different sizes of images, the height of all the images was resized to a standard 800-pixel while maintaining their original width/height ratio.After resizing, each color fundus image was divided into red band (G r ), green band (G n ), and gray band (G y ).We preprocessed the G n by shade correction, followed by histogram equalization to obtain the preprocessed image denoted by G npr .The BVs were extracted from the G n , using two-dimensional (2-D) Gabor wavelet, 20 and removed from the G npr to obtain BV-removed image denoted by G −BV .The MA candidates were obtained in two parallel steps, namely local thresholding and statistical features-based technique.The MAs have low intensity in green band.Using local thresholding, 21 MA candidate pixels with low intensities could be selected from G −BV as potential candidates.Also from the gray band image (G y ), based on pixel contrast and standard deviation, we could identify MA candidates.Only those candidates which were found by both of these methods were retained.In both MA candidate   selections, thresholds were kept at low value to maximize the detection of MAs.Hence, many true MAs were detected along with hundreds of false positives (FPs).These FPs were mainly from the traces of the BVs, the background and bits of larger red lesions.

Candidate Feature Extraction
In general, MAs are red in color and have round shape.Based on their morphology and intensity, we used three feature sets to describe the MAs, namely color-based, Hessian matrix-based, and curvelet coefficients-based features.The color features include (a) standard deviation and mean intensity values in red and green bands, and (b) the histogram of S and V bands in HSV color space.The BVs and other linear objects can be detected based on the eigenvalues of Hessian matrix. 22The features we used based on Hessian matrix are thus the eigenvalues, their product, and their ratio.
The basic concept of curvelets 23 is to represent a curve as a superposition of multiple functions of various lengths and widths obeying the scaling law width ≈ length 2 .CT has a highly redundant dictionary, which can provide sparse representation of signals that have edges along regular curve.It is localized in angular orientation in addition to localization in spatial and frequency domains-a very important feature missing in the classic wavelet transform.Initial construction of curvelets has been redesigned and was reintroduced as fast digital CT. 24 Curvelets are used in many medical image analysis applications like computed tomography, 25 breast cancer diagnosis in digital mammogram, 26 ulcer detection, 27 retinal image analysis, 28 and so on.Most natural images/signals exhibit line-like edges, i.e., discontinuities across curves (so-called line or curve singularities).Traditional wavelets perform well only at representing point singularities, since they ignore the geometric properties of structures and do not exploit the regularity of edges. 29he solution to this problem and some other limitations of the wavelet was provided by CT. 23 Unlike the isotropic elements of wavelets, the needle-shaped elements of this transform possess very high directional sensitivity and anisotropy. 29The algorithm of CT is shown in Fig. 3.It is based on two windows, namely scale window V and radial window W, and consists of four steps: (a) compute the 2-D Fourier transform of the original image, and (b) for each scale s and orientation n, estimate frequency window U s;n as a product of the scale and radial windows, (c) wrap this product around the origin, and (d) compute a 2-D inverse fast Fourier transform to derive the curvelet coefficients.More details can be found in Candes' paper. 24In digital implementation of the CT, the two main parameters are a.number of scales and b. number of orientations at the coarsest level.We found in our case that 2 scales and 16 orientations work well.Based on curvelet coefficients, we calculated aspect ratio, circulatory, mean energy, and standard deviation of energy.

Candidate Classification
A simple rule-based classifier was designed to classify the candidates into MA and non-MA.The classification was done in three sequential stages.In stage one, using color features we removed the FPs.In stage two, using Hessian matrix-based features we removed those candidates that were from traces of BVs and other elongated objects.While in stage three, we utilized the curvelet coefficients-based features to remove noncircular objects.

Results and Discussion
Out of the 50 images, only 37 images contain MAs, while the remaining 13 images do not contain any MA.The total number of MAs in these 37 images is 336.The results of the proposed system and those previously reported in literature are shown in Table 2. Out of the 336 MAs, the proposed approach was able to detect 162 MAs, achieving a sensitivity of 48.21% with 65 FPs per image (FPPI).This result is favorably comparable with the  state-of-the-arts, although only simple rule-based classifier was implemented.Unlike the method by Adal et al. that employed four supervised classifiers and 87 features in total to select an optimum classifier-feature pairing to remove FPs, the computation of the proposed system is simpler and faster albeit at the expense of higher FPPI.In this work, we aimed to achieve high sensitivity in MA detection, i.e., to detect as many MAs as possible from fundus images.We used local thresholding technique in identifying MA candidates and kept the threshold to a low value as our approach to achieve maximum possible sensitivity, at the cost of hundreds of FPPI.These FPs at this initial stage were mainly due to background of the fundus images and BVs.We used statistical features to estimate the background and remove those candidates that are from background.In addition, during the BV extraction stage, we kept the threshold to a low value so that only true BVs are extracted.This helps us in detecting MAs near the BV and ultimately improving the sensitivity.However, this resulted in introducing many FPs as there were many traces of BVs.To eliminate those BVs, we used the Hessian matrix-based features.Thus, the proposed system can detect MA near the BVs with reasonably good specificity.Figure 4 illustrates the examples of MA detected in close vicinity to BVs.As previously mentioned, our objectives are different, in that we have targeted for better sensitivity whereas they focused on specificity.As the "good" MAs (clear circular dark red spots) form only a fraction of all the MAs in this dataset, this has a significant impact on how robust one approach can be optimized in coping with the variance of MA features.Using larger number of features and more complex classifiers, Adal et al. were successful in achieving very good specificity.Increasing its sensitivity (i.e., accepting MAs with larger variance) may however not be straightforward.
Figure 5 shows the free-response receiver operating characteristic (FROC) curve of the proposed system.Based on local thresholding alone, the proposed system would have hundreds of FPPI.More than 42% of those FPs could however be removed using color features, with only 2.3% of the MAs being lost.We started with 178 MAs detected out of possible 336 (with 250 FPPI), after local thresholding.The results were improved to 170 MAs with 144 FPPI, after MA candidate selection based on color features.However, to achieve very high specificity (<100 FPPI), additional features are required to differentiate the actual MAs and the FPs.In this case, we optimized the results using Hessian and curvelet features simultaneously.
To our best knowledge, CT is used for the first time in MA detection.CT is fast and robust at detecting objects with curved singularities.We used features including shape parameters based on curvelet coefficients to discriminate between MAs and non-MAs.The results indicate that curvelets are very effective at detecting round objects such as MAs.Table 2 further shows our approach achieves the higher sensitivity than other reported approaches.Admittedly, the FPPI is higher than Adal's but a simpler solution is perhaps better, as the Occam's razor principle suggests.
The ROC dataset is a very challenging dataset.It has been observed that general image quality, noise, and low and varying contrast make it difficult to detect MAs in this dataset.Figure 6 depicts some of the MA candidates detected by the proposed system.The FPs are mostly from BVs and hemorrhages (large red lesions).Some of the images are very dark, hence in those images the background has also contributed to the FPs.Factors such as variation in fundus image background, low and varying contrast, and artifact are found to further limit the MA detection rate.The cases where the proposed system failed to detect MAs are shown in Fig. 7 and can be summarized into three categories:  Case I: If the central pixel of the MA is not the darkest point, then in some cases that MA will be missed.The hit criteria specified by ROC 18 are very strict.We assumed that the center point of the MA to be the darkest point.But in some cases, the ground truth does not represent the darkest point.In such cases, if the darkest point of the detected lesion is not within the specified distance from the ground truth, then it is declared as missed.Case II: All those MAs which are very faint in color will be missed, as the candidates are extracted based on the assumption that MAs have low intensity in green band, i.e., dark in color as intensity level increases from dark/black (value ¼ 0) to faint/white (value ¼ 255).Thus, those MAs which are faint in color, i.e., having very high intensity value in green band, will be missed.These are the examples of MAs which were missed in the first stage.Most of the missed MAs belong to this category.Case III: Those MAs having abnormal shape will also be missed, since the proposed algorithm assumes MAs to be somehow round in shape.So all those MAs selected at the initial stage will be missed in the final stage of classification if they have noncircular shape.
One possible approach to improve the sensitivity and specificity of the proposed MA detection system is the utilization of multispectral imaging (MSI) approach.0][41][42] A recent study on retinal vein occlusion demonstrated that MSI was able to define vascular abnormalities at a comparable performance as fundus photography, fundus fluorescein angiography, and optical coherence tomography. 43n MSI, image data are captured at specific nonoverlapping frequency bands.Thus, certain features within the field of view can be highlighted.If applied in MA detection, we expect better BV extraction and background separation may be achieved.These will result in less stringent requirement at the classifier stage, and hence improvement in sensitivity.

Conclusion
We have explored a new technique for MA detection.We observed that the main sources/contributors of FPs in automated MA detection such as proposed one are image background and BVs while hemorrhages are the third category of FPs although with fewer in numbers.The proposed system has a high sensitivity and is able to detect MAs near the BVs.Our future work includes investigation of a means to detect hemorrhages and fine BVs to further improve the specificity of our proposed MA detection system.

Figure 2
Figure 2 depicts the complete procedure of the proposed MA detection system which consists of the following steps: 1. Candidate selection a. BV extraction from green band of raw image, preprocessing of green band, removal of BV from preprocessed green band, and MA candidate selection using local thresholding.b. Background estimation using gray band of raw image.

Fig. 1
Fig. 1 Example of MA, where row 1 shows the MA in (a) full-color RGB and (b) a close-up view; row 2 is (c) in green band and (d) with a close-up view.

Fig. 4
Fig. 4 Examples of MA that are very close to the BVs but detected by the proposed system: (a)-(d) fullcolor images and (e)-(h) in green band.

Fig. 6
Fig. 6 Objects encircled in black are true positives, encircled in white are false negatives, whereas objects inside the white squares are FPs: (a) full-color image while (b) in green band.

Fig. 7
Fig. 7 Different cases where the proposed system was unable to detect the MAs: (a)-(d) examples of missed MA because the center pixel does not have the minimum intensity, (e)-(h) examples of missed MA because their colors are very faint, and (i)-(l) examples of missed MA due to abnormal shape.

Table 1
Types of images used in the proposed approach.

Table 2
Result comparison of different MA detection methods.