Automatic detection and recognition of multiple macular lesions in retinal optical coherence tomography images with multi-instance multilabel learning

Leyuan Fang; Liumao Yang; Shutao Li; Hossein Rabbani; Zhimin Liu; Qinghua Peng; Xiangdong Chen

doi:10.1117/1.JBO.22.6.066014

23 June 2017 Automatic detection and recognition of multiple macular lesions in retinal optical coherence tomography images with multi-instance multilabel learning

Leyuan Fang, Liumao Yang, Shutao Li, Hossein Rabbani, Zhimin Liu, Qinghua Peng, Xiangdong Chen

Author Affiliations +

Journal of Biomedical Optics, Vol. 22, Issue 6, 066014 (June 2017). https://doi.org/10.1117/1.JBO.22.6.066014

Abstract

Detection and recognition of macular lesions in optical coherence tomography (OCT) are very important for retinal diseases diagnosis and treatment. As one kind of retinal disease (e.g., diabetic retinopathy) may contain multiple lesions (e.g., edema, exudates, and microaneurysms) and eye patients may suffer from multiple retinal diseases, multiple lesions often coexist within one retinal image. Therefore, one single-lesion-based detector may not support the diagnosis of clinical eye diseases. To address this issue, we propose a multi-instance multilabel-based lesions recognition (MIML-LR) method for the simultaneous detection and recognition of multiple lesions. The proposed MIML-LR method consists of the following steps: (1) segment the regions of interest (ROIs) for different lesions, (2) compute descriptive instances (features) for each lesion region, (3) construct multilabel detectors, and (4) recognize each ROI with the detectors. The proposed MIML-LR method was tested on 823 clinically labeled OCT images with normal macular and macular with three common lesions: epiretinal membrane, edema, and drusen. For each input OCT image, our MIML-LR method can automatically identify the number of lesions and assign the class labels, achieving the average accuracy of 88.72% for the cases with multiple lesions, which better assists macular disease diagnosis and treatment.

1. Introduction

Macular diseases are the leading cause of vision loss among adults and the elderly population.¹^,² The macula, located at the central area of the retina, is responsible for sharp and central vision, which lets us see objects that are straight ahead. If the macular tissues become abnormally damaged or disordered, the lesions will appear. The numbers and types of the lesions often determine the category and severity of eye diseases. For example, lesions, such as epiretinal membrane (ERM), edema, drusens, or cysts, are associated with various common diseases [e.g., diabetic retinopathy (DR) and age-related macular degeneration].³^–⁶ Therefore, detection and recognition of the retinal macular lesions are crucial for the clinical diagnosis and treatment of ophthalmopathy.⁷

As subtle retinal lesions may not be obvious on ophthalmoscopy, the retinal imaging technology has been rapidly developed during the past decades.⁸^,⁹ Optical coherence tomography (OCT) is a noninvasive imaging technique, which utilizes the infrared-light interferometry to image through the eye, aiming to see the structures beyond their surface. Recently, the advanced OCT technique can achieve a three-dimensional (3-D) tomography imaging of the eye with micrometer resolution, providing ophthalmologists the ability to visualize different types of lesions and thus assisting macular diagnosis and treatment.¹⁰^–¹² In the clinical diagnosis, ophthalmologists often need to manually identify the presence of various macular lesions at each cross section of the OCT volume. Such work is tedious, repetitive, and time consuming, which causes serious problems for the early detection and timely treatment of eye diseases. Therefore, development of automatic OCT image analysis techniques is an efficient way to provide supplemental information and support medical decisions for clinicians and ophthalmologists.¹³

During the past years, numerous effective methods for automatic recognition of macular lesions have been introduced.¹⁴^–²⁴ As thickness changes in the layers is an important indication of disease status, some works¹⁴^–¹⁹ first automatically segment the layers of the input OCT image and calculate the thickness between layers as a kind of extracted instance (or called the feature). Then, the calculated thickness is fed into a classifier (e.g., support vector machine (SVM), decision trees, or neural network) to determine the one single lesion from the test OCT image. In addition, other works²⁰^–²² attempt to extract the textures of the input image as a kind of instance. For example, Srinivasan et al.²⁰ utilize the histogram of gradients (HOG) to extract the local texture instance. Then, the extracted texture instance is also inputted into the SVM classifier to differentiate one lesion from another. Unlike the hand-crafted feature extraction methods (e.g., thickness or texture), very recent works²³^,²⁴ use an unsupervised learning algorithm to automatically extract the feature from the input OCT image, and then, the extracted feature is used for the lesion recognition with a classifier. Though the above methods can achieve a promising performance on the classification of each specific lesion, they can only extract one kind of instance from the whole test image and assign one lesion label for that test image.

In clinical diagnosis, multiple lesions usually coexist within one eye (see Fig. 1) as patients may simultaneously suffer from several types of eye diseases, and even one type of the disease may contain multiple lesions. For example, several types of lesions [e.g., macular edema (ME), neovascularization, exudates, vitreous, preretinal hemorrhage, and microaneurysms] may simultaneously exist in the DR [Fig. 1(c)], and identifying these lesions is essential for differentiating the stage of DR.²⁵ In this case, the desired lesion recognition method is expected to detect multiple lesions in the OCT image of the subject. However, the current lesion identification methods can recognize only one type of lesion from the whole image and thus have limitations for the diagnosis of eye disease with multiple lesions or subjects with complex diseases.

Fig. 1

Examples of (a) OCT image from a healthy subject, (b) OCT image from a subject with the ERM, MH, and drusen, and (c) OCT image from a DR subject with the ME and exudates.

To address the above issue, we propose a simple and practical framework called multi-instance multilabel-based lesions recognition (MIML-LR) for the simultaneous detection and recognition of multiple lesions. Motivated by works,²⁶^–²⁹ the proposed MIML-LR method can extract multiple lesion instances from the test OCT image and assign multiple class labels for these extracted instances. Specifically, according to the clinical prior information of the lesion, the MIML-LR first segments the input image into several regions of interests (ROIs), in which each corresponds to a type of lesion.³⁰^–³² Then, we utilize the speeded-up robust features (SURF)³³ and bag-of-visual-words (BoVW) descriptors³⁴^,³⁵ to extract multiple lesion instances from the segmented ROIs. Finally, the extracted lesion instances are forwarded into a multiple SVM classifier to determine the types of lesions in the test OCT image. In this paper, we applied the MIML-LR method to identify the presence of the normal macula and three common lesions, including ERM, edema, and drusen, in spectral-domain OCT (SD-OCT) scans and obtained a promising performance.

2. Methods

The proposed MIML-LR method is designed to detect three different lesions: ERM, edema, and drusen. To achieve this, the proposed MIML-LR method first segments the input OCT image into three ROIs, corresponding to different lesions. Then, we utilize the SURF and BoVW descriptors to extract the instances (features) from each region. After that, we learn the instance-level classifier by training the multiclass SVM on the extracted features. Finally, the trained classifier is used to determine the lesion labels of ROIs. More details of the proposed MIML-LR method are described in the following (flowchart is shown in Fig. 2).

Fig. 2

Flowchart of the MIML-LR system for multiple lesions identification.

2.1.

Regions of Interest Segmentation

Retinal lesion is the abnormal damage or change in the tissues of the retina. According to the clinical experience, lesions needed to be detected usually appear in some specific regions of the input OCT image. For example, ERM is a nonvascular fibrocellular proliferation that situates on the surface of inner limiting membrane (ILM)³⁶ [see Fig. 3(a)]. Edema is created when blood vessels in the retina leak fluids, and the leaked fluid results in the increase of retinal thickness (generally defined as the distance between ILM and retinal pigment epithelium (RPE) layer boundaries)³⁷ [see Fig. 3(b)]. Drusen is formed by the accumulation of extracellular deposits between the RPE basal lamina and the inner collagenous layer of Bruch’s membrane¹^,¹⁵ [see Fig. 3(c)]. Therefore, to detect the above three kinds of lesions, we need to segment the regions of those lesions. Specifically, as described in Chiu’s method,³⁰ ILM and RPE are the most hyperscattering layer boundaries in an OCT image, and we can simply find the two largest gradient values in each B-scan to estimate these two layer boundaries. As SD-OCT images of the retinal layers may be distorted due to scan artifacts, we further utilize the second-order polynomial fitting to smooth the estimated RPE boundaries and obtain an approximately flattened image. After that, we can generally set the ERM ROIs (20 pixels above and below the ILM boundary, respectively) and drusen ROIs (20 pixels above and below the drusen boundary, respectively). In addition, we can set the edema ROIs as the region between the ERM and drusen ROIs. Figure 4 shows the ROIs segmentation step in detail. Note that, before the ROIs segmentation, we also apply the sparsity-based block matching and 3-D filtering algorithm³⁸ to reduce the noise of the test OCT image.

Fig. 3

Examples of OCT B-scans with (a) ERM: a hyperreflective band adhere to or partly separated from the inner surface of the macula; (b) edema: retinal thickness increase with black blob-like fluid; (c) drusen: granular hyperreflective deposits situate between the RPE and Bruch’s membrane, causing the RPE to appear irregular, thinner, and disrupted; (d) combination of ERM and edema; (e) combination of ERM and drusen; and (f) combination of ERM, edema, and drusen.

Fig. 4

Illustration of the ROIs segmentation process.

2.2.

Feature Construction

After the segmentation of the three lesion ROIs, we further extract the features from these regions and use the features to achieve a more effective representation of the lesion in ROIs. Specifically, we adopt the SURF detector³³ and BoVW representation³⁴^,³⁵ for the feature extraction. The SURF detects and describes local features in the input image with 64-dimensional feature vector for each key point [see Fig. (5)]. As the total amount of local descriptors might be very large and will create very high computational cost, we further use the BoVW method to obtain a reduced representation of the local SURF detectors. The BoVW method clusters (e.g., $k$ -means³⁹) and quantizes local descriptors into visual words by vocabulary coding (e.g., vector quantization⁴⁰), which represents the local descriptors as a vector of occurrence counts of words. This can greatly decrease the amount of descriptors and enable the features to be more discriminative. In this way, each lesion region is represented by an effective feature vector.

Fig. 5

Illustration of SURF feature extraction in each region.

2.3.

Detector Construction and Regions of Interests Labeling

For multilesions recognition, each OCT image will be assigned with three lesion labels: ERM, edema, and drusen. Note that we have assumed that each instance is associated with only a single label in the previous section (e.g., region ERM is associated with label ERM). Therefore, predicting the class labels of each lesion can be achieved by separately applying the SVM algorithm⁴¹^,⁴² on the feature vectors from each lesion ROIs. The SVM constructs an optimal hyperplane between two classes by following the maximized margin criterion, and its decision function can be expressed as

Eq. (1)

f (x) = sign [\sum_{i}^{N} y_{i} α_{i} K (x, x_{i}) + b], \forall i = 1,2, \dots, N,

where

x_{i} \in R^{d}

is the training features from data space X,

y_{i} \in {+ 1, - 1}

is the label of

x_{i}

, and

α_{i}

and

b

represent the parameters of the hyperplane. The kernel function type might affect the performance of the SVM classifier. In this work, we adopt the Gaussian or radial basis function kernel, which is defined as

Eq. (2)

K (x, y) = \exp (- γ {‖ x - y ‖}^{2}) .

The functions map the samples from the input space to a higher dimensional space, in which the same samples are very easy to be separable.

The final labelset of the OCT image (object) is the aggregation of all region labels. If all three lesions were not identified, the B-scan is considered to be the normal one.

3. Results

3.1.

Dataset

The proposed MIML-LR method was trained and evaluated on the datasets acquired from 86 subjects, where 66 subjects had ocular diseases and 20 subjects had normal eyes. All SD-OCT volumes were obtained using the Spectralis OCT (Heidelberg Engineering, Heidelberg, Germany) at the First Affiliated Hospital of Hunan University of Chinese Medicine (HUCM), and each volume contains 31 B-scans of size $496 \times 768$ . The study complied with the Declaration of Helsinki, and informed consent was obtained from all participants.

As each B-scan from the OCT volume was acquired from different positions of the retina, it may contain different combinations of lesions. Therefore, instead of examining the lesions at the volume (human subject) level, we aim to detect the numbers and types of lesion for each OCT B-scan. In our experiments, two expert-graded ophthalmologists independently identified the presence or absence of lesions, including the ERM, edema, and drusen, in each B-scan, and B-scans with the same lesion labels from the two experts were collected for the training and testing. The numbers of B-scans with ocular disease and normal category are tabulated in Table 1. The final collected OCT image datasets consist of 823 labeled images, where 289 scans have ERM, 215 scans have edema, 182 scans have drusen, and 320 scans are from normal macular. In Table 2, we list the number of B-scans and subjects with different types of lesions. Here, for the category of subject, if at least one scan from the OCT volume of that subject has one kind of lesion, the subject is given the corresponding lesion label. As can be observed in Table 2, ERM often co-occurs with edema and drusen. Figure 3 shows some examples of OCT B-scans with different kinds of lesions in our dataset.

Table 1

Number of B-scans and subjects with normal eye and ocular diseases used in our experiments.

Statistics	Normal	Ocular diseases
B-scan	320	503
Subject	20	66

Table 2

Number of B-scans and subjects with different types of lesions.

Statistics	Single pathology only			Combination of pathologies
Statistics	ERM	Edema	Drusen	ERM+edema	ERM+drusen	ERM+edema+drusen
B-scan	118	77	137	126	33	12
Subject	23	18	17	25	8	4

3.2.

Experimental Setting

To evaluate the performance of the proposed MIML-LR lesion recognition algorithm, our experiment was performed in a K-fold cross-validation scheme at the subject level, which means that all B-scans from one subject are used as either the training set or the testing set. In the K-fold cross-validation scheme, the whole dataset at the subject level is randomly partitioned into K equal size subsets, where a single fold is used for the testing, whereas the remaining K-1 folds are used as the training set. K repeated runs are conducted to ensure each subset is tested exactly once, and the classification accuracy is then averaged over all K experiments to avoid any introduced bias.

Multiple LR performances are evaluated by calculating three commonly used multilabel evaluation metrics: subset accuracy, which is defined as the rate that the image of all four labels is classified correctly; recall, which is defined as the fraction of predicted correct labels of the actual labels; and precision, which is defined as the proportion of labels correctly classified of the predicted positive labels.⁴³ In our experiments, three independent SVM classifiers associated with lesions labels (ERM/non-ERM, edema/nonedema, and drusen/nondrusen) are applied to three image regions, respectively. Note that the detection of a normal or abnormal label depends on the result of three lesion detectors; if all three lesions were not identified, the B-scan is considered to be the normal one. The proposed algorithm has been implemented in the MATLAB environment on a desktop computer with Core i5-5200 CPU, 2.2 GHz, and 4 GB RAM.

3.3.

Multiple Lesions Recognition Performance

To achieve a more reliable assessment of the proposed method, the data splitting procedure was repeated five times, with the mean, standard deviation, and maximum value of 5 accuracies to be calculated. Note that differences in the sizes of image regions and tissue characteristics may require visual dictionaries of different sizes to achieve the best representation. Here, we set vocabulary size to 1500 for ERM detection, 2000 for edema, and 1500 for drusen, according to the sizes of corresponding ROIs, because a larger region (e.g., edema) has larger variations, thus requiring more words for better representation. The effect of different split value K is shown in Table 3. As can be seen, as the K value increases, which means more training samples are used, the overall performance will increase. Under the 10-fold cross validation, the proposed algorithm can deliver the best multilesions recognition performance with the average accuracy of 88.72%. As our method is based on three independent classifiers corresponding to the ERM, ME, and drusen, Table 4 also reports the results of detecting the three lesions and normal condition under the 10-fold cross validations. The performance is evaluated by the following metrics: accuracy, precision, sensitivity, and specificity. Accuracy is the fraction of predictions that are true. Precision is the fraction of predicted positives that are correct. Sensitivity measures the proportion of actual positives that are correctly identified. Specificity measures the proportion of actual negatives that are correctly identified. In our experiments, the average recognition time for each B-scan was 0.25 s. Note that as the classifier training can be an offline process, it does not need to be considered in the testing process.

Table 3

Multilesions recognition cross-validated results (mean±standard deviation).

K	Subset accuracy (%)	Recall (%)	Precision (%)
2	$83.48 \pm 1.48$	$88.51 \pm 0.81$	$89.12 \pm 1.16$
3	$84.42 \pm 1.36$	$90.23 \pm 1.09$	$89.48 \pm 0.92$
5	$86.36 \pm 1.11$	$89.14 \pm 1.48$	$92.22 \pm 1.19$
7	$86.84 \pm 1.02$	$90.26 \pm 1.31$	$93.54 \pm 0.93$
10	$88.72 \pm 0.84$	$91.21 \pm 0.53$	$92.83 \pm 0.74$

Table 4

Single-lesion recongnition cross-validated results (mean±standard deviation).

Class	Accuracy (%)	Precision (%)	Sensitivity (%)	Specificity (%)
ERM	$92.21 \pm 0.36$	$90.89 \pm 1.22$	$85.02 \pm 2.11$	$95.32 \pm 1.01$
Edema	$97.28 \pm 0.42$	$96.42 \pm 0.89$	$92.38 \pm 1.52$	$98.79 \pm 0.49$
Drusen	$97.83 \pm 0.24$	$96.51 \pm 0.82$	$90.32 \pm 0.87$	$99.02 \pm 0.22$
Normal	$96.74 \pm 0.58$	$94.21 \pm 1.22$	$96.17 \pm 0.92$	$96.61 \pm 0.88$

In addition, the MIML-LR algorithm is compared with the state-of-the-art single-lesion classification algorithm proposed by Srinivasan et al.²⁰ under B-scans with only one lesion. As this single-lesion detection method²⁰ can detect only the normal B-scans and B-scans with only drusen and edema, these single lesion or normal B-scans (534 B-scans) selected from our whole datasets are used for the training and testing. We also added a comparative experiment to verify the significance of the ROIs segmentation step in the proposed algorithm for single-lesion identification. Here, the SURF descriptor is applied on the whole image with a dense sampling, which is called the SURF-whole method. Table 5 shows the detection results under the 10-fold condition. As can be seen, our proposed method delivers higher accuracy than the single-lesion method and the SURF-whole method. This is because the relationship between lesion labels and retinal regions is considered (e.g., ERM can be seen only on the surface of the ILM layer) in the proposed MIML-LR method. That is, our clinical-experience-based adaptive ROIs segmentation algorithm combined with the feature extraction method, can better describe the morphological structures and capture more meaningful information of the lesion, thus enabling the detector to obtain a better recognition performance.

Table 5

Algorithm comparison.

Methods	Lesion types	Number	Feature	Mean accuracy (%)
Srinivasan et al.²⁰	Edema, drusen, and normal	534	HOG	88.28
SURF-whole	Edema, drusen, and normal	534	Dense sampling SURF	90.21
MIML-LR	Edema, drusen, and normal	534	ROIs segmentation SURF	95.62
MIML-LR	ERM, edema, drusen, and normal	823	ROIs segmentation SURF	88.72

4. Discussion and Conclusions

In this paper, we presented an MIML-LR method for the automatic detection and recognition of multiple lesions in the OCT B-scans. Unlike the previous single-lesion-based work that designs a single specific detector for the lesion, the proposed MIML-LR method extracts multiple instances and constructs multiple detectors for the lesions. The constructed multiple detectors can assign labels for the lesions to achieve the recognition of multiple lesions. Our experimental results on clinically acquired datasets with multiple lesions demonstrate the promising performance of the proposed method. In addition, even using the datasets with one single lesion, our proposed method still outperforms the state-of-the-art single-lesion recognition method. The proposed method can reduce the workload and duplication of ophthalmologists in clinical practice and better assist macular diagnosis and treatment.

Part of the correctly classified and misclassified cases of the proposed MIML-LR method is shown in Fig. 6. For better visual comparisons, the segmented ROIs for three lesions are covered with different colors, where the red band corresponds to ERM, green corresponds to edema, and yellow corresponds to drusen. As can be observed, shadowing effects of retinal blood vessels and the curvature of the retina [Fig. 6(d)] may have an adverse effect on the recognition performance. In addition, our algorithm also fails to detect some of the minor pathological changes like mild cases of ERMs [Fig. 6(e)] and microcystic ME [Fig. 6(f)]. Note that, some preprocessing methods (e.g., flattening or contrast adjustment) are expected to further solve the above problems.

Fig. 6

Example classification results. Correctly classified case: (a) classified as ERM and drusen, (b) classified as ERM and edema, and (c) classified as normal. Misclassified case: (d) misclassified as normal, the actual category is drusen; (e) misclassified as edema, the actual category is ERM and edema; and (f) misclassified as ERM, the actual category is ERM and edema.

This work is a step toward the realization of fully automated diagnosis of complex eye diseases. In our future work, we will construct a larger dataset with more types of lesions [e.g., choroidal neovascularization, retinal pigment epithelial dystrophy, and macular hole (MH)] to train and test our proposed method. In addition, our proposed method adopts only the typical feature extraction method (e.g., SURF) and classifier (e.g., SVM). Other advanced feature extraction methods and classifiers may also be incorporated into the proposed framework to further improve the performance.

Disclosures

The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.

Acknowledgments

This work was supported by the National Natural Science Fund of China for Distinguished Young Scholars under Grant No. 61325007, the National Natural Science Fund of China for International Cooperation and Exchanges under Grant No. 61520106001, and the National Natural Science Foundation for Young Scientist of China under Grant No. 61501180.

References

1.

L. S. Lim et al., “Age-related macular degeneration,” Lancet, 379 (9827), 1728 –1738 (2012). http://dx.doi.org/10.1016/S0140-6736(12)60282-7 LANCAO 0140-6736 Google Scholar

2.

N. Cheung et al., “Diabetic retinopathy,” Lancet, 376 (9735), 124 –136 (2010). http://dx.doi.org/10.1016/S0140-6736(09)62124-3 LANCAO 0140-6736 Google Scholar

3.

G. J. Jaffe and J. Caprioli, “Optical coherence tomography to detect and manage retinal disease and glaucoma,” Am. J. Ophthalmol., 137 (1), 156 –169 (2004). http://dx.doi.org/10.1016/S0002-9394(03)00792-X AJOPAA 0002-9394 Google Scholar

4.

E. K. Swingle et al., “Microcystic macular edema detection in retina OCT images,” Proc. SPIE, 9038 90380G (2014). http://dx.doi.org/10.1117/12.2043910 Google Scholar

5.

M. Usman Akram et al., “Detection and classification of retinal lesions for grading of diabetic retinopathy,” Comput. Biol. Med., 45 (2), 161 –171 (2014). http://dx.doi.org/10.1016/j.compbiomed.2013.11.014 CBMDAW 0010-4825 Google Scholar

6.

Z. Hu et al., “Semiautomated segmentation and analysis of retinal layers in three-dimensional spectral-domain optical coherence tomography images of patients with atrophic age-related macular degeneration,” Neurophotonics, 4 (1), 011012 (2017). http://dx.doi.org/10.1117/1.NPh.4.1.011012 Google Scholar

7.

B. K. Rutledge et al., “Optical coherence tomography of macular lesions associated with optic nerve head pits,” Ophthalmology, 103 (7), 1047 –1053 (1996). http://dx.doi.org/10.1016/S0161-6420(96)30568-X OPANEW 0743-751X Google Scholar

8.

Z. Zhang et al., “A survey on computer aided diagnosis for ocular diseases,” BMC Med. Inf. Decis. Making, 14 (12), 80 (2014). http://dx.doi.org/10.1186/1472-6947-14-80 Google Scholar

9.

M. Usman, M. M. Fraz and S. A. Barman, “Computer vision techniques applied for diagnostic analysis of retinal OCT images: a review,” Arch. Comput. Method Eng., 24 (3), 449 –465 (2017). http://dx.doi.org/10.1007/s11831-016-9174-3 Google Scholar

10.

J. F. de Boer et al., “Improved signal-to-noise ratio in spectral-domain compared with time-domain optical coherence tomography,” Opt. Lett., 28 (21), 2067 –2069 (2003). http://dx.doi.org/10.1364/OL.28.002067 OPLEDP 0146-9592 Google Scholar

11.

J. S. D. Mehreen Adhi, “Optical coherence tomography–current and future applications,” Curr. Opin. Ophthalmol., 24 (3), 213 –221 (2013). http://dx.doi.org/10.1097/ICU.0b013e32835f8bf8 COOTEF Google Scholar

12.

L. Fang et al., “3-D adaptive sparsity based image compression with applications to optical coherence tomography,” IEEE Trans. Med. Imaging, 34 (6), 1306 –1320 (2015). http://dx.doi.org/10.1109/TMI.2014.2387336 ITMID4 0278-0062 Google Scholar

13.

M. D. Abràmoff, M. K. Garvin and M. Sonka, “Retinal imaging and image analysis,” IEEE Rev. Biomed. Eng., 3 (12), 169 –208 (2010). http://dx.doi.org/10.1109/RBME.2010.2084567 Google Scholar

14.

Q. Chen et al., “Automated drusen segmentation and quantification in SD-OCT images,” Med. Image Anal., 17 (8), 1058 –1072 (2013). http://dx.doi.org/10.1016/j.media.2013.06.003 Google Scholar

15.

S. Farsiu et al., “Quantitative classification of eyes with and without intermediate age-related macular degeneration using optical coherence tomography,” Ophthalmology, 121 (1), 162 –172 (2014). http://dx.doi.org/10.1016/j.ophtha.2013.07.013 OPANEW 0743-751X Google Scholar

16.

S. Roychowdhury et al., “Automated localization of cysts in diabetic macular edema using optical coherence tomography images,” in Conf. Proc. IEEE Engineering Medicine and Biology Society, 1426 –1429 (2013). http://dx.doi.org/10.1109/EMBC.2013.6609778 Google Scholar

17.

T. Hassan et al., “Automated segmentation of subretinal layers for the detection of macular edema,” Appl. Opt., 55 (3), 454 –461 (2016). http://dx.doi.org/10.1364/AO.55.000454 APOPAI 0003-6935 Google Scholar

18.

A. Lang et al., “Automatic segmentation of microcystic macular edema in OCT,” Biomed. Opt. Express, 6 (1), 155 –169 (2015). http://dx.doi.org/10.1364/BOE.6.000155 BOEICL 2156-7085 Google Scholar

19.

B. Hassan et al., “Structure tensor based automated detection of macular edema and central serous retinopathy using optical coherence tomography images,” J. Opt. Soc. Am. A, 33 (4), 455 –463 (2016). http://dx.doi.org/10.1364/JOSAA.33.000455 JOAOD6 0740-3232 Google Scholar

20.

P. P. Srinivasan et al., “Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images,” Biomed. Opt. Express, 5 (10), 3568 –3577 (2014). http://dx.doi.org/10.1364/BOE.5.003568 BOEICL 2156-7085 Google Scholar

21.

K. Alsaih et al., “Classification of SD-OCT volumes with multi pyramids, LBP and HOG descriptors: application to DME detections,” in IEEE Engineering in Medicine and Biology, 1344 –1347 (2016). http://dx.doi.org/10.1109/EMBC.2016.7590956 Google Scholar

22.

W. Yu et al., “Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images,” Biomed. Opt. Express, 7 (12), 4928 (2016). http://dx.doi.org/10.1364/BOE.7.004928 BOEICL 2156-7085 Google Scholar

23.

A. F. G. Venhuizen et al., “Automated age-related macular degeneration classification in OCT using unsupervised feature learning,” Proc. SPIE, 9414 94141I (2015). http://dx.doi.org/10.1117/12.2081521 Google Scholar

24.

Y. Sun et al., “Fully automated macular pathology detection in retina optical coherence tomography images using sparse coding and dictionary learning,” J. Biomed. Opt., 22 (1), 016012 (2017). http://dx.doi.org/10.1117/1.JBO.22.1.016012 JBOPFO 1083-3668 Google Scholar

25.

C. P. Wilkinson et al., “Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales,” Ophthalmology, 110 (9), 1677 –1682 (2003). http://dx.doi.org/10.1016/S0161-6420(03)00475-5 OPANEW 0743-751X Google Scholar

26.

C. Mercan et al., “Multi-instance multi-label learning for whole slide breast histopathology,” Proc. SPIE, 9791 979108 (2016). http://dx.doi.org/10.1117/12.2216458 Google Scholar

27.

Z. Zheng-Jun et al., “Joint multi-label multi-instance learning for image classification,” in Proc. Computer Vision and Pattern Recognition (CVPR), 1 –8 (2008). http://dx.doi.org/10.1109/CVPR.2008.4587384 Google Scholar

28.

Z. H. Zhou et al., “Multi-instance multi-label learning,” Artif. Intell., 176 (1), 2291 –2320 (2012). http://dx.doi.org/10.1016/j.artint.2011.10.002 AINTBB 0004-3702 Google Scholar

29.

L. Fang and S. Li, “Face recognition by exploiting local gabor features with multitask adaptive sparse representation,” IEEE Trans. Instrum. Meas., 64 (10), 2605 –2615 (2015). http://dx.doi.org/10.1109/TIM.2015.2427893 IEIMAO 0018-9456 Google Scholar

30.

S. J. Chiu et al., “Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation,” Opt. Express, 18 (18), 19413 (2010). http://dx.doi.org/10.1364/OE.18.019413 OPEXFF 1094-4087 Google Scholar

31.

A. Almujaini et al., “Optical coherence tomography: clinical applications in medical practice,” Oman Med. J., 28 (2), 86 –91 (2013). http://dx.doi.org/10.5001/omj.2013.24 Google Scholar

32.

L. Fang et al., “Segmentation based sparse reconstruction of optical coherence tomography images,” IEEE Trans. Med. Imaging, 36 (2), 407 –421 (2017). http://dx.doi.org/10.1109/TMI.2016.2611503 ITMID4 0278-0062 Google Scholar

33.

H. Bay et al., “Speeded-up robust features (SURF),” Comput. Vision Image Understanding, 110 (3), 346 –359 (2008). http://dx.doi.org/10.1016/j.cviu.2007.09.014 CVIUF4 1077-3142 Google Scholar

34.

F. F. Li et al., “A bayesian hierarchical model for learning natural scene categories,” in Proc. Computer Vision and Pattern Recognition (CVPR), 524 –531 (2005). http://dx.doi.org/10.1109/CVPR.2005.16 Google Scholar

35.

G. Csurka et al., “Visual categorization with bags of keypoints,” in European Conf. on Computer Vision, 1 –22 (2004). Google Scholar

36.

D. H. Nam et al., “Intraoperative spectral domain optical coherence tomography imaging after internal limiting membrane peeling in idiopathic epiretinal membrane with connecting strands,” Retina, 35 (8), 1622 –1630 (2015). http://dx.doi.org/10.1097/IAE.0000000000000534 RETIDX 0275-004X Google Scholar

37.

M. R. Munk et al., “Differential diagnosis of macular edema of different pathophysiologic origins by spectral domain optical coherence tomography,” Retina, 34 (11), 2218 –2232 (2014). http://dx.doi.org/10.1097/IAE.0000000000000228 RETIDX 0275-004X Google Scholar

38.

K. Dabov et al., “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Trans. Image Process., 16 (8), 2080 –2095 (2007). http://dx.doi.org/10.1109/TIP.2007.901238 IIPRE4 1057-7149 Google Scholar

39.

T. Kanungo et al., “An efficient k-means clustering algorithm: analysis and implementation,” IEEE Trans. Pattern Anal. Mach. Intell., 24 (7), 881 –892 (2002). http://dx.doi.org/10.1109/TPAMI.2002.1017616 ITPIDJ 0162-8828 Google Scholar

40.

Y. L. Boureau et al., “Learning mid-level features for recognition,” in Proc. Computer Vision and Pattern Recognition (CVPR), 2559 –2566 (2010). http://dx.doi.org/10.1109/CVPR.2010.5539963 Google Scholar

41.

C. C. Chang et al., “LIBSVM: a library for support vector machines,” ACM Trans. Intell. Syst. Technol., 2 (3), 27 (2011). https://doi.org/10.1145/1961189.1961199 Google Scholar

42.

K. R. Müller et al., “An introduction to kernel-based learning algorithms,” IEEE Trans. Neural Network, 12 (2), 181 –201 (2001). http://dx.doi.org/10.1109/72.914517 ITNNEP 1045-9227 Google Scholar

43.

M. L. Zhang and Z. H. Zhou, “A review on multi-label learning algorithms,” IEEE Trans. Knowl. Data Eng., 26 (8), 1819 –1837 (2014). http://dx.doi.org/10.1109/TKDE.2013.39 ITKEEH 1041-4347 Google Scholar

Biographies for the authors are not available.

Citation Download Citation

Leyuan Fang, Liumao Yang, Shutao Li, Hossein Rabbani, Zhimin Liu, Qinghua Peng, and Xiangdong Chen "Automatic detection and recognition of multiple macular lesions in retinal optical coherence tomography images with multi-instance multilabel learning," Journal of Biomedical Optics 22(6), 066014 (23 June 2017). https://doi.org/10.1117/1.JBO.22.6.066014

Received: 18 April 2017; Accepted: 2 June 2017; Published: 23 June 2017

Access the abstract

JOURNAL ARTICLE
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 10 scholarly publications.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Optical coherence tomography

Sensors

Eye

1.

Introduction