This PDF file contains the front matter associated with SPIE Proceedings Volume 10134, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing.

We are developing a computerized system for detection of bladder cancer in CT urography. In this study, we used a deep-learning convolutional neural network (DL-CNN) to segment the bladder wall. This task is challenging due to differences in the wall between the contrast and non-contrast-filled regions, significant variations in appearance, size, and shape of the bladder among cases, overlap of the prostate with the bladder wall, and the wall being extremely thin compared to the overall size of the bladder. We trained a DL-CNN to estimate the likelihood that a given pixel would be inside the wall of the bladder using neighborhood information. A segmented bladder wall was then obtained using level sets with this likelihood map as a term in the level set energy formulation to obtain contours of the inner and outer bladder walls. The accuracy of the segmentation was evaluated by comparing the segmented wall outlines to hand outlines for a set of 79 training cases and 15 test cases using the average volume intersection % as the metric. For the training set, the inner wall achieved an average volume intersection of 90.0±8.7% and the outer wall achieved 93.7±3.9%. For the test set, the inner wall achieved an average volume intersection of 87.6±7.6% and the outer wall achieved 87.2±9.3%. The results show that the DL-CNN with level sets was effective in segmenting the inner and outer bladder walls.

We are developing a computer-aided detection system for bladder cancer in CT urography (CTU). We have previously developed methods for detection of bladder masses within the contrast-enhanced and the non-contrastenhanced regions of the bladder individually. In this study, we investigated methods for detection of bladder masses within the entire bladder. The bladder was segmented using our method that combined deep-learning convolutional neural network with level sets. The non-contrast-enhanced region was separated from the contrast-enhanced region with a maximum-intensity-projection-based method. The non-contrast region was smoothed and gray level threshold was applied to the contrast and non-contrast regions separately to extract the bladder wall and potential masses. The bladder wall was transformed into a straightened thickness profile, which was analyzed to identify lesion candidates in a prescreening step. The candidates were mapped back to the 3D CT volume and segmented using our auto-initialized cascaded level set (AI-CALS) segmentation method. Twenty-seven morphological features were extracted for each candidate. A data set of 57 patients with 71 biopsy-proven bladder lesions was used, which was split into independent training and test sets: 42 training cases with 52 lesions, and 15 test cases with 19 lesions. Using the training set, feature selection was performed and a linear discriminant (LDA) classifier was designed to merge the selected features for classification of bladder lesions and false positives. The trained classifier was evaluated with the test set. FROC analysis showed that the system achieved a sensitivity of 86.5% at 3.3 FPs/case for the training set, and 84.2% at 3.7 FPs/case for the test set.

We are developing a CAD system for bladder cancer treatment response assessment in CT. We compared the performance of the deep-learning convolution neural network (DL-CNN) using different network sizes, and with and without transfer learning using natural scene images or regions of interest (ROIs) inside and outside the bladder. The DL-CNN was trained to identify responders (T0 disease) and non-responders to chemotherapy. ROIs were extracted from segmented lesions in pre- and post-treatment scans of a patient and paired to generate hybrid pre-post-treatment paired ROIs. The 87 lesions from 82 patients generated 104 temporal lesion pairs and 6,700 pre-post-treatment paired ROIs. Two-fold cross-validation and receiver operating characteristic analysis were performed and the area under the curve (AUC) was calculated for the DL-CNN estimates. The AUCs for prediction of T0 disease after treatment were 0.77±0.08 and 0.75±0.08, respectively, for the two partitions using DL-CNN without transfer learning and a small network, and were 0.74±0.07 and 0.74±0.08 with a large network. The AUCs were 0.73±0.08 and 0.62±0.08 with transfer learning using a small network pre-trained with bladder ROIs. The AUC values were 0.77±0.08 and 0.73±0.07 using the large network pre-trained with the same bladder ROIs. With transfer learning using the large network pretrained with the Canadian Institute for Advanced Research (CIFAR-10) data set, the AUCs were 0.72±0.06 and 0.64±0.09, respectively, for the two partitions. None of the differences in the methods reached statistical significance. Our study demonstrated the feasibility of using DL-CNN for the estimation of treatment response in CT. Transfer learning did not improve the treatment response estimation. The DL-CNN performed better when transfer learning with bladder images was used instead of natural scene images.

Prostate cancer (PCa) is the second most common cause of cancer related deaths in men. Multiparametric MRI (mpMRI) is the most accurate imaging method for PCa detection; however, it requires the expertise of experienced radiologists leading to inconsistency across readers of varying experience. To increase inter-reader agreement and sensitivity, we developed a computer-aided detection (CAD) system that can automatically detect lesions on mpMRI that readers can use as a reference. We investigated a convolutional neural network based deep-learing (DCNN) architecture to find an improved solution for PCa detection on mpMRI. We adopted a network architecture from a state-of-the-art edge detector that takes an image as an input and produces an image probability map. Two-fold cross validation along with a receiver operating characteristic (ROC) analysis and free-response ROC (FROC) were used to determine our deep-learning based prostate-CAD’s (CADDL) performance. The efficacy was compared to an existing prostate CAD system that is based on hand-crafted features, which was evaluated on the same test-set. CADDL had an 86% detection rate at 20% false-positive rate while the top-down learning CAD had 80% detection rate at the same false-positive rate, which translated to 94% and 85% detection rate at 10 false-positives per patient on the FROC. A CNN based CAD is able to detect cancerous lesions on mpMRI of the prostate with results comparable to an existing prostate-CAD showing potential for further development.

Functional analysis of the placenta is important to analyze and understand its role in fetal growth and development. BOLD MR is a non-invasive technique that has been extensively used for functional analysis of the brain. During the last years, several studies have shown that this dynamic image modality is also useful to extract functional information of the placenta. We propose in this paper a method to track the placenta from a sequence of BOLD MR images acquired under normoxia and hyperoxia conditions with the goal of quantifying how the placenta adapts to oxygenation changes. The method is based on a spatiotemporal transformation model that ensures temporal coherence of the tracked structures. The method was initially applied to four patients with healthy pregnancies. An average MR signal increase of 16.96±8.39% during hyperoxia was observed. These automated results are in line with state-of-the-art reports using time-consuming manual segmentations subject to inter-observer errors.

The infiltrative lung diseases are a class of irreversible, non-neoplastic lung pathologies requiring regular follow-up with CT imaging. Quantifying the evolution of the patient status imposes the development of automated classification tools for lung texture. Traditionally, such classification relies on a two-dimensional analysis of axial CT images. This paper proposes a cascade of the existing CNN based CAD system, specifically tuned-up. The advantage of using a deep learning approach is a better regularization of the classification output. In a preliminary evaluation, the combined approach was tested on a 13 patient database of various lung pathologies, showing an increase of 10% in True Positive Rate (TPR) with respect to the best suited state of the art CNN for this task.

A fully-automated computer algorithm has been developed to identify early-stage Usual Interstitial Pneumonia (UIP) using features computed from low-dose CT scans. In each scan, the pre-segmented lung region is divided into N subsections (N = 1, 8, 27, 64) by separating the lung from anterior/posterior, left/right and superior/inferior in 3D space. Each subsection has approximately the same volume. In each subsection, a classic density measurement (fractional high-density volume h) is evaluated to characterize the disease severity in that subsection, resulting in a feature vector of length N for each lung. Features are then combined in two different ways: concatenation (2*N features) and taking the maximum in each of the two corresponding subsections in the two lungs (N features).

The algorithm was evaluated on a dataset consisting of 51 UIP and 56 normal cases, a combined feature vector was computed for each case and an SVM classifier (RBF kernel) was used to classify them into UIP or normal using ten-fold cross validation. A receiver operating characteristic (ROC) area under the curve (AUC) was used for evaluation. The highest AUC of 0.95 was achieved by using concatenated features and an N of 27. Using lung partition (N = 27, 64) with concatenated features had significantly better result over not using partitions (N = 1) (p-value < 0.05). Therefore this equal-volume partition fractional high-density volume method is useful in distinguishing early-stage UIP from normal cases.

Deep convolutional neural networks (CNNs) form the backbone of many state-of-the-art computer vision systems for classification and segmentation of 2D images. The same principles and architectures can be extended to three dimensions to obtain 3D CNNs that are suitable for volumetric data such as CT scans. In this work, we train a 3D CNN for automatic detection of pulmonary nodules in chest CT images using volumes of interest extracted from the LIDC dataset. We then convert the 3D CNN which has a fixed field of view to a 3D fully convolutional network (FCN) which can generate the score map for the entire volume efficiently in a single pass. Compared to the sliding window approach for applying a CNN across the entire input volume, the FCN leads to a nearly 800-fold speed-up, and thereby fast generation of output scores for a single case. This screening FCN is used to generate difficult negative examples that are used to train a new discriminant CNN. The overall system consists of the screening FCN for fast generation of candidate regions of interest, followed by the discrimination CNN.

Lung cancer is the leading cause of cancer deaths worldwide. Early diagnosis is critical in increasing the 5-year survival rate of lung cancer, so the efficient and accurate detection of lung nodules, potential precursors to lung cancer, is evermore important. In this paper, a computer-aided lung nodule detection system using convolution neural networks (CNN) and handcrafted features for false positive reduction is developed. The CNNs were trained with three types of images: lung CT images, their nodule-enhanced images, and their blood vessel-enhanced images. For each nodule candidate, nine 2D patches from differently oriented planes were extracted from each type of images. Patches of the same orientation from the same type of image across different candidates were used to train the CNNs independently, which were used to extract 864 features. 88 handcrafted features including intensity, shape, and texture features were also obtained from the lung CT images. The CNN features and handcrafted features were then combined to train a classifier, and a support vector machine was adopted to achieve the final classification results. The proposed method was evaluated on 1004 CT scans from the LIDC-IDRI database using 10-fold cross-validation. Compared with the traditional CNN method using only lung CT images, the proposed method boosted the sensitivity of nodule detection from 89.0% to 90.9% at 4 FPs/scan and from 71.6% to 78.2% at 1 FP/scan. This indicates that a combination of handcrafted features and CNN features from both lung CT images and enhanced images is a promising method for lung nodule detection.

For pediatric oncology patients, CT scans are performed to assess treatment response and disease progression. CAD may be used to detect lung nodules which would reflect metastatic disease. The purpose of this study was to investigate the effects of reducing radiation dose and varying slice thickness on CAD performance in the detection of solid lung nodules in pediatric patients. The dataset consisted of CT scans of 58 pediatric chest cases, from which 7 cases had lung nodules detected by radiologist, and a total of 28 nodules were marked. For each case, the original raw data (sinogram data) was collected and a noise addition model was used to simulate reduced-dose scans of 50%, 25% and 10% of the original dose. In addition, the original and reduced-dose raw data were reconstructed at slice thicknesses of 1.5 and 3 mm using a medium sharp (B45) kernel; the result was eight datasets (4 dose levels x 2 thicknesses) for each case An in-house CAD tool was applied on all reconstructed scans, and results were compared with the radiologist’s markings. Patient level mean sensitivities at 3mm thickness were 24%, 26%, 25%, 27%, and at 1.5 mm thickness were 23%, 29%, 35%, 36% for 10%, 25%, 50%, and 100% dose level, respectively. Mean FP numbers were 1.5, 0.9, 0.8, 0.7 at 3 mm and 11.4, 3.5, 2.8, 2.8 at 1.5 mm thickness for 10%, 25%, 50%, and 100% dose level respectively. CAD sensitivity did not change with dose level for 3mm thickness, but did change with dose for 1.5 mm. False Positives increased at low dose levels where noise values were high.

In this study, we propose a novel method of lung lesion detection in FDG-PET/CT volumes without labeling lesions. In our method, the probability distribution over normal standardized uptake values (SUVs) is estimated from the features extracted from the corresponding volume of interest (VOI) in the CT volume, which include gradient-based and texture-based features. To estimate the distribution, we use Gaussian process regression with an automatic relevance determination kernel, which provides the relevance of feature values to estimation. Our model was trained using FDG-PET/CT volumes of 121 normal cases. In the lesion detection phase, the actual SUV is judged as normal or abnormal by comparison with the estimated SUV distribution. According to the validation using 28 FDG-PET/CT volumes with 34 lung lesions, the sensitivity of the proposed method at 5.0 false positives per case was 81.9%.

Volumetric Laser Endomicroscopy (VLE) is a promising technique for the detection of early neoplasia in Barrett’s Esophagus (BE). VLE generates hundreds of high resolution, grayscale, cross-sectional images of the esophagus. However, at present, classifying these images is a time consuming and cumbersome effort performed by an expert using a clinical prediction model. This paper explores the feasibility of using computer vision techniques to accurately predict the presence of dysplastic tissue in VLE BE images. Our contribution is threefold. First, a benchmarking is performed for widely applied machine learning techniques and feature extraction methods. Second, three new features based on the clinical detection model are proposed, having superior classification accuracy and speed, compared to earlier work. Third, we evaluate automated parameter tuning by applying simple grid search and feature selection methods. The results are evaluated on a clinically validated dataset of 30 dysplastic and 30 non-dysplastic VLE images. Optimal classification accuracy is obtained by applying a support vector machine and using our modified Haralick features and optimal image cropping, obtaining an area under the receiver operating characteristic of 0.95 compared to the clinical prediction model at 0.81. Optimal execution time is achieved using a proposed mean and median feature, which is extracted at least factor 2.5 faster than alternative features with comparable performance.

We developed a novel electronic cleansing (EC) method for dual-energy CT colonography (DE-CTC) based on an ensemble deep convolution neural network (DCNN) and multi-spectral multi-slice image patches. In the method, an ensemble DCNN is used to classify each voxel of a DE-CTC image volume into five classes: luminal air, soft tissue, tagged fecal materials, and partial-volume boundaries between air and tagging and those between soft tissue and tagging. Each DCNN acts as a voxel classifier, where an input image patch centered at the voxel is generated as input to the DCNNs. An image patch has three channels that are mapped from a region-of-interest containing the image plane of the voxel and the two adjacent image planes. Six different types of spectral input image datasets were derived using two dual-energy CT images, two virtual monochromatic images, and two material images. An ensemble DCNN was constructed by use of a meta-classifier that combines the output of multiple DCNNs, each of which was trained with a different type of multi-spectral image patches. The electronically cleansed CTC images were calculated by removal of regions classified as other than soft tissue, followed by a colon surface reconstruction. For pilot evaluation, 359 volumes of interest (VOIs) representing sources of subtraction artifacts observed in current EC schemes were sampled from 30 clinical CTC cases. Preliminary results showed that the ensemble DCNN can yield high accuracy in labeling of the VOIs, indicating that deep learning of multi-spectral EC with multi-slice imaging could accurately remove residual fecal materials from CTC images without generating major EC artifacts.

Colorectal cancer (CRC) is one of the most common and deadliest forms of cancer, accounting for nearly 10% of all forms of cancer in the world. Even though colonoscopy is considered the most effective method for screening and diagnosis, the success of the procedure is highly dependent on the operator skills and level of hand-eye coordination. In this work, we propose to adapt fully convolution neural networks (FCN), to identify and segment polyps in colonoscopy images. We converted three established networks into a fully convolution architecture and fine-tuned their learned representations to the polyp segmentation task. We validate our framework on the 2015 MICCAI polyp detection challenge dataset, surpassing the state-of-the-art in automated polyp detection. Our method obtained high segmentation accuracy and a detection precision and recall of 73.61% and 86.31%, respectively.

Robust training of a deep convolutional neural network (DCNN) requires a very large number of annotated datasets that are currently not available in CT colonography (CTC). We previously demonstrated that deep transfer learning provides an effective approach for robust application of a DCNN in CTC. However, at high detection accuracy, the differentiation of small polyps from non-polyps was still challenging. In this study, we developed and evaluated a deep ensemble learning (DEL) scheme for reviewing of virtual endoluminal images to improve the performance of computer-aided detection (CADe) of polyps in CTC. Nine different types of image renderings were generated from virtual endoluminal images of polyp candidates detected by a conventional CADe system. Eleven DCNNs that represented three types of publically available pre-trained DCNN models were re-trained by transfer learning to identify polyps from the virtual endoluminal images. A DEL scheme that determines the final detected polyps by a review of the nine types of VE images was developed by combining the DCNNs using a random forest classifier as a meta-classifier. For evaluation, we sampled 154 CTC cases from a large CTC screening trial and divided the cases randomly into a training dataset and a test dataset. At 3.9 falsepositive (FP) detections per patient on average, the detection sensitivities of the conventional CADe system, the highestperforming single DCNN, and the DEL scheme were 81.3%, 90.7%, and 93.5%, respectively, for polyps ≥6 mm in size. For small polyps, the DEL scheme reduced the number of false positives by up to 83% over that of using a single DCNN alone. These preliminary results indicate that the DEL scheme provides an effective approach for improving the polyp detection performance of CADe in CTC, especially for small polyps.

Serrated polyps were previously believed to be benign lesions with no cancer potential. However, recent studies have revealed a novel molecular pathway where also serrated polyps can develop into colorectal cancer. CT colonography (CTC) can detect serrated polyps using the radiomic biomarker of contrast coating, but this requires expertise from the reader and current computer-aided detection (CADe) systems have not been designed to detect the contrast coating. The purpose of this study was to develop a novel CADe method that makes use of deep learning to detect serrated polyps based on their contrast-coating biomarker in CTC. In the method, volumetric shape-based features are used to detect polyp sites over soft-tissue and fecal-tagging surfaces of the colon. The detected sites are imaged using multi-angular 2D image patches. A deep convolutional neural network (DCNN) is used to review the image patches for the presence of polyps. The DCNN-based polyp-likelihood estimates are merged into an aggregate likelihood index where highest values indicate the presence of a polyp. For pilot evaluation, the proposed DCNN-CADe method was evaluated with a 10-fold cross-validation scheme using 101 colonoscopy-confirmed cases with 144 biopsy-confirmed serrated polyps from a CTC screening program, where the patients had been prepared for CTC with saline laxative and fecal tagging by barium and iodine-based diatrizoate. The average per-polyp sensitivity for serrated polyps ≥6 mm in size was 93±7% at 0:8±1:8 false positives per patient on average. The detection accuracy was substantially higher that of a conventional CADe system. Our results indicate that serrated polyps can be detected automatically at high accuracy in CTC.

Accurate identification of polyps is the ultimate goal of Computed Tomography Colonography (CTC). While oral contrast agents were originally used to tag stool and fluid for the ultimate goal of CTC, recently their effect on coating the surface of polyps has been observed. This study aims to evaluate (1) the frequency at which the oral contrast adhered to polyp surfaces and (2) if there was a difference in contrast adherence with respect to diverse polyp types. To eliminate gravity as a factor in this study, the polyps in contact with tagged fluid pools, particularly on the bottom of the colon wall were excluded. A total of 150 polyps were selected under the above condition from a CTC database and screened for any adherent contrast on the luminal edge. Among the total, 53% of the screened polyps had adherent contrast. Serrated adenomas and hyperplastic polyps had a higher tagging percentage, 77.80% and 62.50% respectively, than tubular adenomas and tubulovillous adenomas, 44.40% and 43% respectively. Other factors that were analyzed for the effect on coating include size and location of the polyps. The higher tagging percentage of serrated adenomas and hyperplastic polyps may be due to their similar cellular features. The average size of the polyps was 8.9 mm. When the polyps were separated by size into small (5-9mm) and large (10-26mm) groups, the large group had a higher tagging percentage. The polyp types were also classified by location with the major findings being: 1) Tubular adenomas were present in all segments of the colon and 2) that serrated adenomas were present at a higher percentage in the proximal colon. These findings shall facilitate characterizing tagging agents and improve computer aided detection and classification of polyps via CTC.

We propose a method for computer-assisted optical biopsy for colorectal polyps, with the final goal of assisting the medical expert during the colonoscopy. In particular, we target the problem of automatic classification of polyp images in two classes: adenomatous vs non-adenoma. Our approach is based on recent advancements in convolutional neural networks (CNN) for image representation. In the paper, we describe and compare four different methodologies to address the binary classification task: a baseline with classical features and a Random Forest classifier, two methods based on features obtained from a pre-trained network, and finally, the end-to-end training of a CNN. With the pre-trained network, we show the feasibility of transferring a feature extraction mechanism trained on millions of natural images, to the task of classifying adenomatous polyps. We then demonstrate further performance improvements when training the CNN for our specific classification task. In our study, 776 polyp images were acquired and histologically analyzed after polyp resection. We report a performance increase of the CNN-based approaches with respect to both, the conventional engineered features and to a state-of-the-art method based on videos and 3D shape features.

Cardiothoracic ratio (CTR) is a widely used radiographic index to assess heart size on chest X-rays (CXRs). Recent studies have suggested that also two-dimensional CTR might contain clinical information about the heart function. However, manual measurement of such indices is both subjective and time consuming. This study proposes a fast algorithm to automatically estimate CTR indices based on CXRs. The algorithm has three main steps: 1) model based lung segmentation, 2) estimation of heart boundaries from lung contours, and 3) computation of cardiothoracic indices from the estimated boundaries. We extended a previously employed lung detection algorithm to automatically estimate heart boundaries without using ground truth heart markings. We used two datasets: a publicly available dataset with 247 images as well as clinical dataset with 167 studies from Geisinger Health System. The models of lung fields are learned from both datasets. The lung regions in a given test image are estimated by registering the learned models to patient CXRs. Then, heart region is estimated by applying Harris operator on segmented lung fields to detect the corner points corresponding to the heart boundaries. The algorithm calculates three indices, CTR1D, CTR2D, and cardiothoracic area ratio (CTAR). The method was tested on 103 clinical CXRs and average error rates of 7.9%, 25.5%, and 26.4% (for CTR1D, CTR2D, and CTAR respectively) were achieved. The proposed method outperforms previous CTR estimation methods without using any heart templates. This method can have important clinical implications as it can provide fast and accurate estimate of cardiothoracic indices.

A fully automated computer algorithm has been developed to evaluate coronary artery calcification (CAC) from lowdose CT scans. CAC is identified and evaluated in three main coronary artery groups: Left Main and Left Anterior Descending Artery (LM + LAD) CAC, Left Circumflex Artery (LCX) CAC, and Right Coronary Artery (RCA) CAC. The artery labeling is achieved by segmenting all CAC candidates in the heart region and applying geometric constraints on the candidates using locally pre-identified anatomy regions. This algorithm was evaluated on 1,359 low-dose ungated CT scans, in which each artery CAC content was categorically visually scored by a radiologist into none, mild, moderate and extensive. The Spearman correlation coefficient R was used to assess the agreement between three automated CAC scores (Agatston-weighted, volume, and mass) and categorical visual scores. For Agatston-weighted automated scores, R was 0.87 for total CAC, 0.82 for LM + LAD CAC, 0.66 for LCX CAC and 0.72 for RCA CAC; results using volume and mass scores were similar. CAC detection sensitivities were: 0.87 for total, 0.82 for LM + LAD, 0.65 for LCX and 0.74 for RCA. To assess the impact of image noise, the dataset was further partitioned into three subsets based on heart region noise level (low<=80HU, medium=(80HU, 110HU], high>110HU). The low and medium noise subsets had higher sensitivities and correlations than the high noise subset. These results indicate that location specific heart risk assessment is possible from low-dose chest CT images.

Coronary artery calcification (CAC) is a typical marker of the coronary artery disease, which is one of the biggest causes of mortality in the U.S. This study evaluates the feasibility of using a deep convolutional neural network (DCNN) to automatically detect CAC in X-ray images. 1768 posteroanterior (PA) view chest X-Ray images from Sichuan Province Peoples Hospital, China were collected retrospectively. Each image is associated with a corresponding diagnostic report written by a trained radiologist (907 normal, 861 diagnosed with CAC). Onequarter of the images were randomly selected as test samples; the rest were used as training samples. DCNN models consisting of 2,4,6 and 8 convolutional layers were designed using blocks of pre-designed CNN layers. Each block was implemented in Theano with Graphics Processing Units (GPU). Human-in-the-loop learning was also performed on a subset of 165 images with framed arteries by trained physicians. The results from the DCNN models were compared to the diagnostic reports. The average diagnostic accuracies for models with 2,4,6,8 layers were 0.85, 0.87, 0.88, and 0.89 respectively. The areas under the curve (AUC) were 0.92, 0.95, 0.95, and 0.96. As the model grows deeper, the AUC or diagnostic accuracies did not have statistically significant changes. The results of this study indicate that DCNN models have promising potential in the field of intelligent medical image diagnosis practice.

Flow Doppler imaging is widely used by clinicians to detect diseases of the valves. In particular, continuous wave (CW) Doppler mode scan is routinely done during echocardiography and shows Doppler signal traces over multiple heart cycles. Traditionally, echocardiographers have manually traced such velocity envelopes to extract measurements such as decay time and pressure gradient which are then matched to normal and abnormal values based on clinical guidelines. In this paper, we present a fully automatic approach to deriving these measurements for aortic stenosis retrospectively from echocardiography videos. Comparison of our method with measurements made by echocardiographers shows large agreement as well as identification of new cases missed by echocardiographers.

Late Gadolinium-Enhanced Cardiac MRI (LGE CMRI) is an emerging non-invasive technique to image and quantify preablation native and post-ablation atrial scarring. Previous studies have reported that enhanced image intensities of the atrial scarring in the LGE CMRI inversely correlate with the left atrial endocardial voltage invasively obtained by electro-anatomical mapping. However, the reported reproducibility of using LGE CMRI to identify and quantify atrial scarring is variable. This may be due to two reasons: first, delineation of the left atrium (LA) and pulmonary veins (PVs) anatomy generally relies on manual operation that is highly subjective, and this could substantially affect the subsequent atrial scarring segmentation; second, simple intensity based image features may not be good enough to detect subtle changes in atrial scarring. In this study, we hypothesized that texture analysis can provide reliable image features for the LGE CMRI images subject to accurate and objective delineation of the heart anatomy based on a fully-automated whole heart segmentation (WHS) method. We tested the extracted texture features to differentiate between pre-ablation and post-ablation LGE CMRI studies in longstanding persistent atrial fibrillation patients. These patients often have extensive native scarring and differentiation from post-ablation scarring can be difficult. Quantification results showed that our method is capable of solving this classification task, and we can envisage further deployment of this texture analysis based method for other clinical problems using LGE CMRI.

Screening mammography is the most effective and low-cost method to date for early cancer detection. Mammographic breast density has been shown to be highly correlated with breast cancer risk. We are developing a radiomic model for BI-RADS density categorization on digital mammography (FFDM) with a supervised machine learning approach. With IRB approval, we retrospectively collected 478 FFDMs from 478 women. As a gold standard, breast density was assessed by an MQSA radiologist based on BI-RADS categories. The raw FFDMs were used for computerized density assessment. The raw FFDM first underwent log-transform to approximate the x-ray sensitometric response, followed by multiscale processing to enhance the fibroglandular densities and parenchymal patterns. Three ROIs were automatically identified based on the keypoint distribution, where the keypoints were obtained as the extrema in the image Gaussian scale-space. A total of 73 features, including intensity and texture features that describe the density and the parenchymal pattern, were extracted from each breast. Our BI-RADS density estimator was constructed by using a random forest classifier. We used a 10-fold cross validation resampling approach to estimate the errors. With the random forest classifier, computerized density categories for 412 of the 478 cases agree with radiologist’s assessment (weighted kappa = 0.93). The machine learning method with radiomic features as predictors demonstrated a high accuracy in classifying FFDMs into BI-RADS density categories. Further work is underway to improve our system performance as well as to perform an independent testing using a large unseen FFDM set.

We proposed the neutrosophic approach for segmenting breast lesions in breast Computer Tomography (bCT) images. The neutrosophic set (NS) considers the nature and properties of neutrality (or indeterminacy), which is neither true nor false. We considered the image noise as an indeterminate component, while treating the breast lesion and other breast areas as true and false components. We first transformed the image into the NS domain. Each voxel in the image can be described as its membership in True, Indeterminate, and False sets. Operations α-mean, β-enhancement, and γ-plateau iteratively smooth and contrast-enhance the image to reduce the noise level of the true set. Once the true image no longer changes, we applied one existing algorithm for bCT images, the RGI segmentation, on the resulting image to segment the breast lesions. We compared the segmentation performance of the proposed method (named as NS-RGI) to that of the regular RGI segmentation. We used a total of 122 breast lesions (44 benign, 78 malignant) of 123 non-contrasted bCT cases. We measured the segmentation performances of the NS-RGI and the RGI using the DICE coefficient. The average DICE value of the NS-RGI was 0.82 (STD: 0.09), while that of the RGI was 0.8 (STD: 0.12). The difference between the two DICE values was statistically significant (paired t test, p-value = 0.0007). We conducted a subsequent feature analysis on the resulting segmentations. The classifier performance for the NS-RGI (AUC = 0.8) improved over that of the RGI (AUC = 0.69, p-value = 0.006).

Breast cancer is the most common cancer diagnosed among US women and the second leading cause of cancer death 1 . Breast density is an independent risk factor for breast cancer and more than 25 states mandate its reporting to patients as part of the lay mammogram report ² . Recent publications have demonstrated that breast density measured from low-dose chest CT (LDCT) correlates well with that measured from mammograms and MRIs ^3-4 , thereby providing valuable information for many women who have undergone LDCT but not recent mammograms. A fully automated framework for breast density assessment from LDCT is presented in this paper. The whole breast region is first segmented using an anatomy-orientated novel approach based on the propagation of muscle fronts for separating the fibroglandular tissue from the underlying muscles. The fibroglandular tissue regions are then identified from the segmented whole breast and the percentage density is calculated based on the volume ratio of the fibroglandular tissue to the local whole breast region. The breast region segmentation framework was validated with 1270 LDCT scans, with 96.1% satisfactory outcomes based on visual inspection. The density assessment was evaluated by comparing with BI-RADS density grades established by an experienced radiologist in 100 randomly selected LDCT scans of female subjects. The continuous breast density measurement was shown to be consistent with the reference subjective grading, with the Spearman’s rank correlation 0.91 (p-value < 0.001). After converting the continuous density to categorical grades, the automated density assessment was congruous with the radiologist’s reading in 91% cases.

We assess the feasibility of a parenchymal texture feature fusion approach, utilizing a convolutional neural network (ConvNet) architecture, to benefit breast cancer risk assessment. Hypothesizing that by capturing sparse, subtle interactions between localized motifs present in two-dimensional texture feature maps derived from mammographic images, a multitude of texture feature descriptors can be optimally reduced to five meta-features capable of serving as a basis on which a linear classifier, such as logistic regression, can efficiently assess breast cancer risk. We combine this methodology with our previously validated lattice-based strategy for parenchymal texture analysis and we evaluate the feasibility of this approach in a case-control study with 424 digital mammograms. In a randomized split-sample setting, we optimize our framework in training/validation sets (N=300) and evaluate its descriminatory performance in an independent test set (N=124). The discriminatory capacity is assessed in terms of the the area under the curve (AUC) of the receiver operator characteristic (ROC). The resulting meta-features exhibited strong classification capability in the test dataset (AUC = 0.90), outperforming conventional, non-fused, texture analysis which previously resulted in an AUC=0.85 on the same case-control dataset. Our results suggest that informative interactions between localized motifs exist and can be extracted and summarized via a fairly simple ConvNet architecture.

Understanding the key radiogenomic associations for breast cancer between DCE-MRI and micro-RNA expressions is the foundation for the discovery of radiomic features as biomarkers for assessing tumor progression and prognosis. We conducted a study to analyze the radiogenomic associations for breast cancer using the TCGA-TCIA data set. The core idea that tumor etiology is a function of the behavior of miRNAs is used to build the regression models. The associations based on regression are analyzed for three study outcomes: diagnosis, prognosis, and treatment. The diagnosis group consists of miRNAs associated with clinicopathologic features of breast cancer and significant aberration of expression in breast cancer patients. The prognosis group consists of miRNAs which are closely associated with tumor suppression and regulation of cell proliferation and differentiation. The treatment group consists of miRNAs that contribute significantly to the regulation of metastasis thereby having the potential to be part of therapeutic mechanisms. As a first step, important miRNA expressions were identified and their ability to classify the clinical phenotypes based on the study outcomes was evaluated using the area under the ROC curve (AUC) as a figure-of-merit. The key mapping between the selected miRNAs and radiomic features were determined using least absolute shrinkage and selection operator (LASSO) regression analysis within a two-loop leave-one-out cross-validation strategy. These key associations indicated a number of radiomic features from DCE-MRI to be potential biomarkers for the three study outcomes.

DCE-MRI datasets have a temporal aspect to them, resulting in multiple regions of interest (ROIs) per subject, based on contrast time points. It is unclear how the different contrast time points vary in terms of usefulness for computer-aided diagnosis tasks in conjunction with deep learning methods. We thus sought to compare the different DCE-MRI contrast time points with regard to how well their extracted features predict response to neoadjuvant chemotherapy within a deep convolutional neural network. Our dataset consisted of 561 ROIs from 64 subjects. Each subject was categorized as a non-responder or responder, determined by recurrence-free survival. First, features were extracted from each ROI using a convolutional neural network (CNN) pre-trained on non-medical images. Linear discriminant analysis classifiers were then trained on varying subsets of these features, based on their contrast time points of origin. Leave-one-out cross validation (by subject) was used to assess performance in the task of estimating probability of response to therapy, with area under the ROC curve (AUC) as the metric. The classifier trained on features from strictly the pre-contrast time point performed the best, with an AUC of 0.85 (SD = 0.033). The remaining classifiers resulted in AUCs ranging from 0.71 (SD = 0.028) to 0.82 (SD = 0.027). Overall, we found the pre-contrast time point to be the most effective at predicting response to therapy and that including additional contrast time points moderately reduces variance.

Chromoendoscopy (CH) is a gastroenterology imaging modality that involves the staining of tissues with methylene blue, which reacts with the internal walls of the gastrointestinal tract, improving the visual contrast in mucosal surfaces and thus enhancing a doctor’s ability to screen precancerous lesions or early cancer. This technique helps identify areas that can be targeted for biopsy or treatment and in this work we will focus on gastric cancer detection. Gastric chromoendoscopy for cancer detection has several taxonomies available, one of which classifies CH images into three classes (normal, metaplasia, dysplasia) based on color, shape and regularity of pit patterns. Computer-assisted diagnosis is desirable to help us improve the reliability of the tissue classification and abnormalities detection. However, traditional computer vision methodologies, mainly segmentation, do not translate well to the specific visual characteristics of a gastroenterology imaging scenario. We propose the exploitation of a first unsupervised segmentation via superpixel, which groups pixels into perceptually meaningful atomic regions, used to replace the rigid structure of the pixel grid. For each superpixel, a set of features is extracted and then fed to a random forest based classifier, which computes a model used to predict the class of each superpixel. The average general accuracy of our model is 92.05% in the pixel domain (86.62% in the superpixel domain), while detection accuracies on the normal and abnormal class are respectively 85.71% and 95%. Eventually, the whole image class can be predicted image through a majority vote on each superpixel's predicted class.

Pancreatic cancer is the most lethal cancer with an overall 5-year survival rate of 7%¹ due to the late stage at diagnosis and the ineffectiveness of current therapeutic strategies. Given the poor prognosis, early detection at a pre-cancerous stage is the best tool for preventing this disease. Intraductal papillary mucinous neoplasms (IPMN), cystic tumors of the pancreas, represent the only radiographically identifiable precursor lesion of pancreatic cancer and are known to evolve stepwise from low-to-high-grade dysplasia before progressing into an invasive carcinoma. Observation is usually recommended for low-risk (low- and intermediate-grade dysplasia) patients, while high-risk (high-grade dysplasia and invasive carcinoma) patients undergo resection; hence, patient selection is critically important in the management of pancreatic cysts.² Radiologists use standard criteria such as main pancreatic duct size, cyst size, or presence of a solid enhancing component in the cyst to optimally select patients for surgery.³ However, these findings are subject to a radiologist’s interpretation and have been shown to be inconsistent with regards to the presence of a mural nodule or solid component.⁴ We propose objective classification of risk groups based on quantitative imaging features extracted from CT scans. We apply new features that represent the solid component (i.e. areas of high intensity) within the cyst and extract standard texture features. An adaptive boost classifier⁵ achieves the best performance with area under receiver operating characteristic curve (AUC) of 0.73 and accuracy of 77.3% for texture features. The random forest classifier achieves the best performance with AUC of 0.71 and accuracy of 70.8% with the solid component features.

Magnetic Resonance Elastography (MRE) is a phase-contrast MRI technique which calculates quantitative stiffness images, called elastograms, by imaging the propagation of acoustic waves in tissues. It is used clinically to diagnose liver fibrosis. Automated analysis of MRE is difficult as the corresponding MRI magnitude images (which contain anatomical information) are affected by intensity inhomogeneity, motion artifact, and poor tissue- and edge-contrast. Additionally, areas with low wave amplitude must be excluded. An automated algorithm has already been successfully developed and validated for clinical 2D MRE. 3D MRE acquires substantially more data and, due to accelerated acquisition, has exacerbated image artifacts. Also, the current 3D MRE processing does not yield a confidence map to indicate MRE wave quality and guide ROI selection, as is the case in 2D. In this study, extension of the 2D automated method, with a simple wave-amplitude metric, was developed and validated against an expert reader in a set of 57 patient exams with both 2D and 3D MRE. The stiffness discrepancy with the expert for 3D MRE was -0.8% ± 9.45% and was better than discrepancy with the same reader for 2D MRE (-3.2% ± 10.43%), and better than the inter-reader discrepancy observed in previous studies. There were no automated processing failures in this dataset. Thus, the automated liver elasticity calculation (ALEC) algorithm is able to calculate stiffness from 3D MRE data with minimal bias and good precision, while enabling stiffness measurements to be fully reproducible and to be easily performed on the large 3D MRE datasets.

Tumor volume measured from computed tomography images is considered a biomarker for disease progression or treatment response. The estimation of the tumor volume depends on the imaging system parameters selected, as well as lesion characteristics. In this study, we examined how different image reconstruction methods affect the measurement of lesions in an anthropomorphic liver phantom with a non-uniform background. Iterative statistics-based and model-based reconstructions, as well as filtered back-projection, were evaluated and compared in this study. Statistics-based and filtered back-projection yielded similar estimation performance, while model-based yielded higher precision but lower accuracy in the case of small lesions. Iterative reconstructions exhibited higher signal-to-noise ratio but slightly lower contrast of the lesion relative to the background. A better understanding of lesion volumetry performance as a function of acquisition parameters and lesion characteristics can lead to its incorporation as a routine sizing tool.

Hepatocellular carcinoma (HCC) is the most common liver cancer and the third leading cause of cancer-related death worldwide.¹ Resection or liver transplantation may be curative in patients with early-stage HCC but early recurrence is common.^{2, 3} Microvascular invasion (MVI) is one of the most important predictors of early recurrence.³ The identification of MVI prior to surgery would optimally select patients for potentially curative resection or liver transplant. However, MVI can only be diagnosed by microscopic assessment of the resected tumor. The aim of the present study is to apply CT-based texture analysis to identify pre-operative imaging predictors of MVI in patients with HCC. Texture features are derived from CT and analyzed individually as well as in combination, to evaluate their ability to predict MVI. A two-stage classification is employed: HCC tumors are automatically categorized into uniform or heterogenous groups followed by classification into the presence or absence of MVI. We achieve an area under the receiver operating characteristic curve (AUC) of 0.76 and accuracy of 76.7% for uniform lesions and AUC of 0.79 and accuracy of 74.06% for heterogeneous tumors. These results suggest that MVI can be accurately and objectively predicted from preoperative CT scans.

Knee Osteoarthritis (OA) is the most common form of arthritis, affecting millions of people around the world. The effects of the disease have been studied using the shape and texture features of bones in PosteriorAnterior (PA) and Lateral radiographs separately. In this work we compare the utility of features from each view, and evaluate whether combining features from both is advantageous. We built a fully automated system to independently locate landmark points in both radiographic images using Random Forest Constrained Local Models. We extracted discriminative features from the two bony outlines using Appearance Models. The features were used to train Random Forest classifiers to solve three specific tasks: (i) OA classification, distinguishing patients with structural signs of OA from the others; (ii) predicting future onset of the disease and (iii) predicting which patients with no current pain will have a positive pain score later in a follow-up visit. Using a subset of the MOST dataset we show that the PA view has more discriminative features to classify and predict OA, while the lateral view contains features that achieve better performance in predicting pain, and that combining the features from both views gives a small improvement in accuracy of the classification compared to the individual views.

The effectiveness of phase contrast X-ray computed tomography (PCI-CT) in visualizing human patellar cartilage matrix has been demonstrated due to its ability to capture soft tissue contrast on a micrometer resolution scale. Recent studies have shown that off-the-shelf Convolutional Neural Network (CNN) features learned from a nonmedical data set can be used for medical image classification. In this paper, we investigate the ability of features extracted from two different CNNs for characterizing chondrocyte patterns in the cartilage matrix. We obtained features from 842 regions of interest annotated on PCI-CT images of human patellar cartilage using CaffeNet and Inception-v3 Network, which were then used in a machine learning task involving support vector machines with radial basis function kernel to classify the ROIs as healthy or osteoarthritic. Classification performance was evaluated using the area (AUC) under the Receiver Operating Characteristic (ROC) curve. The best classification performance was observed with features from Inception-v3 network (AUC = 0.95), which outperforms features extracted from CaffeNet (AUC = 0.91). These results suggest that such characterization of chondrocyte patterns using features from internal layers of CNNs can be used to distinguish between healthy and osteoarthritic tissue with high accuracy.

Few studies have analyzed the microstructural properties of bone in cases of Osteogenenis Imperfecta (OI), or ‘brittle bone disease’. Current approaches mainly focus on bone mineral density measurements as an indirect indicator of bone strength and quality. It has been shown that bone strength would depend not only on composition but also structural organization. This study aims to characterize 3D structure of the cortical bone in high-resolution micro CT images. A total of 40 bone fragments from 28 subjects (13 with OI and 15 healthy controls) were imaged using micro tomography using a synchrotron light source (SRµCT). Minkowski functionals - volume, surface, curvature, and Euler characteristics - describing the topological organization of the bone were computed from the images. The features were used in a machine learning task to classify between healthy and OI bone. The best classification performance (mean AUC – 0.96) was achieved with a combined 4-dimensional feature of all Minkowski functionals. Individually, the best feature performance was seen using curvature (mean AUC - 0.85), which characterizes the edges within a binary object. These results show that quantitative analysis of cortical bone microstructure, in a computer-aided diagnostics framework, can be used to distinguish between healthy and OI bone with high accuracy.

Melanoma is considered a major health problem since it is the deadliest form of skin cancer. The early diagnosis through periodic screening with dermoscopic images can significantly improve the survival rate as well as reduce the treatment cost and consequent suffering of patients. Dermoscopy or skin surface microscopy provides in vivo inspection of color and morphologic structures of pigmented skin lesions (PSLs), rendering higher accuracy for detecting suspicious cases than it is possible via inspecting with naked eye. However, interpretation of dermoscopic images is time consuming and subjective, even for trained dermatologists. Therefore, there is currently a great interest in the development of computeraided diagnosis (CAD) systems for automated melanoma recognition. However, the majority of the CAD systems are still in the early development stage with lack of descriptive feature generation and benchmark evaluation in ground-truth datasets. This work is focusing on by addressing the various issues related to the development of such a CAD system with effective feature extraction from Non-Subsampled Contourlet Transform (NSCT) and Eig(Hess) histogram of oriented gradients (HOG) and lesion classification with efficient Extreme Learning Machine (ELM) due to its good generalization abilities and a high learning efficiency and evaluating its effectiveness in a benchmark data set of dermoscopic images towards the goal of realistic comparison and real clinical integration. The proposed research on melanoma recognition has huge potential for offering powerful services that would significantly benefit the present Biomedical Information Systems.

The abstract is not available

A convolutional network architecture is presented to determine bounding boxes around six organs in thoraxabdomen CT scans. A single network for each orthogonal view determines the presence of lungs, kidneys, spleen and liver. We show that an architecture that takes additional slices before and after the slice of interest as an additional input outperforms an architecture that processes single slices. From the slice-based analysis, a bounding box around the structures of interest can be computed. The system uses 6 convolutional, 4 pooling and one fully connected layer and uses 333 scans for training and 110 for validation. The test set contains 110 scans. The average Dice score of the proposed method was 0.95 and 0.95 for the lungs, 0.59 and 0.58 for the kidneys, 0.83 for the liver and 0.63 for the spleen. This paper shows that automatic localization of organs using multi-label convolution neural networks is possible. This architecture can likely be used to identify other organs of interest as well.

Image-derived features of breast parenchymal texture patterns have emerged as promising risk factors for breast cancer, paving the way towards personalized recommendations regarding women’s cancer risk evaluation and screening. The main steps to extract texture features of the breast parenchyma are the selection of regions of interest (ROIs) where texture analysis is performed, the texture feature calculation and the texture feature summarization in case of multiple ROIs. In this study, we incorporate breast anatomy in these three key steps by (a) introducing breast anatomical sampling for the definition of ROIs, (b) texture feature calculation aligned with the structure of the breast and (c) weighted texture feature summarization considering the spatial position and the underlying tissue composition of each ROI. We systematically optimize this novel framework for parenchymal tissue characterization in a case-control study with digital mammograms from 424 women. We also compare the proposed approach with a conventional methodology, not considering breast anatomy, recently shown to enhance the case-control discriminatory capacity of parenchymal texture analysis. The case-control classification performance is assessed using elastic-net regression with 5-fold cross validation, where the evaluation measure is the area under the curve (AUC) of the receiver operating characteristic. Upon optimization, the proposed breast-anatomy-driven approach demonstrated a promising case-control classification performance (AUC=0.87). In the same dataset, the performance of conventional texture characterization was found to be significantly lower (AUC=0.80, DeLong's test p-value<0.05). Our results suggest that breast anatomy may further leverage the associations of parenchymal texture features with breast cancer, and may therefore be a valuable addition in pipelines aiming to elucidate quantitative mammographic phenotypes of breast cancer risk.

The application of endovascular aortic aneurysm repair has expanded over the last decade. However, the long-term performance of stent grafts, in particular durable fixation and sealing to the aortic wall, remains the main concern of this treatment. The sealing and fixation are challenged at every heartbeat due to downward and radial pulsatile forces. Yet knowledge on cardiac-induced dynamics of implanted stent grafts is sparse, as it is not measured in routine clinical follow-up. Such knowledge is particularly relevant to perform fatigue tests, to predict failure in the individual patient and to improve stent graft designs. Using a physical dynamic stent graft model in an anthropomorphic phantom, we have evaluated the performance of our previously proposed segmentation and registration algorithm to detect periodic motion of stent grafts on ECG-gated (3D+t) CT data. Abdominal aortic motion profiles were simulated in two series of Gaussian based patterns with different amplitudes and frequencies. Experiments were performed on a 64-slice CT scanner with a helical scan protocol and retrospective gating. Motion patterns as estimated by our algorithm were compared to motion patterns obtained from optical camera recordings of the physical stent graft model in motion. Absolute errors of the patterns' amplitude were smaller than 0.28 mm. Even the motion pattern with an amplitude of 0.23 mm was measured, although the amplitude of motion was overestimated by the algorithm with 43%. We conclude that the algorithm performs well for measurement of stent graft motion in the mm and sub-mm range. This ultimately is expected to aid in patient-specific risk assessment and improving stent graft designs.

Hereditary hemorrhagic telangiectasia (HHT) is an autosomic dominant disorder, which is characterized by the development of multiple arterio-venous malformations in the skin, mucous membranes, and/or visceral organs. Pulmonary Arterio-Venous Malformation (PAVM) is an abnormal connection where feeding arteries shunt directly into draining veins with no intervening capillary bed. This condition may lead to paradoxical embolism and hemorrhagic complications. PAVMs patients should systematically be screened as the spontaneous complication rate is high, reaching almost 50%. Chest enhanced contrast CT scanner is the reference screening and follow-up examination. When performed by experienced operators as the prime treatment, percutaneous embolization of PAVMs is a safe, efficient and sustained therapy. The accuracy of PAVM detection and quantification of its progression over time is the key of embolotherapy success. In this paper, we propose an automatic method for PAVM detection and quantification relying on a modeling of vessel deformation, i.e. local caliber increase, based on mathematical morphology. The pulmonary field and vessels are first segmented using geodesic operators. The vessel caliber is estimated by means of a granulometric measure and the local caliber increase is detected by using a geodesic operator, the h-maxdomes. The detection sensitivity can be tuned up according to the choice of the h value which models the irregularity of the vessel caliber along its axis and the PAVM selection is performed according to a clinical criterion of >3 mm diameter of the feeding artery of the PAVM. The developed method was tested on a 20 patient dataset. A sensitivity study allowed choosing the irregularity parameter to maximize the true positive ratio reaching 85.4% in average. A specific false positive reduction procedure targeting the vessel trunks of the arterio-venous tree near mediastinum allows a precision increase from 13% to 67% with an average number of 1.15 false positives per scan.

Artery calcification is observed commonly in elderly patients, especially in patients with chronic kidney disease, and may affect coronary, carotid and peripheral arteries. Vascular calcification has been associated with many clinical outcomes. Manual identification of calcification in CT scans requires substantial expert interaction, which makes it time-consuming and infeasible for large-scale studies. Many works have been proposed for coronary artery calcification detection in cardiac CT scans. In these works, coronary artery extraction is commonly required for calcification detection. However, there are few works about abdominal or pelvic artery calcification detection. In this work, we present a method for automatic pelvic artery calcification detection on CT scan. This method uses the recent advanced faster region-based convolutional neural network (R-CNN) to directly identify artery calcification without a need for artery extraction since pelvic artery extraction itself is challenging. Our method first generates category-independent region proposals for each slice of the input CT scan using region proposal networks (RPN). Then, each region proposal is jointly classified and refined by softmax classifier and bounding box regressor. We applied the detection method to 500 images from 20 CT scans of patients for evaluation. The detection system achieved a 77.4% average precision and a 85% sensitivity at 1 false positive per image.

4D arterial spin labeling magnetic resonance angiography (4D ASL MRA) is a non-invasive and safe modality for cerebrovascular imaging procedures. It uses the patient’s magnetically labeled blood as intrinsic contrast agent, so that no external contrast media is required. It provides important 3D structure and blood flow information but a sufficient cerebrovascular segmentation is important since it can help clinicians to analyze and diagnose vascular diseases faster, and with higher confidence as compared to simple visual rating of raw ASL MRA images. This work presents a new method for automatic cerebrovascular segmentation in 4D ASL MRA images of the brain. In this process images are denoised, corresponding image label/control image pairs of the 4D ASL MRA sequences are subtracted, and temporal intensity averaging is used to generate a static representation of the vascular system. After that, sets of vessel and background seeds are extracted and provided as input for the image foresting transform algorithm to segment the vascular system. Four 4D ASL MRA datasets of the brain arteries of healthy subjects and corresponding time-of-flight (TOF) MRA images were available for this preliminary study. For evaluation of the segmentation results of the proposed method, the cerebrovascular system was automatically segmented in the high-resolution TOF MRA images using a validated algorithm and the segmentation results were registered to the 4D ASL datasets. Corresponding segmentation pairs were compared using the Dice similarity coefficient (DSC). On average, a DSC of 0.9025 was achieved, indicating that vessels can be extracted successfully from 4D ASL MRA datasets by the proposed segmentation method.

Neurosurgeons currently base most of their treatment decisions for intracranial aneurysms (IAs) on morphological measurements made manually from 2D angiographic images. These measurements tend to be inaccurate because 2D measurements cannot capture the complex geometry of IAs and because manual measurements are variable depending on the clinician’s experience and opinion. Incorrect morphological measurements may lead to inappropriate treatment strategies. In order to improve the accuracy and consistency of morphological analysis of IAs, we have developed an image-based computational tool, AView. In this study, we quantified the accuracy of computer-assisted adjuncts of AView for aneurysmal morphologic assessment by performing measurement on spheres of known size and anatomical IA models. AView has an average morphological error of 0.56% in size and 2.1% in volume measurement. We also investigate the clinical utility of this tool on a retrospective clinical dataset and compare size and neck diameter measurement between 2D manual and 3D computer-assisted measurement. The average error was 22% and 30% in the manual measurement of size and aneurysm neck diameter, respectively. Inaccuracies due to manual measurements could therefore lead to wrong treatment decisions in 44% and inappropriate treatment strategies in 33% of the IAs. Furthermore, computer-assisted analysis of IAs improves the consistency in measurement among clinicians by 62% in size and 82% in neck diameter measurement. We conclude that AView dramatically improves accuracy for morphological analysis. These results illustrate the necessity of a computer-assisted approach for the morphological analysis of IAs.

In this paper, we propose a novel supervoxel segmentation method designed for mediastinal lymph node by embedding Hessian-based feature extraction. Starting from a popular supervoxel segmentation method, SLIC, which computes supervoxels by minimising differences of intensity and distance, we overcome this method's limitation of merging neighboring regions with similar intensity by introducing Hessian-based feature analysis into the supervoxel formation. We call this structure-oriented voxel clustering, which allows more accurate division into distinct regions having blob-, line- or sheet-like structures. This way, different tissue types in chest CT volumes can be segmented individually, even if neighboring tissues have similar intensity or are of non- spherical extent. We demonstrate the performance of the Hessian-assisted supervoxel technique by applying it to mediastinal lymph node detection in 47 chest CT volumes, resulting in false positive reductions from lymph node candidate regions. 89 % of lymph nodes whose short axis is at least 10 mm could be detected with 5.9 false positives per case using our method, compared to our previous method having 83 % of detection rate with 6.4 false positives per case.

Recent breakthroughs in training deep neural network architectures, in particular deep Convolutional Neural Networks (CNNs), made a big impact on vision research and are increasingly responsible for advances in Computer Aided Diagnosis (CAD). Since many natural scenes and medical images vary in size and are too large to feed to the networks as a whole, two stage systems are typically employed, where in the first stage, small regions of interest in the image are located and presented to the network as training and test data. These systems allow us to harness accurate region based annotations, making the problem easier to learn. However, information is processed purely locally and context is not taken into account. In this paper, we present preliminary work on the employment of a Conditional Random Field (CRF) that is trained on top the CNN to model contextual interactions such as the presence of other suspicious regions, for mammography CAD. The model can easily be extended to incorporate other sources of information, such as symmetry, temporal change and various patient covariates and is general in the sense that it can have application in other CAD problems.

We investigated whether deep learning has potential to aid in the diagnosis of breast cancer when applied to mammograms and biologic tissue composition images derived from three-compartment (3CB) imaging. The dataset contained diagnostic mammograms and 3CB images (water, lipid, and protein content) of biopsy-sampled BIRADS 4 and 5 lesions in 195 patients. In 58 patients, the lesion manifested as a mass (13 malignant vs. 45 benign), in 87 as microcalcifications (19 vs. 68), and in 56 as (focal) asymmetry or architectural distortion (11 vs. 45). Six patients had both a mass and calcifications. For each mammogram and corresponding 3CB images, a 128x128 region of interest containing the lesion was selected by an expert radiologist and used directly as input to a deep learning method pretrained on a very large independent set of non-medical images. We used a nested leave-one-out-by-case (patient) model selection and classification protocol. The area under the ROC curve (AUC) for the task of distinguishing between benign and malignant lesions was used as performance metric. For the cases with mammographic masses, the AUC increased from 0.83 (mammograms alone) to 0.89 (mammograms+3CB, p=.162). For the microcalcification and asymmetry/architectural distortion cases the AUC increased from 0.84 to 0.91 (p=.116) and from 0.61 to 0.87 (p=.006), respectively. Our results indicate great potential for the application of deep learning methods in the diagnosis of breast cancer and additional knowledge of the biologic tissue composition appeared to improve performance, especially for lesions mammographically manifesting as asymmetries or architectural distortions.

Intuitive segmentation-based CADx/radiomic features, calculated from the lesion segmentations of dynamic contrast-enhanced magnetic resonance images (DCE-MRIs) have been utilized in the task of distinguishing between malignant and benign lesions. Additionally, transfer learning with pre-trained deep convolutional neural networks (CNNs) allows for an alternative method of radiomics extraction, where the features are derived directly from the image data. However, the comparison of computer-extracted segmentation-based and CNN features in MRI breast lesion characterization has not yet been conducted. In our study, we used a DCE-MRI database of 640 breast cases – 191 benign and 449 malignant. Thirty-eight segmentation-based features were extracted automatically using our quantitative radiomics workstation. Also, 2D ROIs were selected around each lesion on the DCE-MRIs and directly input into a pre-trained CNN AlexNet, yielding CNN features. Each method was investigated separately and in combination in terms of performance in the task of distinguishing between benign and malignant lesions. Area under the ROC curve (AUC) served as the figure of merit. Both methods yielded promising classification performance with round-robin cross-validated AUC values of 0.88 (se =0.01) and 0.76 (se=0.02) for segmentationbased and deep learning methods, respectively. Combination of the two methods enhanced the performance in malignancy assessment resulting in an AUC value of 0.91 (se=0.01), a statistically significant improvement over the performance of the CNN method alone.

Diagnostic interpretation of breast MRI studies requires meticulous work and a high level of expertise. Computerized algorithms can assist radiologists by automatically characterizing the detected lesions. Deep learning approaches have shown promising results in natural image classification, but their applicability to medical imaging is limited by the shortage of large annotated training sets. In this work, we address automatic classification of breast MRI lesions using two different deep learning approaches. We propose a novel image representation for dynamic contrast enhanced (DCE) breast MRI lesions, which combines the morphological and kinetics information in a single multi-channel image. We compare two classification approaches for discriminating between benign and malignant lesions: training a designated convolutional neural network and using a pre-trained deep network to extract features for a shallow classifier. The domain-specific trained network provided higher classification accuracy, compared to the pre-trained model, with an area under the ROC curve of 0.91 versus 0.81, and an accuracy of 0.83 versus 0.71. Similar accuracy was achieved in classifying benign lesions, malignant lesions, and normal tissue images. The trained network was able to improve accuracy by using the multi-channel image representation, and was more robust to reductions in the size of the training set. A small-size convolutional neural network can learn to accurately classify findings in medical images using only a few hundred images from a few dozen patients. With sufficient data augmentation, such a network can be trained to outperform a pre-trained out-of-domain classifier. Developing domain-specific deep-learning models for medical imaging can facilitate technological advancements in computer-aided diagnosis.

Predicting the risk of occult invasive disease in ductal carcinoma in situ (DCIS) is an important task to help address the overdiagnosis and overtreatment problems associated with breast cancer. In this work, we investigated the feasibility of using computer-extracted mammographic features to predict occult invasive disease in patients with biopsy proven DCIS. We proposed a computer-vision algorithm based approach to extract mammographic features from magnification views of full field digital mammography (FFDM) for patients with DCIS. After an expert breast radiologist provided a region of interest (ROI) mask for the DCIS lesion, the proposed approach is able to segment individual microcalcifications (MCs), detect the boundary of the MC cluster (MCC), and extract 113 mammographic features from MCs and MCC within the ROI. In this study, we extracted mammographic features from 99 patients with DCIS (74 pure DCIS; 25 DCIS plus invasive disease). The predictive power of the mammographic features was demonstrated through binary classifications between pure DCIS and DCIS with invasive disease using linear discriminant analysis (LDA). Before classification, the minimum redundancy Maximum Relevance (mRMR) feature selection method was first applied to choose subsets of useful features. The generalization performance was assessed using Leave-One-Out Cross-Validation and Receiver Operating Characteristic (ROC) curve analysis. Using the computer-extracted mammographic features, the proposed model was able to distinguish DCIS with invasive disease from pure DCIS, with an average classification performance of AUC = 0.61 ± 0.05. Overall, the proposed computer-extracted mammographic features are promising for predicting occult invasive disease in DCIS.

When humans identify objects in images, context is an important cue; a cheetah is more likely to be a domestic cat when a television set is recognised in the background. Similar principles apply to the analysis of medical images. The detection of diseases that manifest unilaterally in symmetrical organs or organ pairs can in part be facilitated by a search for symmetrical discrepancies in or between the organs in question. During a mammographic exam, images are recorded of each breast and absence of a certain structure around the same location in the contralateral image will render the area under scrutiny more suspicious and conversely, the presence of similar tissue less so. In this paper, we present a fusion scheme for a deep Convolutional Neural Network (CNN) architecture with the goal to optimally capture such asymmetries. The method is applied to the domain of mammography CAD, but can be relevant to other medical image analysis tasks where symmetry is important such as lung, prostate or brain images.

Retinal vessel caliber changes are associated with several major diseases, such as diabetes and hypertension. These caliber changes can be evaluated using eye fundus images. However, the clinical assessment is tiresome and prone to errors, motivating the development of automatic methods. An automatic method based on vessel crosssection intensity profile model fitting for the estimation of vessel caliber in retinal images is herein proposed. First, vessels are segmented from the image, vessel centerlines are detected and individual segments are extracted and smoothed. Intensity profiles are extracted perpendicularly to the vessel, and the profile lengths are determined. Then, model fitting is applied to the smoothed profiles. A novel parametric model (DoG-L7) is used, consisting on a Difference-of-Gaussians multiplied by a line which is able to describe profile asymmetry. Finally, the parameters of the best-fit model are used for determining the vessel width through regression using ensembles of bagged regression trees with random sampling of the predictors (random forests). The method is evaluated on the REVIEW public dataset. A precision close to the observers is achieved, outperforming other state-of-the-art methods. The method is robust and reliable for width estimation in images with pathologies and artifacts, with performance independent of the range of diameters.

The Arteriolar-to-Venular Ratio (AVR) is a popular dimensionless measure which allows the assessment of patients’ condition for the early diagnosis of different diseases, including hypertension and diabetic retinopathy. This paper presents two new approaches for AVR computation in retinal photographs which include a sequence of automated processing steps: vessel segmentation, caliber measurement, optic disc segmentation, artery/vein classification, region of interest delineation, and AVR calculation. Both approaches have been tested on the INSPIRE-AVR dataset, and compared with a ground-truth provided by two medical specialists. The obtained results demonstrate the reliability of the fully automatic approach which provides AVR ratios very similar to at least one of the observers. Furthermore, the semi-automatic approach, which includes the manual modification of the artery/vein classification if needed, allows to significantly reduce the error to a level below the human error.

Diabetic retinopathy (DR) is a major cause of irreversible vision loss. DR screening relies on retinal clinical signs (features). Opportunities for computer-aided DR feature detection have emerged with the development of Ultra-WideField (UWF) digital scanning laser technology. UWF imaging covers 82% greater retinal area (200°), against 45° in conventional cameras³ , allowing more clinically relevant retinopathy to be detected⁴ . UWF images also provide a high resolution of 3078 x 2702 pixels. Currently DR screening uses 7 overlapping conventional fundus images, and the UWF images provide similar results^1,4. However, in 40% of cases, more retinopathy was found outside the 7-field ETDRS) fields by UWF and in 10% of cases, retinopathy was reclassified as more severe⁴ . This is because UWF imaging allows examination of both the central retina and more peripheral regions, with the latter implicated in DR⁶ . We have developed an algorithm for automatic recognition of DR features, including bright (cotton wool spots and exudates) and dark lesions (microaneurysms and blot, dot and flame haemorrhages) in UWF images. The algorithm extracts features from grayscale (green “red-free” laser light) and colour-composite UWF images, including intensity, Histogram-of-Gradient and Local binary patterns. Pixel-based classification is performed with three different classifiers. The main contribution is the automatic detection of DR features in the peripheral retina. The method is evaluated by leave-one-out cross-validation on 25 UWF retinal images with 167 bright lesions, and 61 other images with 1089 dark lesions. The SVM classifier performs best with AUC of 94.4% / 95.31% for bright / dark lesions.

Automated detection and quantification of spatio-temporal retinal changes is an important step to objectively assess disease progression and treatment effects for dynamic retinal diseases such as diabetic retinopathy (DR). However, detecting retinal changes caused by early DR lesions such as microaneurysms and dot hemorrhages from longitudinal pairs of fundus images is challenging due to intra and inter-image illumination variation between fundus images. This paper explores a method for automated detection of retinal changes from illumination normalized fundus images using a deep convolutional neural network (CNN), and compares its performance with two other CNNs trained separately on color and green channel fundus images. Illumination variation was addressed by correcting for the variability in the luminosity and contrast estimated from a large scale retinal regions. The CNN models were trained and evaluated on image patches extracted from a registered fundus image set collected from 51 diabetic eyes that were screened at two different time-points. The results show that using normalized images yield better performance than color and green channel images, suggesting that illumination normalization greatly facilitates CNNs to quickly and correctly learn distinctive local image features of DR related retinal changes.

Automated tissue characterization is one of the major applications of computer-aided diagnosis systems. Deep learning techniques have recently demonstrated impressive performance for the image patch-based tissue characterization. However, existing patch-based tissue classification techniques struggle to exploit the useful shape information. Local and global shape knowledge such as the regional boundary changes, diameter, and volumetrics can be useful in classifying the tissues especially in scenarios where the appearance signature does not provide significant classification information. In this work, we present a deep neural network-based method for the automated segmentation of the tumors referred to as optic pathway gliomas (OPG) located within the anterior visual pathway (AVP; optic nerve, chiasm or tracts) using joint shape and appearance learning. Voxel intensity values of commonly used MRI sequences are generally not indicative of OPG. To be considered an OPG, current clinical practice dictates that some portion of AVP must demonstrate shape enlargement. The method proposed in this work integrates multiple sequence magnetic resonance image (T1, T2, and FLAIR) along with local boundary changes to train a deep neural network. For training and evaluation purposes, we used a dataset of multiple sequence MRI obtained from 20 subjects (10 controls, 10 NF1+OPG). To our best knowledge, this is the first deep representation learning-based approach designed to merge shape and multi-channel appearance data for the glioma detection. In our experiments, mean misclassification errors of 2:39% and 0:48% were observed respectively for glioma and control patches extracted from the AVP. Moreover, an overall dice similarity coefficient of 0:87±0:13 (0:93±0:06 for healthy tissue, 0:78±0:18 for glioma tissue) demonstrates the potential of the proposed method in the accurate localization and early detection of OPG.

Deep learning and convolutional neural networks (CNNs) in particular are increasingly popular tools for segmentation and classification of medical images. CNNs were shown to be successful for segmentation of brain tumors into multiple regions or labels. However, in the environment which fosters data-sharing and collection of multi-institutional datasets, a question arises: does training with data from another institution with potentially different imaging equipment, contrast protocol, and patient population impact the segmentation performance of the CNN? Our study presents preliminary data towards answering this question. Specifically, we used MRI data of glioblastoma (GBM) patients for two institutions present in The Cancer Imaging Archive. We performed a process of training and testing CNN multiple times such that half of the time the CNN was tested on data from the same institution that was used for training and half of the time it was tested on another institution, keeping the training and testing set size constant. We observed a decrease in performance as measured by Dice coefficient when the CNN was trained with data from a different institution as compared to training with data from the same institution. The changes in performance for the entire tumor and for four different labels within the tumor were: 0.72 to 0.65 (p=0.06), 0.61 to 0.58 (p=0.49), 0.54 to 0.51 (p=0.82), 0.31 to 0.24 (p<0.03), and 0.43 to 0.31(p<0.003) respectively. In summary, we found that while data across institutions can be used for development of CNNs, this might be associated with a decrease in performance.

Tourette syndrome (TS) is a childhood-onset neurobehavioral disorder. To date, TS is still misdiagnosed due to its varied presentation and lacking of obvious clinical symptoms. Therefore, studies of objective imaging biomarkers are of great importance for early TS diagnosis. As tic generation has been linked to disturbed structural networks, and many efforts have been made recently to investigate brain functional or structural networks using machine learning methods, for the purpose of disease diagnosis. However, few studies were related to TS and some drawbacks still existed in them. Therefore, we propose a novel classification framework integrating a multi-threshold strategy and a network fusion scheme to address the preexisting drawbacks. Here we used diffusion MRI probabilistic tractography to construct the structural networks of 44 TS children and 48 healthy children. We ameliorated the similarity network fusion algorithm specially to fuse the multi-threshold structural networks. Graph theoretical analysis was then implemented, and nodal degree, nodal efficiency and nodal betweenness centrality were selected as features. Finally, support vector machine recursive feature extraction (SVM-RFE) algorithm was used for feature selection, and then optimal features are fed into SVM to automatically discriminate TS children from controls. We achieved a high accuracy of 89.13% evaluated by a nested cross validation, demonstrated the superior performance of our framework over other comparison methods. The involved discriminative regions for classification primarily located in the basal ganglia and frontal cortico-cortical networks, all highly related to the pathology of TS. Together, our study may provide potential neuroimaging biomarkers for early-stage TS diagnosis.

Multi-Voxel Magnetic Resonance Spectroscopy (MV-MRS) provides an important and insightful technique for the examination of the chemical composition of brain tissue, making it an attractive medical imaging modality for the examination of brain tumours. MRS, however, is affected by the issue of the Partial Volume Effect (PVE), where the signals of multiple tissue types can be found within a single voxel and provides an obstacle to the interpretation of the data. The PVE results from the low resolution achieved in MV-MRS images relating to the signal to noise ratio (SNR). To counteract PVE, this paper proposes a novel Pairwise Mixture Model (PMM), that extends a recently reported Signal Mixture Model (SMM) for representing the MV-MRS signal as normal, low or high grade tissue types. Inspired by Conditional Random Field (CRF) and its continuous variant the PMM incorporates the surrounding voxel neighbourhood into an optimisation problem, the solution of which provides an estimation to a set of coefficients. The values of the estimated coefficients represents the amount of each tissue type (normal, low or high) found within a voxel. These coefficients can then be visualised as a nosological rendering using a coloured grid representing the MV-MRS image overlaid on top of a structural image, such as a Magnetic Resonance Image (MRI). Experimental results show an accuracy of 92.69% in classifying patient tumours as either low or high grade compared against the histopathology for each patient. Compared to 91.96% achieved by the SMM, the proposed PMM method demonstrates the importance of incorporating spatial coherence into the estimation as well as its potential clinical usage.

Purpose: To 1) find effective texture features from multimodal MRI that can distinguish IDH mutant and wild status, and 2) propose a radiomic strategy for preoperatively detecting IDH mutation patients with glioma. Materials and Methods: 152 patients with glioma were retrospectively included from the Cancer Genome Atlas. Corresponding T1-weighted image before- and post-contrast, T2-weighted image and fluid-attenuation inversion recovery image from the Cancer Imaging Archive were analyzed. Specific statistical tests were applied to analyze the different kind of baseline information of LrGG patients. Finally, 168 texture features were derived from multimodal MRI per patient. Then the support vector machine-based recursive feature elimination (SVM-RFE) and classification strategy was adopted to find the optimal feature subset and build the identification models for detecting the IDH mutation. Results: Among 152 patients, 92 and 60 were confirmed to be IDH-wild and mutant, respectively. Statistical analysis showed that the patients without IDH mutation was significant older than patients with IDH mutation (p<0.01), and the distribution of some histological subtypes was significant different between IDH wild and mutant groups (p<0.01). After SVM-RFE, 15 optimal features were determined for IDH mutation detection. The accuracy, sensitivity, specificity, and AUC after SVM-RFE and parameter optimization were 82.2%, 85.0%, 78.3%, and 0.841, respectively. Conclusion: This study presented a radiomic strategy for noninvasively discriminating IDH mutation of patients with glioma. It effectively incorporated kinds of texture features from multimodal MRI, and SVM-based classification strategy. Results suggested that features selected from SVM-RFE were more potential to identifying IDH mutation. The proposed radiomics strategy could facilitate the clinical decision making in patients with glioma.

Recent studies showed that genomic analysis of lower grade gliomas can be very effective for stratification of patients into groups with different prognosis and proposed specific genomic classifications. In this study, we explore the association of one of those genomic classifications with imaging parameters to determine whether imaging could serve a similar role to genomics in cancer patient treatment. Specifically, we analyzed imaging and genomics data for 110 patients from 5 institutions from The Cancer Genome Atlas and The Cancer Imaging Archive datasets. The analyzed imaging data contained preoperative FLAIR sequence for each patient. The images were analyzed using the in-house algorithms which quantify 2D and 3D aspects of the tumor shape. Genomic data consisted of a cluster of clusters classification proposed in a very recent and leading publication in the field of lower grade glioma genomics. Our statistical analysis showed that there is a strong association between the tumor cluster-of-clusters subtype and two imaging features: bounding ellipsoid volume ratio and angular standard deviation. This result shows high promise for the potential use of imaging as a surrogate measure for genomics in the decision process regarding treatment of lower grade glioma patients.

Glioblastoma Multiforme (GBM) is a highly aggressive brain tumor with a median survival of 14 months. Hypoxia is a hallmark trait in GBM that is known to be associated with angiogenesis, tumor growth, and resistance to conventional therapy, thereby limiting treatment options for GBM patients. There is thus an urgent clinical need for non-invasively capturing tumor hypoxia in GBM towards identifying a subset of patients who would likely benefit from anti-angiogenic therapies (bevacizumab) in the adjuvant setting. In this study, we employed radiomic descriptors to (a) capture molecular variations of tumor hypoxia on routine MRI that are otherwise not appreciable; and (b) employ the radiomic correlates of hypoxia to discriminate patients with short-term survival (STS, overall survival (OS) < 7 months), mid-term survival (MTS) (7 months<OS<16 months), and long-term survival (LTS, OS>16 months). A total of 97 studies (25 STS, 36 MTS, 36 LTS) with Gadolinium T1-contrast (Gd-T1c), T2w, and FLAIR protocols with their corresponding gene expression profiles were obtained from the cancer genome atlas (TCGA) database. For each MRI study, necrotic, enhancing tumor, and edematous regions were segmented by an expert. A total of 30 radiomic descriptors (i.e. Haralick, Laws energy, Gabor) were extracted from every region across all three MRI protocols. By performing unsupervised clustering of the expression profile of hypoxia associated genes, a "low", "medium", or "high" index was defined for every study. Spearman correlation was then used to identify the most significantly correlated MRI features with the hypoxia index for every study. These features were further used to categorize each study as STS, MTS, and LTS using Kaplan-Meier (KM) analysis. Our results revealed that the most significant features (p < 0.05) were identified as Laws energy and Haralick features that capture image heterogeneity on FLAIR and Gd-T1w sequences. We also found these radiomic features to be significantly associated with survival, distinguishing MTS from LTS (p=.005) and STS from LTS (p=.0008).

With the recent success of machine learning based solutions for automatic image parsing, the availability of reference image annotations for algorithm training is one of the major bottlenecks in medical image segmentation. We are interested in interactive semantic segmentation methods that can be used in an online fashion to generate expert segmentations. These can be used to train automated segmentation techniques or, from an application perspective, for quick and accurate tumor progression monitoring.

Using simulated user interactions in a MRI glioblastoma segmentation task, we show that if the user possesses knowledge of the correct segmentation it is significantly (p ≤ 0.009) better to present data and current segmentation to the user in such a manner that they can easily identify falsely classified regions compared to guiding the user to regions where the classifier exhibits high uncertainty, resulting in differences of mean Dice scores between +0.070 (Whole tumor) and +0.136 (Tumor Core) after 20 iterations. The annotation process should cover all classes equally, which results in a significant (p ≤ 0.002) improvement compared to completely random annotations anywhere in falsely classified regions for small tumor regions such as the necrotic tumor core (mean Dice +0.151 after 20 it.) and non-enhancing abnormalities (mean Dice +0.069 after 20 it.). These findings provide important insights for the development of efficient interactive segmentation systems and user interfaces.

In dental X-ray images, an accurate detection of cephalometric landmarks plays an important role in clinical diagnosis, treatment and surgical decisions for dental problems. In this work, we propose an end-to-end deep learning system for cephalometric landmark detection in dental X-ray images, using convolutional neural networks (CNN). For detecting 19 cephalometric landmarks in dental X-ray images, we develop a detection system using CNN-based coordinate-wise regression systems. By viewing x- and y-coordinates of all landmarks as 38 independent variables, multiple CNN-based regression systems are constructed to predict the coordinate variables from input X-ray images. First, each coordinate variable is normalized by the length of either height or width of an image. For each normalized coordinate variable, a CNN-based regression system is trained on training images and corresponding coordinate variable, which is a variable to be regressed. We train 38 regression systems with the same CNN structure on coordinate variables, respectively. Finally, we compute 38 coordinate variables with these trained systems from unseen images and extract 19 landmarks by pairing the regressed coordinates. In experiments, the public database from the Grand Challenges in Dental X-ray Image Analysis in ISBI 2015 was used and the proposed system showed promising performance by successfully locating the cephalometric landmarks within considerable margins from the ground truths.

In this study, we propose an automated otoscopy image analysis system called Autoscope. To the best of our knowledge, Autoscope is the first system designed to detect a wide range of eardrum abnormalities by using high-resolution otoscope images and report the condition of the eardrum as “normal” or “abnormal.” In order to achieve this goal, first, we developed a preprocessing step to reduce camera-specific problems, detect the region of interest in the image, and prepare the image for further analysis. Subsequently, we designed a new set of clinically motivated eardrum features (CMEF). Furthermore, we evaluated the potential of the visual MPEG-7 descriptors for the task of tympanic membrane image classification. Then, we fused the information extracted from the CMEF and state-of-the-art computer vision features (CVF), which included MPEG-7 descriptors and two additional features together, using a state of the art classifier. In our experiments, 247 tympanic membrane images with 14 different types of abnormality were used, and Autoscope was able to classify the given tympanic membrane images as normal or abnormal with 84.6% accuracy.

The tympanic membrane (TM) is the bridging element between the pressure waves of sound in air and the ossicular chain. It allows for sound to be conducted into the inner ear, achieving the human sense of hearing. Otitis media with effusion (OME, commonly referred to as ‘glue ear’) is a typical condition in infants that prevents the vibration of the TM and causes conductive hearing loss, this can lead to stunting early stage development if undiagnosed. Furthermore, OME is hard to identify in this age group; as they cannot respond to typical audiometry tests. Tympanometry allows for the mobility of the TM to be examined without patient response, but requires expensive apparatus and specialist training. By combining a smartphone equipped with a 240 frames per second video recording capability with an otoscopic clip-on accessory, this paper presents a novel application of Eulerian Video Magnification (EVM) to video-otology, that could provide assistance in diagnosing OME. We present preliminary results showing a spatio-temporal slice taken from an exaggerated video visualization of the TM being excited in vivo on a healthy ear. Our preliminary results demonstrate the potential for using such an approach for diagnosing OME under visual inspection as alternative to tympanometry, which could be used remotely and hence help diagnosis in a wider population pool.

Accurate tumor progression prediction for oropharyngeal cancers is crucial for identifying patients who would best be treated with optimized treatment and therefore minimize the risk of under- or over-treatment. An objective decision support system that can merge the available radiomics, histopathologic and molecular biomarkers in a predictive model based on statistical outcomes of previous cases and machine learning may assist clinicians in making more accurate assessment of oropharyngeal tumor progression. In this study, we evaluated the feasibility of developing individual and combined predictive models based on quantitative image analysis from radiomics, histopathology and molecular biomarkers for oropharyngeal tumor progression prediction. With IRB approval, 31, 84, and 127 patients with head and neck CT (CT-HN), tumor tissue microarrays (TMAs) and molecular biomarker expressions, respectively, were collected. For 8 of the patients all 3 types of biomarkers were available and they were sequestered in a test set. The CT-HN lesions were automatically segmented using our level sets based method. Morphological, texture and molecular based features were extracted from CT-HN and TMA images, and selected features were merged by a neural network. The classification accuracy was quantified using the area under the ROC curve (AUC). Test AUCs of 0.87, 0.74, and 0.71 were obtained with the individual predictive models based on radiomics, histopathologic, and molecular features, respectively. Combining the radiomics and molecular models increased the test AUC to 0.90. Combining all 3 models increased the test AUC further to 0.94. This preliminary study demonstrates that the individual domains of biomarkers are useful and the integrated multi-domain approach is most promising for tumor progression prediction.

Segmentation of anatomical structures is fundamental in the development of computer aided diagnosis systems for cerebral pathologies. Manual annotations are laborious, time consuming and subject to human error and observer variability. Accurate quantification of cerebrospinal fluid (CSF) can be employed as a morphometric measure for diagnosis and patient outcome prediction. However, segmenting CSF in non-contrast CT images is complicated by low soft tissue contrast and image noise. In this paper we propose a state-of-the-art method using a multi-scale three-dimensional (3D) fully convolutional neural network (CNN) to automatically segment all CSF within the cranial cavity. The method is trained on a small dataset comprised of four manually annotated cerebral CT images. Quantitative evaluation of a separate test dataset of four images shows a mean Dice similarity coefficient of 0.87 ± 0.01 and mean absolute volume difference of 4.77 ± 2.70 %. The average prediction time was 68 seconds. Our method allows for fast and fully automated 3D segmentation of cerebral CSF in non-contrast CT, and shows promising results despite a limited amount of training data.

The imaging biomarkers EmphysemaPresence and NoduleSpiculation are crucial inputs for most models aiming to predict the risk of indeterminate pulmonary nodules detected at CT screening. To increase reproducibility and to accelerate screening workflow it is desirable to assess these biomarkers automatically. Validation on NLST images indicates that standard histogram measures are not sufficient to assess EmphysemaPresence in screenees. However, automatic scoring of bulla-resembling low attenuation areas can achieve agreement with experts with close to 80% sensitivity and specificity. NoduleSpiculation can be automatically assessed with similar accuracy. We find a dedicated spiculi tracing score to slightly outperform generic combinations of texture features with classifiers.

We investigated the feasibility of classifying pathologic invasive nodules and pre-invasive or benign nodules by quantitative analysis of the CT attenuation distribution patterns and other radiomic features of lung nodule components. We developed a new 3D adaptive multi-component Expectation-Maximization (EM) analysis method to segment the solid and non-solid nodule components and the surrounding lung parenchymal region. Features were extracted to characterize the size, shape, and the CT attenuation distribution of the entire nodule as well as the individual regions. With permission of the National Lung Screening Trial (NLST) project, a data set containing the baseline low dose CT scans of 53 cases with known pathologic tumor type categorization was obtained. The 53 cases contain 45 invasive nodules (group 1) and 42 pre-invasive nodules (group 2). A logistic regression model (LRM) was built using leave-one-case-out resampling and receiver operating characteristic (ROC) analysis for classification of group 1 and group 2, using the pathologic categorization as ground truth. With 4 selected features, the LRM achieved a test area under the curve (AUC) value of 0.877±0.036. The results demonstrated that the pathologic invasiveness of lung adenocarcinomas could be categorized according to the CT attenuation distribution patterns of the nodule components manifested on LDCT images.

Lung cancer is the most common cause of cancer-related death. To diagnose lung cancers in early stages, numerous studies and approaches have been developed for cancer screening with computed tomography (CT) imaging. In recent years, convolutional neural networks (CNN) have become one of the most common and reliable techniques in computer aided detection (CADe) and diagnosis (CADx) by achieving state-of-the-art-level performances for various tasks. In this study, we propose a CNN classification system for false positive reduction of initially detected lung nodule candidates. First, image patches of lung nodule candidates are extracted from CT scans to train a CNN classifier. To reflect the volumetric contextual information of lung nodules to 2D image patch, we propose a weighted average image patch (WAIP) generation by averaging multiple slice images of lung nodule candidates. Moreover, to emphasize central slices of lung nodules, slice images are locally weighted according to Gaussian distribution and averaged to generate the 2D WAIP. With these extracted patches, 2D CNN is trained to achieve the classification of WAIPs of lung nodule candidates into positive and negative labels. We used LUNA 2016 public challenge database to validate the performance of our approach for false positive reduction in lung CT nodule classification. Experiments show our approach improves the classification accuracy of lung nodules compared to the baseline 2D CNN with patches from single slice image.

In this paper, we investigated the problem of diagnostic lung nodule malignancy prediction using thoracic Computed Tomography (CT) screening. Unlike most existing studies classify the nodules into two types benign and malignancy, we interpreted the nodule malignancy prediction as a regression problem to predict continuous malignancy level. We proposed a joint multi-task learning algorithm using Convolutional Neural Network (CNN) to capture nodule heterogeneity by extracting discriminative features from alternatingly stacked layers. We trained a CNN regression model to predict the nodule malignancy, and designed a multi-task learning mechanism to simultaneously share knowledge among 9 different nodule characteristics (Subtlety, Calcification, Sphericity, Margin, Lobulation, Spiculation, Texture, Diameter and Malignancy), and improved the final prediction result. Each CNN would generate characteristic-specific feature representations, and then we applied multi-task learning on the features to predict the corresponding likelihood for that characteristic. We evaluated the proposed method on 2620 nodules CT scans from LIDC-IDRI dataset with the 5-fold cross validation strategy. The multitask CNN regression result for regression RMSE and mapped classification ACC were 0.830 and 83.03%, while the results for single task regression RMSE 0.894 and mapped classification ACC 74.9%. Experiments show that the proposed method could predict the lung nodule malignancy likelihood effectively and outperforms the state-of-the-art methods. The learning framework could easily be applied in other anomaly likelihood prediction problem, such as skin cancer and breast cancer. It demonstrated the possibility of our method facilitating the radiologists for nodule staging assessment and individual therapeutic planning.

To determine the potential usefulness of quantified diagnostic image features as inputs to a CAD system, we investigate the predictive capabilities of statistical learning methods for classifying nodule malignancy, utilizing the Lung Image Database Consortium (LIDC) dataset, and only employ the radiologist-assigned diagnostic feature values for the lung nodules therein, as well as our derived estimates of the diameter and volume of the nodules from the radiologists' annotations. We calculate theoretical upper bounds on the classification accuracy that is achievable by an ideal classifier that only uses the radiologist-assigned feature values, and we obtain an accuracy of 85.74 (±1.14)% which is, on average, 4.43% below the theoretical maximum of 90.17%. The corresponding area-under-the-curve (AUC) score is 0.932 (±0.012), which increases to 0.949 (±0.007) when diameter and volume features are included, along with the accuracy to 88.08 (±1.11)%. Our results are comparable to those in the literature that use algorithmically-derived image-based features, which supports our hypothesis that lung nodules can be classified as malignant or benign using only quantified, diagnostic image features, and indicates the competitiveness of this approach. We also analyze how the classification accuracy depends on specific features, and feature subsets, and we rank the features according to their predictive power, statistically demonstrating the top four to be spiculation, lobulation, subtlety, and calcification.

Patient-targeted treatment of non-small cell lung carcinoma (NSCLC) has been well documented according to the histologic subtypes over the past decade. In parallel, recent development of quantitative image biomarkers has recently been highlighted as important diagnostic tools to facilitate histological subtype classification. In this study, we present a radiomics analysis that classifies the adenocarcinoma (ADC) and squamous cell carcinoma (SqCC). We extract 52-dimensional, CT-based features (7 statistical features and 45 image texture features) to represent each nodule. We evaluate our approach on a clinical dataset including 324 ADCs and 110 SqCCs patients with CT image scans. Classification of these features is performed with four different machine-learning classifiers including Support Vector Machines with Radial Basis Function kernel (RBF-SVM), Random forest (RF), K-nearest neighbor (KNN), and RUSBoost algorithms. To improve the classifiers’ performance, optimal feature subset is selected from the original feature set by using an iterative forward inclusion and backward eliminating algorithm. Extensive experimental results demonstrate that radiomics features achieve encouraging classification results on both complete feature set (AUC=0.89) and optimal feature subset (AUC=0.91).

Support vector machines (SVM) are applied to the problem of prostate lesion classification for the SPIE ProstateX Challenge 2016, achieving a score of 0.82 AUC on held-out test data. Square 5mm transverse image patches are extracted around each lesion center from aligned MRI scans. Three MRI modalities are simultaneously analyzed: T2-weighted, apparent diffusion coefficient (ADC) and volume transfer constant (K^trans). Extracted patches are used to train a binary classifier to predict clinical significance. The machine learning algorithm is trained on 76 positive cases and 254 negative cases (330 total) from the challenge. The method is conceptually simple, trains in a few seconds and yields competitive results.

A novel deep learning architecture (XmasNet) based on convolutional neural networks was developed for the classification of prostate cancer lesions, using the 3D multiparametric MRI data provided by the PROSTATEx challenge. End-to-end training was performed for XmasNet, with data augmentation done through 3D rotation and slicing, in order to incorporate the 3D information of the lesion. XmasNet outperformed traditional machine learning models based on engineered features, for both train and test data. For the test data, XmasNet outperformed 69 methods from 33 participating groups and achieved the second highest AUC (0.84) in the PROSTATEx challenge. This study shows the great potential of deep learning for cancer imaging.

In this manuscript, we describe our approach and methods to the ProstateX challenge, which achieved an overall AUC of 0.84 and the runner-up position. We train a deep convolutional neural network to classify lesions marked on multiparametric MRI of the prostate as clinically significant or not.

We implement a novel addition to the standard convolutional architecture described as auto-windowing which is clinically inspired and designed to overcome some of the difficulties faced in MRI interpretation, where high dynamic ranges and low contrast edges may cause difficulty for traditional convolutional neural networks trained on high contrast natural imagery. We demonstrate that this system can be trained end to end and outperforms a similar architecture without such additions. Although a relatively small training set was provided, we use extensive data augmentation to prevent overfitting and transfer learning to improve convergence speed, showing that deep convolutional neural networks can be feasibly trained on small datasets.

Prostate cancer (PCa) remains a leading cause of cancer mortality among American men. Multi-parametric magnetic resonance imaging (mpMRI) is widely used to assist with detection of PCa and characterization of its aggressiveness. Computer-aided diagnosis (CADx) of PCa in MRI can be used as clinical decision support system to aid radiologists in interpretation and reporting of mpMRI. We report on the development of a convolution neural network (CNN) model to support CADx in PCa based on the appearance of prostate tissue in mpMRI, conducted as part of the SPIE-AAPM-NCI PROSTATEx challenge. The performance of different combinations of mpMRI inputs to CNN was assessed and the best result was achieved using DWI and DCE-MRI modalities together with the zonal information of the finding. On the test set, the model achieved an area under the receiver operating characteristic curve of 0.80.

¹⁸F-DMFP-PET is a neuroimaging modality that allows us to analyze the striatal dopamine. Thus, it is recently emerging as an effective tool to assist the diagnosis of Parkinsonism and differentiate among parkinsonian syndromes. However the analysis of these data, which require specific preprocessing methods, is still poorly covered. In this work we demonstrate a novel methodology based on Hidden Markov Random Fields (HMRF) and the Gaussian distribution to preprocess ¹⁸F-DMFP-PET data. First, we performed a selection of voxels based on the analysis of the histogram in order to remove low-signal regions and regions outside the brain. Specifically, we modeled the histogram of intensities of a neuroimage with a mixture of two Gaussians and then, using a HMRF algorithm the voxels corresponding to the low-intensity Gaussian were discarded. This procedure is similar to the tissue segmentation usually applied to Magnetic Resonance Imaging data. Secondly, the intensity of the selected voxels was scaled so that the Gaussian that models the histogram for each neuroimage has same mean and standard deviation. This step made comparable the data from different patients, without removing the characteristic patterns of each patient's disorder. The proposed approach was evaluated using a computer system based on statistical classification that separated the neuroimages according to the parkinsonian variant they represented. The proposed approach achieved higher accuracy rates than standard approaches for voxel selection (based on atlases) and intensity normalization (based on the global mean).

The early diagnosis of idiopathic normal pressure hydrocephalus (iNPH) considered as a treatable dementia is important. The iNPH causes enlargement of lateral ventricles (LVs). The degree of the enlargement of the LVs on CT or MR images is evaluated by using a diagnostic imaging criterion, Evans index. Evans index is defined as the ratio of the maximal width of frontal horns (FH) of the LVs to the maximal width of the inner skull (IS). Evans index is the most commonly used parameter for the evaluation of ventricular enlargement. However, manual measurement of Evans index is a time-consuming process. In this study, we present an automated method to compute Evans index on brain CT images. The algorithm of the method consisted of five major steps: standardization of CT data to an atlas, extraction of FH and IS regions, the search for the outmost points of bilateral FH regions, determination of the maximal widths of both the FH and the IS, and calculation of Evans index. The standardization to the atlas was performed by using linear affine transformation and non-linear wrapping techniques. The FH regions were segmented by using a three dimensional region growing technique. This scheme was applied to CT scans from 44 subjects, including 13 iNPH patients. The average difference in Evans index between the proposed method and manual measurement was 0.01 (1.6%), and the correlation coefficient of these data for the Evans index was 0.98. Therefore, this computerized method may have the potential to accurately compute Evans index for the diagnosis of iNPH on CT images.

Statistical learning and decision theory play a key role in many areas of science and engineering. Some examples include time series regression and prediction, optical character recognition, signal detection in communications or biomedical applications for diagnosis and prognosis. This paper deals with the topic of learning from biomedical image data in the classification problem. In a typical scenario we have a training set that is employed to fit a prediction model or learner and a testing set on which the learner is applied to in order to predict the outcome for new unseen patterns. Both processes are usually completely separated to avoid over-fitting and due to the fact that, in practice, the unseen new objects (testing set) have unknown outcomes. However, the outcome yields one of a discrete set of values, i.e. the binary diagnosis problem. Thus, assumptions on these outcome values could be established to obtain the most likely prediction model at the training stage, that could improve the overall classification accuracy on the testing set, or keep its performance at least at the level of the selected statistical classifier. In this sense, a novel case-based learning (c-learning) procedure is proposed which combines hypothesis testing from a discrete set of expected outcomes and a cross-validated classification stage.

Prediction of survival time from brain tumor magnetic resonance images (MRI) is not commonly performed and would ordinarily be a time consuming process. However, current cross-sectional imaging techniques, particularly MRI, can be used to generate many features that may provide information on the patient’s prognosis, including survival. This information can potentially be used to identify individuals who would benefit from more aggressive therapy. Rather than using pre-defined and hand-engineered features as with current radiomics methods, we investigated the use of deep features extracted from pre-trained convolutional neural networks (CNNs) in predicting survival time. We also provide evidence for the power of domain specific fine-tuning in improving the performance of a pre-trained CNN’s, even though our data set is small. We fine-tuned a CNN initially trained on a large natural image recognition dataset (Imagenet ILSVRC) and transferred the learned feature representations to the survival time prediction task, obtaining over 81% accuracy in a leave one out cross validation.

The world’s aging population has given rise to an increasing awareness towards neurodegenerative disorders, including Alzheimers Disease (AD). Treatment options for AD are currently limited, but it is believed that future success depends on our ability to detect the onset of the disease in its early stages. The most frequently used tools for this include neuropsychological assessments, along with genetic, proteomic, and image-based diagnosis. Recently, the applicability of Diffusion Magnetic Resonance Imaging (dMRI) analysis for early diagnosis of AD has also been reported. The sensitivity of dMRI to the microstructural organization of cerebral tissue makes it particularly well-suited to detecting changes which are known to occur in the early stages of AD. Existing dMRI approaches can be divided into two broad categories: region-based and tract-based. In this work, we propose a new approach, which extends region-based approaches to the simultaneous characterization of multiple brain regions. Given a predefined set of features derived from dMRI data, we compute the probabilistic distances between different brain regions and treat the resulting connectivity pattern as an undirected, fully-connected graph. The characteristics of this graph are then used as markers to discriminate between AD subjects and normal controls (NC). Although in this preliminary work we omit subjects in the prodromal stage of AD, mild cognitive impairment (MCI), our method demonstrates perfect separability between AD and NC subject groups with substantial margin, and thus holds promise for fine-grained stratification of NC, MCI and AD populations.

Traumatic brain injury (TBI) is a major cause of death and disability in the United States. Time to treatment is often related to patient outcome. Access to cerebral imaging data in a timely manner is a vital component of patient care. Current methods of detecting and quantifying intracranial pathology can be time-consuming and require careful review of 2D/3D patient images by a radiologist. Additional time is needed for image protocoling, acquisition, and processing. These steps often occur in series, adding more time to the process and potentially delaying time-dependent management decisions for patients with traumatic brain injury.

Our team adapted machine learning and computer vision methods to develop a technique that rapidly and automatically detects CT-identifiable lesions. Specifically, we use scale invariant feature transform (SIFT)¹ and deep convolutional neural networks (CNN)² to identify important image features that can distinguish TBI lesions from background data. Our learning algorithm is a linear support vector machine (SVM)³. Further, we also employ tools from topological data analysis (TDA) for gleaning insights into the correlation patterns between healthy and pathological data. The technique was validated using 409 CT scans of the brain, acquired via the Progesterone for the Treatment of Traumatic Brain Injury phase III clinical trial (ProTECT_III) which studied patients with moderate to severe TBI⁴. CT data were annotated by a central radiologist and included patients with positive and negative scans. Additionally, the largest lesion on each positive scan was manually segmented. We reserved 80% of the data for training the SVM and used the remaining 20% for testing. Preliminary results are promising with 92.55% prediction accuracy (sensitivity = 91.15%, specificity = 93.45%), indicating the potential usefulness of this technique in clinical scenarios.

Parkinsonian syndromes encompass a spectrum of neurodegenerative diseases, which can be classified into various subtypes. The differentiation of these subtypes is typically conducted based on clinical criteria. Due to the overlap of intra-syndrome symptoms, the accurate differential diagnosis based on clinical guidelines remains a challenge with failure rates up to 25%. The aim of this study is to present an image-based classification method of patients with Parkinson’s disease (PD) and patients with progressive supranuclear palsy (PSP), an atypical variant of PD. Therefore, apparent diffusion coefficient (ADC) parameter maps were calculated based on diffusion-tensor magnetic resonance imaging (MRI) datasets. Mean ADC values were determined in 82 brain regions using an atlas-based approach. The extracted mean ADC values for each patient were then used as features for classification using a linear kernel support vector machine classifier. To increase the classification accuracy, a feature selection was performed, which resulted in the top 17 attributes to be used as the final input features. A leave-one-out cross validation based on 56 PD and 21 PSP subjects revealed that the proposed method is capable of differentiating PD and PSP patients with an accuracy of 94.8%. In conclusion, the classification of PD and PSP patients based on ADC features obtained from diffusion MRI datasets is a promising new approach for the differentiation of Parkinsonian syndromes in the broader context of decision support systems.

Detection of residual brain tumor is important to evaluate efficacy of brain cancer surgery, determine optimal strategy of further radiation therapy if needed, and assess ultimate prognosis of the patients. Brain MR is a commonly used imaging modality for this task. In order to distinguish between residual tumor and surgery induced scar tissues, two sets of MRI scans are conducted pre- and post-gadolinium contrast injection. The residual tumors are only enhanced in the post-contrast injection images. However, subjective reading and quantifying this type of brain MR images faces difficulty in detecting real residual tumor regions and measuring total volume of the residual tumor. In order to help solve this clinical difficulty, we developed and tested a new interactive computer-aided detection scheme, which consists of three consecutive image processing steps namely, 1) segmentation of the intracranial region, 2) image registration and subtraction, 3) tumor segmentation and refinement. The scheme also includes a specially designed and implemented graphical user interface (GUI) platform. When using this scheme, two sets of pre- and post-contrast injection images are first automatically processed to detect and quantify residual tumor volume. Then, a user can visually examine segmentation results and conveniently guide the scheme to correct any detection or segmentation errors if needed. The scheme has been repeatedly tested using five cases. Due to the observed high performance and robustness of the testing results, the scheme is currently ready for conducting clinical studies and helping clinicians investigate the association between this quantitative image marker and outcome of patients.

Confocal laser endomicroscopy (CLE), although capable of obtaining images at cellular resolution during surgery of brain tumors in real time, creates as many non-diagnostic as diagnostic images. Non-useful images are often distorted due to relative motion between probe and brain or blood artifacts. Many images, however, simply lack diagnostic features immediately informative to the physician. Examining all the hundreds or thousands of images from a single case to discriminate diagnostic images from nondiagnostic ones can be tedious. Providing a real time diagnostic value assessment of images (fast enough to be used during the surgical acquisition process and accurate enough for the pathologist to rely on) to automatically detect diagnostic frames would streamline the analysis of images and filter useful images for the pathologist/surgeon. We sought to automatically classify images as diagnostic or non-diagnostic. AlexNet, a deep-learning architecture, was used in a 4-fold cross validation manner. Our dataset includes 16,795 images (8572 nondiagnostic and 8223 diagnostic) from 74 CLE-aided brain tumor surgery patients. The ground truth for all the images is provided by the pathologist. Average model accuracy on test data was 91% overall (90.79 % accuracy, 90.94 % sensitivity and 90.87 % specificity). To evaluate the model reliability we also performed receiver operating characteristic (ROC) analysis yielding 0.958 average for area under ROC curve (AUC). These results demonstrate that a deeply trained AlexNet network can achieve a model that reliably and quickly recognizes diagnostic CLE images.

Stroke is a leading cause of death and disability in the western hemisphere. Acute ischemic strokes can be broadly classified based on the underlying cause into atherosclerotic strokes, cardioembolic strokes, small vessels disease, and stroke with other causes. The ability to determine the exact origin of an acute ischemic stroke is highly relevant for optimal treatment decision and preventing recurrent events. However, the differentiation of atherosclerotic and cardioembolic phenotypes can be especially challenging due to similar appearance and symptoms. The aim of this study was to develop and evaluate the feasibility of an image-based machine learning approach for discriminating between arteriosclerotic and cardioembolic acute ischemic strokes using 56 apparent diffusion coefficient (ADC) datasets from acute stroke patients. For this purpose, acute infarct lesions were semi-atomically segmented and 30,981 geometric and texture image features were extracted for each stroke volume. To improve the performance and accuracy, categorical Pearson’s χ² test was used to select the most informative features while removing redundant attributes. As a result, only 289 features were finally included for training of a deep multilayer feed-forward neural network without bootstrapping. The proposed method was evaluated using a leave-one-out cross validation scheme. The proposed classification method achieved an average area under receiver operator characteristic curve value of 0.93 and a classification accuracy of 94.64%. These first results suggest that the proposed image-based classification framework can support neurologists in clinical routine differentiating between atherosclerotic and cardioembolic phenotypes.

In this work, we propose a novel method to improve texture based tumor segmentation by fusing cell density patterns that are generated from tumor growth modeling. To model tumor growth, we solve the reaction-diffusion equation by using Lattice-Boltzmann method (LBM). Computational tumor growth modeling obtains the cell density distribution that potentially indicates the predicted tissue locations in the brain over time. The density patterns is then considered as novel features along with other texture (such as fractal, and multifractal Brownian motion (mBm)), and intensity features in MRI for improved brain tumor segmentation. We evaluate the proposed method with about one hundred longitudinal MRI scans from five patients obtained from public BRATS 2015 data set, validated by the ground truth. The result shows significant improvement of complete tumor segmentation using ANOVA analysis for five patients in longitudinal MR images.

Breast cancer is the leading cause of death for women in most countries. The high levels of mortality relate mostly to late diagnosis and to the direct proportionally relationship between breast density and breast cancer development. Therefore, the correct assessment of breast density is important to provide better screening for higher risk patients. However, in modern digital mammography the discrimination among breast densities is highly complex due to increased contrast and visual information for all densities. Thus, a computational system for classifying breast density might be a useful tool for aiding medical staff. Several machine-learning algorithms are already capable of classifying small number of classes with good accuracy. However, machinelearning algorithms main constraint relates to the set of features extracted and used for classification. Although well-known feature extraction techniques might provide a good set of features, it is a complex task to select an initial set during design of a classifier. Thus, we propose feature extraction using a Convolutional Neural Network (CNN) for classifying breast density by a usual machine-learning classifier. We used 307 mammographic images downsampled to 260x200 pixels to train a CNN and extract features from a deep layer. After training, the activation of 8 neurons from a deep fully connected layer are extracted and used as features. Then, these features are feedforward to a single hidden layer neural network that is cross-validated using 10-folds to classify among four classes of breast density. The global accuracy of this method is 98.4%, presenting only 1.6% of misclassification. However, the small set of samples and memory constraints required the reuse of data in both CNN and MLP-NN, therefore overfitting might have influenced the results even though we cross-validated the network. Thus, although we presented a promising method for extracting features and classifying breast density, a greater database is still required for evaluating the results.

Mammography analysis is an important tool that helps detecting breast cancer at the very early stages of the disease, thus increasing the quality of life of hundreds of thousands of patients worldwide. In Computer-Aided Detection systems, the identification of mammograms with and without masses (without clinical findings) is highly needed to reduce the false positive rates regarding the automatic selection of regions of interest that may contain some suspicious content. In this work, the introduce a variant of the Optimum-Path Forest (OPF) classifier for breast mass identification, as well as we employed an ensemble-based approach that can enhance the effectiveness of individual classifiers aiming at dealing with the aforementioned purpose. The experimental results also comprise the naïve OPF and a traditional neural network, being the most accurate results obtained through the ensemble of classifiers, with an accuracy nearly to 86%.

Mammographic breast density is a well-established marker for breast cancer risk. However, accurate measurement of dense tissue is a difficult task due to faint contrast and significant variations in background fatty tissue. This study presents a novel method for automated mammographic density estimation based on Convolutional Neural Network (CNN). A total of 397 full-field digital mammograms were selected from Seoul National University Hospital. Among them, 297 mammograms were randomly selected as a training set and the rest 100 mammograms were used for a test set. We designed a CNN architecture suitable to learn the imaging characteristic from a multitudes of sub-images and classify them into dense and fatty tissues. To train the CNN, not only local statistics but also global statistics extracted from an image set were used. The image set was composed of original mammogram and eigen-image which was able to capture the X-ray characteristics in despite of the fact that CNN is well known to effectively extract features on original image. The 100 test images which was not used in training the CNN was used to validate the performance. The correlation coefficient between the breast estimates by the CNN and those by the expert’s manual measurement was 0.96. Our study demonstrated the feasibility of incorporating the deep learning technology into radiology practice, especially for breast density estimation. The proposed method has a potential to be used as an automated and quantitative assessment tool for mammographic breast density in routine practice.

High mammographic density is associated with an increased risk of breast cancer, however whether the association is stronger when there is agreement across measures is unclear. This study investigates whether a combination of density measures is a better predictor of breast cancer risk than individual methods alone. Women recruited to the Predicting Risk of Cancer At Screening (PROCAS) study and with mammographic density assessed using three different methods were included (n=33,304). Density was assessed visually using Visual Analogue Scales (VAS) and by two fully automated methods, Quantra and Volpara. Percentage breast density was divided into (high, medium and low) and combinations of measures were used to further categorise individuals (e.g. ‘all high’). A total of 667 breast cancers were identified and logistic regression was used to determine the relationship between breast density and breast cancer risk. In total, 44% of individuals were in the same tertile for all three measures, 8.6% were in non-adjacent (high and low) or mixed categories (high, medium and low). For individual methods the strongest association with breast cancer risk was for medium and high tertiles of VAS with odds ratios (OR) adjusted for age and BMI of 1.63 (95% CI 1.31-2.03) and 2.33 (1.87-2.90) respectively. For the combination of density methods the strongest association was for ‘all high’ (OR 2.42, 1.77-3.31) followed by “two high” (OR 1.90, 1.35-3.31) and “two medium” (OR 1.88, 1.40-2.52). Combining density measures did not affect the magnitude of risk compared to using individual methods.

This paper describes a novel approach for the automatic assessment of breast density in non-contrast three-dimensional computed tomography (3D CT) images. The proposed approach trains and uses a deep convolutional neural network (CNN) from scratch to classify breast tissue density directly from CT images without segmenting the anatomical structures, which creates a bottleneck in conventional approaches. Our scheme determines breast density in a 3D breast region by decomposing the 3D region into several radial 2D-sections from the nipple, and measuring the distribution of breast tissue densities on each 2D section from different orientations. The whole scheme is designed as a compact network without the need for post-processing and provides high robustness and computational efficiency in clinical settings. We applied this scheme to a dataset of 463 non-contrast CT scans obtained from 30- to 45-year-old-women in Japan. The density of breast tissue in each CT scan was assigned to one of four categories (glandular tissue within the breast <25%, 25%–50%, 50%–75%, and >75%) by a radiologist as ground truth. We used 405 CT scans for training a deep CNN and the remaining 58 CT scans for testing the performance. The experimental results demonstrated that the findings of the proposed approach and those of the radiologist were the same in 72% of the CT scans among the training samples and 76% among the testing samples. These results demonstrate the potential use of deep CNN for assessing breast tissue density in non-contrast 3D CT images.