This PDF file contains the front matter associated with SPIE Proceedings Volume 10134, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing.

We are developing a computerized system for detection of bladder cancer in CT urography. In this study, we used a deep-learning convolutional neural network (DL-CNN) to segment the bladder wall. This task is challenging due to differences in the wall between the contrast and non-contrast-filled regions, significant variations in appearance, size, and shape of the bladder among cases, overlap of the prostate with the bladder wall, and the wall being extremely thin compared to the overall size of the bladder. We trained a DL-CNN to estimate the likelihood that a given pixel would be inside the wall of the bladder using neighborhood information. A segmented bladder wall was then obtained using level sets with this likelihood map as a term in the level set energy formulation to obtain contours of the inner and outer bladder walls. The accuracy of the segmentation was evaluated by comparing the segmented wall outlines to hand outlines for a set of 79 training cases and 15 test cases using the average volume intersection % as the metric. For the training set, the inner wall achieved an average volume intersection of 90.0±8.7% and the outer wall achieved 93.7±3.9%. For the test set, the inner wall achieved an average volume intersection of 87.6±7.6% and the outer wall achieved 87.2±9.3%. The results show that the DL-CNN with level sets was effective in segmenting the inner and outer bladder walls.

We are developing a computer-aided detection system for bladder cancer in CT urography (CTU). We have previously developed methods for detection of bladder masses within the contrast-enhanced and the non-contrastenhanced regions of the bladder individually. In this study, we investigated methods for detection of bladder masses within the entire bladder. The bladder was segmented using our method that combined deep-learning convolutional neural network with level sets. The non-contrast-enhanced region was separated from the contrast-enhanced region with a maximum-intensity-projection-based method. The non-contrast region was smoothed and gray level threshold was applied to the contrast and non-contrast regions separately to extract the bladder wall and potential masses. The bladder wall was transformed into a straightened thickness profile, which was analyzed to identify lesion candidates in a prescreening step. The candidates were mapped back to the 3D CT volume and segmented using our auto-initialized cascaded level set (AI-CALS) segmentation method. Twenty-seven morphological features were extracted for each candidate. A data set of 57 patients with 71 biopsy-proven bladder lesions was used, which was split into independent training and test sets: 42 training cases with 52 lesions, and 15 test cases with 19 lesions. Using the training set, feature selection was performed and a linear discriminant (LDA) classifier was designed to merge the selected features for classification of bladder lesions and false positives. The trained classifier was evaluated with the test set. FROC analysis showed that the system achieved a sensitivity of 86.5% at 3.3 FPs/case for the training set, and 84.2% at 3.7 FPs/case for the test set.

We are developing a CAD system for bladder cancer treatment response assessment in CT. We compared the performance of the deep-learning convolution neural network (DL-CNN) using different network sizes, and with and without transfer learning using natural scene images or regions of interest (ROIs) inside and outside the bladder. The DL-CNN was trained to identify responders (T0 disease) and non-responders to chemotherapy. ROIs were extracted from segmented lesions in pre- and post-treatment scans of a patient and paired to generate hybrid pre-post-treatment paired ROIs. The 87 lesions from 82 patients generated 104 temporal lesion pairs and 6,700 pre-post-treatment paired ROIs. Two-fold cross-validation and receiver operating characteristic analysis were performed and the area under the curve (AUC) was calculated for the DL-CNN estimates. The AUCs for prediction of T0 disease after treatment were 0.77±0.08 and 0.75±0.08, respectively, for the two partitions using DL-CNN without transfer learning and a small network, and were 0.74±0.07 and 0.74±0.08 with a large network. The AUCs were 0.73±0.08 and 0.62±0.08 with transfer learning using a small network pre-trained with bladder ROIs. The AUC values were 0.77±0.08 and 0.73±0.07 using the large network pre-trained with the same bladder ROIs. With transfer learning using the large network pretrained with the Canadian Institute for Advanced Research (CIFAR-10) data set, the AUCs were 0.72±0.06 and 0.64±0.09, respectively, for the two partitions. None of the differences in the methods reached statistical significance. Our study demonstrated the feasibility of using DL-CNN for the estimation of treatment response in CT. Transfer learning did not improve the treatment response estimation. The DL-CNN performed better when transfer learning with bladder images was used instead of natural scene images.

Prostate cancer (PCa) is the second most common cause of cancer related deaths in men. Multiparametric MRI (mpMRI) is the most accurate imaging method for PCa detection; however, it requires the expertise of experienced radiologists leading to inconsistency across readers of varying experience. To increase inter-reader agreement and sensitivity, we developed a computer-aided detection (CAD) system that can automatically detect lesions on mpMRI that readers can use as a reference. We investigated a convolutional neural network based deep-learing (DCNN) architecture to find an improved solution for PCa detection on mpMRI. We adopted a network architecture from a state-of-the-art edge detector that takes an image as an input and produces an image probability map. Two-fold cross validation along with a receiver operating characteristic (ROC) analysis and free-response ROC (FROC) were used to determine our deep-learning based prostate-CAD’s (CADDL) performance. The efficacy was compared to an existing prostate CAD system that is based on hand-crafted features, which was evaluated on the same test-set. CADDL had an 86% detection rate at 20% false-positive rate while the top-down learning CAD had 80% detection rate at the same false-positive rate, which translated to 94% and 85% detection rate at 10 false-positives per patient on the FROC. A CNN based CAD is able to detect cancerous lesions on mpMRI of the prostate with results comparable to an existing prostate-CAD showing potential for further development.

Functional analysis of the placenta is important to analyze and understand its role in fetal growth and development. BOLD MR is a non-invasive technique that has been extensively used for functional analysis of the brain. During the last years, several studies have shown that this dynamic image modality is also useful to extract functional information of the placenta. We propose in this paper a method to track the placenta from a sequence of BOLD MR images acquired under normoxia and hyperoxia conditions with the goal of quantifying how the placenta adapts to oxygenation changes. The method is based on a spatiotemporal transformation model that ensures temporal coherence of the tracked structures. The method was initially applied to four patients with healthy pregnancies. An average MR signal increase of 16.96±8.39% during hyperoxia was observed. These automated results are in line with state-of-the-art reports using time-consuming manual segmentations subject to inter-observer errors.

The infiltrative lung diseases are a class of irreversible, non-neoplastic lung pathologies requiring regular follow-up with CT imaging. Quantifying the evolution of the patient status imposes the development of automated classification tools for lung texture. Traditionally, such classification relies on a two-dimensional analysis of axial CT images. This paper proposes a cascade of the existing CNN based CAD system, specifically tuned-up. The advantage of using a deep learning approach is a better regularization of the classification output. In a preliminary evaluation, the combined approach was tested on a 13 patient database of various lung pathologies, showing an increase of 10% in True Positive Rate (TPR) with respect to the best suited state of the art CNN for this task.

A fully-automated computer algorithm has been developed to identify early-stage Usual Interstitial Pneumonia (UIP) using features computed from low-dose CT scans. In each scan, the pre-segmented lung region is divided into N subsections (N = 1, 8, 27, 64) by separating the lung from anterior/posterior, left/right and superior/inferior in 3D space. Each subsection has approximately the same volume. In each subsection, a classic density measurement (fractional high-density volume h) is evaluated to characterize the disease severity in that subsection, resulting in a feature vector of length N for each lung. Features are then combined in two different ways: concatenation (2*N features) and taking the maximum in each of the two corresponding subsections in the two lungs (N features).

The algorithm was evaluated on a dataset consisting of 51 UIP and 56 normal cases, a combined feature vector was computed for each case and an SVM classifier (RBF kernel) was used to classify them into UIP or normal using ten-fold cross validation. A receiver operating characteristic (ROC) area under the curve (AUC) was used for evaluation. The highest AUC of 0.95 was achieved by using concatenated features and an N of 27. Using lung partition (N = 27, 64) with concatenated features had significantly better result over not using partitions (N = 1) (p-value < 0.05). Therefore this equal-volume partition fractional high-density volume method is useful in distinguishing early-stage UIP from normal cases.

Deep convolutional neural networks (CNNs) form the backbone of many state-of-the-art computer vision systems for classification and segmentation of 2D images. The same principles and architectures can be extended to three dimensions to obtain 3D CNNs that are suitable for volumetric data such as CT scans. In this work, we train a 3D CNN for automatic detection of pulmonary nodules in chest CT images using volumes of interest extracted from the LIDC dataset. We then convert the 3D CNN which has a fixed field of view to a 3D fully convolutional network (FCN) which can generate the score map for the entire volume efficiently in a single pass. Compared to the sliding window approach for applying a CNN across the entire input volume, the FCN leads to a nearly 800-fold speed-up, and thereby fast generation of output scores for a single case. This screening FCN is used to generate difficult negative examples that are used to train a new discriminant CNN. The overall system consists of the screening FCN for fast generation of candidate regions of interest, followed by the discrimination CNN.

Lung cancer is the leading cause of cancer deaths worldwide. Early diagnosis is critical in increasing the 5-year survival rate of lung cancer, so the efficient and accurate detection of lung nodules, potential precursors to lung cancer, is evermore important. In this paper, a computer-aided lung nodule detection system using convolution neural networks (CNN) and handcrafted features for false positive reduction is developed. The CNNs were trained with three types of images: lung CT images, their nodule-enhanced images, and their blood vessel-enhanced images. For each nodule candidate, nine 2D patches from differently oriented planes were extracted from each type of images. Patches of the same orientation from the same type of image across different candidates were used to train the CNNs independently, which were used to extract 864 features. 88 handcrafted features including intensity, shape, and texture features were also obtained from the lung CT images. The CNN features and handcrafted features were then combined to train a classifier, and a support vector machine was adopted to achieve the final classification results. The proposed method was evaluated on 1004 CT scans from the LIDC-IDRI database using 10-fold cross-validation. Compared with the traditional CNN method using only lung CT images, the proposed method boosted the sensitivity of nodule detection from 89.0% to 90.9% at 4 FPs/scan and from 71.6% to 78.2% at 1 FP/scan. This indicates that a combination of handcrafted features and CNN features from both lung CT images and enhanced images is a promising method for lung nodule detection.

For pediatric oncology patients, CT scans are performed to assess treatment response and disease progression. CAD may be used to detect lung nodules which would reflect metastatic disease. The purpose of this study was to investigate the effects of reducing radiation dose and varying slice thickness on CAD performance in the detection of solid lung nodules in pediatric patients. The dataset consisted of CT scans of 58 pediatric chest cases, from which 7 cases had lung nodules detected by radiologist, and a total of 28 nodules were marked. For each case, the original raw data (sinogram data) was collected and a noise addition model was used to simulate reduced-dose scans of 50%, 25% and 10% of the original dose. In addition, the original and reduced-dose raw data were reconstructed at slice thicknesses of 1.5 and 3 mm using a medium sharp (B45) kernel; the result was eight datasets (4 dose levels x 2 thicknesses) for each case An in-house CAD tool was applied on all reconstructed scans, and results were compared with the radiologist’s markings. Patient level mean sensitivities at 3mm thickness were 24%, 26%, 25%, 27%, and at 1.5 mm thickness were 23%, 29%, 35%, 36% for 10%, 25%, 50%, and 100% dose level, respectively. Mean FP numbers were 1.5, 0.9, 0.8, 0.7 at 3 mm and 11.4, 3.5, 2.8, 2.8 at 1.5 mm thickness for 10%, 25%, 50%, and 100% dose level respectively. CAD sensitivity did not change with dose level for 3mm thickness, but did change with dose for 1.5 mm. False Positives increased at low dose levels where noise values were high.

In this study, we propose a novel method of lung lesion detection in FDG-PET/CT volumes without labeling lesions. In our method, the probability distribution over normal standardized uptake values (SUVs) is estimated from the features extracted from the corresponding volume of interest (VOI) in the CT volume, which include gradient-based and texture-based features. To estimate the distribution, we use Gaussian process regression with an automatic relevance determination kernel, which provides the relevance of feature values to estimation. Our model was trained using FDG-PET/CT volumes of 121 normal cases. In the lesion detection phase, the actual SUV is judged as normal or abnormal by comparison with the estimated SUV distribution. According to the validation using 28 FDG-PET/CT volumes with 34 lung lesions, the sensitivity of the proposed method at 5.0 false positives per case was 81.9%.

Volumetric Laser Endomicroscopy (VLE) is a promising technique for the detection of early neoplasia in Barrett’s Esophagus (BE). VLE generates hundreds of high resolution, grayscale, cross-sectional images of the esophagus. However, at present, classifying these images is a time consuming and cumbersome effort performed by an expert using a clinical prediction model. This paper explores the feasibility of using computer vision techniques to accurately predict the presence of dysplastic tissue in VLE BE images. Our contribution is threefold. First, a benchmarking is performed for widely applied machine learning techniques and feature extraction methods. Second, three new features based on the clinical detection model are proposed, having superior classification accuracy and speed, compared to earlier work. Third, we evaluate automated parameter tuning by applying simple grid search and feature selection methods. The results are evaluated on a clinically validated dataset of 30 dysplastic and 30 non-dysplastic VLE images. Optimal classification accuracy is obtained by applying a support vector machine and using our modified Haralick features and optimal image cropping, obtaining an area under the receiver operating characteristic of 0.95 compared to the clinical prediction model at 0.81. Optimal execution time is achieved using a proposed mean and median feature, which is extracted at least factor 2.5 faster than alternative features with comparable performance.

We developed a novel electronic cleansing (EC) method for dual-energy CT colonography (DE-CTC) based on an ensemble deep convolution neural network (DCNN) and multi-spectral multi-slice image patches. In the method, an ensemble DCNN is used to classify each voxel of a DE-CTC image volume into five classes: luminal air, soft tissue, tagged fecal materials, and partial-volume boundaries between air and tagging and those between soft tissue and tagging. Each DCNN acts as a voxel classifier, where an input image patch centered at the voxel is generated as input to the DCNNs. An image patch has three channels that are mapped from a region-of-interest containing the image plane of the voxel and the two adjacent image planes. Six different types of spectral input image datasets were derived using two dual-energy CT images, two virtual monochromatic images, and two material images. An ensemble DCNN was constructed by use of a meta-classifier that combines the output of multiple DCNNs, each of which was trained with a different type of multi-spectral image patches. The electronically cleansed CTC images were calculated by removal of regions classified as other than soft tissue, followed by a colon surface reconstruction. For pilot evaluation, 359 volumes of interest (VOIs) representing sources of subtraction artifacts observed in current EC schemes were sampled from 30 clinical CTC cases. Preliminary results showed that the ensemble DCNN can yield high accuracy in labeling of the VOIs, indicating that deep learning of multi-spectral EC with multi-slice imaging could accurately remove residual fecal materials from CTC images without generating major EC artifacts.

Colorectal cancer (CRC) is one of the most common and deadliest forms of cancer, accounting for nearly 10% of all forms of cancer in the world. Even though colonoscopy is considered the most effective method for screening and diagnosis, the success of the procedure is highly dependent on the operator skills and level of hand-eye coordination. In this work, we propose to adapt fully convolution neural networks (FCN), to identify and segment polyps in colonoscopy images. We converted three established networks into a fully convolution architecture and fine-tuned their learned representations to the polyp segmentation task. We validate our framework on the 2015 MICCAI polyp detection challenge dataset, surpassing the state-of-the-art in automated polyp detection. Our method obtained high segmentation accuracy and a detection precision and recall of 73.61% and 86.31%, respectively.

Robust training of a deep convolutional neural network (DCNN) requires a very large number of annotated datasets that are currently not available in CT colonography (CTC). We previously demonstrated that deep transfer learning provides an effective approach for robust application of a DCNN in CTC. However, at high detection accuracy, the differentiation of small polyps from non-polyps was still challenging. In this study, we developed and evaluated a deep ensemble learning (DEL) scheme for reviewing of virtual endoluminal images to improve the performance of computer-aided detection (CADe) of polyps in CTC. Nine different types of image renderings were generated from virtual endoluminal images of polyp candidates detected by a conventional CADe system. Eleven DCNNs that represented three types of publically available pre-trained DCNN models were re-trained by transfer learning to identify polyps from the virtual endoluminal images. A DEL scheme that determines the final detected polyps by a review of the nine types of VE images was developed by combining the DCNNs using a random forest classifier as a meta-classifier. For evaluation, we sampled 154 CTC cases from a large CTC screening trial and divided the cases randomly into a training dataset and a test dataset. At 3.9 falsepositive (FP) detections per patient on average, the detection sensitivities of the conventional CADe system, the highestperforming single DCNN, and the DEL scheme were 81.3%, 90.7%, and 93.5%, respectively, for polyps ≥6 mm in size. For small polyps, the DEL scheme reduced the number of false positives by up to 83% over that of using a single DCNN alone. These preliminary results indicate that the DEL scheme provides an effective approach for improving the polyp detection performance of CADe in CTC, especially for small polyps.

Serrated polyps were previously believed to be benign lesions with no cancer potential. However, recent studies have revealed a novel molecular pathway where also serrated polyps can develop into colorectal cancer. CT colonography (CTC) can detect serrated polyps using the radiomic biomarker of contrast coating, but this requires expertise from the reader and current computer-aided detection (CADe) systems have not been designed to detect the contrast coating. The purpose of this study was to develop a novel CADe method that makes use of deep learning to detect serrated polyps based on their contrast-coating biomarker in CTC. In the method, volumetric shape-based features are used to detect polyp sites over soft-tissue and fecal-tagging surfaces of the colon. The detected sites are imaged using multi-angular 2D image patches. A deep convolutional neural network (DCNN) is used to review the image patches for the presence of polyps. The DCNN-based polyp-likelihood estimates are merged into an aggregate likelihood index where highest values indicate the presence of a polyp. For pilot evaluation, the proposed DCNN-CADe method was evaluated with a 10-fold cross-validation scheme using 101 colonoscopy-confirmed cases with 144 biopsy-confirmed serrated polyps from a CTC screening program, where the patients had been prepared for CTC with saline laxative and fecal tagging by barium and iodine-based diatrizoate. The average per-polyp sensitivity for serrated polyps ≥6 mm in size was 93±7% at 0:8±1:8 false positives per patient on average. The detection accuracy was substantially higher that of a conventional CADe system. Our results indicate that serrated polyps can be detected automatically at high accuracy in CTC.

Accurate identification of polyps is the ultimate goal of Computed Tomography Colonography (CTC). While oral contrast agents were originally used to tag stool and fluid for the ultimate goal of CTC, recently their effect on coating the surface of polyps has been observed. This study aims to evaluate (1) the frequency at which the oral contrast adhered to polyp surfaces and (2) if there was a difference in contrast adherence with respect to diverse polyp types. To eliminate gravity as a factor in this study, the polyps in contact with tagged fluid pools, particularly on the bottom of the colon wall were excluded. A total of 150 polyps were selected under the above condition from a CTC database and screened for any adherent contrast on the luminal edge. Among the total, 53% of the screened polyps had adherent contrast. Serrated adenomas and hyperplastic polyps had a higher tagging percentage, 77.80% and 62.50% respectively, than tubular adenomas and tubulovillous adenomas, 44.40% and 43% respectively. Other factors that were analyzed for the effect on coating include size and location of the polyps. The higher tagging percentage of serrated adenomas and hyperplastic polyps may be due to their similar cellular features. The average size of the polyps was 8.9 mm. When the polyps were separated by size into small (5-9mm) and large (10-26mm) groups, the large group had a higher tagging percentage. The polyp types were also classified by location with the major findings being: 1) Tubular adenomas were present in all segments of the colon and 2) that serrated adenomas were present at a higher percentage in the proximal colon. These findings shall facilitate characterizing tagging agents and improve computer aided detection and classification of polyps via CTC.

We propose a method for computer-assisted optical biopsy for colorectal polyps, with the final goal of assisting the medical expert during the colonoscopy. In particular, we target the problem of automatic classification of polyp images in two classes: adenomatous vs non-adenoma. Our approach is based on recent advancements in convolutional neural networks (CNN) for image representation. In the paper, we describe and compare four different methodologies to address the binary classification task: a baseline with classical features and a Random Forest classifier, two methods based on features obtained from a pre-trained network, and finally, the end-to-end training of a CNN. With the pre-trained network, we show the feasibility of transferring a feature extraction mechanism trained on millions of natural images, to the task of classifying adenomatous polyps. We then demonstrate further performance improvements when training the CNN for our specific classification task. In our study, 776 polyp images were acquired and histologically analyzed after polyp resection. We report a performance increase of the CNN-based approaches with respect to both, the conventional engineered features and to a state-of-the-art method based on videos and 3D shape features.

Cardiothoracic ratio (CTR) is a widely used radiographic index to assess heart size on chest X-rays (CXRs). Recent studies have suggested that also two-dimensional CTR might contain clinical information about the heart function. However, manual measurement of such indices is both subjective and time consuming. This study proposes a fast algorithm to automatically estimate CTR indices based on CXRs. The algorithm has three main steps: 1) model based lung segmentation, 2) estimation of heart boundaries from lung contours, and 3) computation of cardiothoracic indices from the estimated boundaries. We extended a previously employed lung detection algorithm to automatically estimate heart boundaries without using ground truth heart markings. We used two datasets: a publicly available dataset with 247 images as well as clinical dataset with 167 studies from Geisinger Health System. The models of lung fields are learned from both datasets. The lung regions in a given test image are estimated by registering the learned models to patient CXRs. Then, heart region is estimated by applying Harris operator on segmented lung fields to detect the corner points corresponding to the heart boundaries. The algorithm calculates three indices, CTR1D, CTR2D, and cardiothoracic area ratio (CTAR). The method was tested on 103 clinical CXRs and average error rates of 7.9%, 25.5%, and 26.4% (for CTR1D, CTR2D, and CTAR respectively) were achieved. The proposed method outperforms previous CTR estimation methods without using any heart templates. This method can have important clinical implications as it can provide fast and accurate estimate of cardiothoracic indices.

A fully automated computer algorithm has been developed to evaluate coronary artery calcification (CAC) from lowdose CT scans. CAC is identified and evaluated in three main coronary artery groups: Left Main and Left Anterior Descending Artery (LM + LAD) CAC, Left Circumflex Artery (LCX) CAC, and Right Coronary Artery (RCA) CAC. The artery labeling is achieved by segmenting all CAC candidates in the heart region and applying geometric constraints on the candidates using locally pre-identified anatomy regions. This algorithm was evaluated on 1,359 low-dose ungated CT scans, in which each artery CAC content was categorically visually scored by a radiologist into none, mild, moderate and extensive. The Spearman correlation coefficient R was used to assess the agreement between three automated CAC scores (Agatston-weighted, volume, and mass) and categorical visual scores. For Agatston-weighted automated scores, R was 0.87 for total CAC, 0.82 for LM + LAD CAC, 0.66 for LCX CAC and 0.72 for RCA CAC; results using volume and mass scores were similar. CAC detection sensitivities were: 0.87 for total, 0.82 for LM + LAD, 0.65 for LCX and 0.74 for RCA. To assess the impact of image noise, the dataset was further partitioned into three subsets based on heart region noise level (low<=80HU, medium=(80HU, 110HU], high>110HU). The low and medium noise subsets had higher sensitivities and correlations than the high noise subset. These results indicate that location specific heart risk assessment is possible from low-dose chest CT images.

Coronary artery calcification (CAC) is a typical marker of the coronary artery disease, which is one of the biggest causes of mortality in the U.S. This study evaluates the feasibility of using a deep convolutional neural network (DCNN) to automatically detect CAC in X-ray images. 1768 posteroanterior (PA) view chest X-Ray images from Sichuan Province Peoples Hospital, China were collected retrospectively. Each image is associated with a corresponding diagnostic report written by a trained radiologist (907 normal, 861 diagnosed with CAC). Onequarter of the images were randomly selected as test samples; the rest were used as training samples. DCNN models consisting of 2,4,6 and 8 convolutional layers were designed using blocks of pre-designed CNN layers. Each block was implemented in Theano with Graphics Processing Units (GPU). Human-in-the-loop learning was also performed on a subset of 165 images with framed arteries by trained physicians. The results from the DCNN models were compared to the diagnostic reports. The average diagnostic accuracies for models with 2,4,6,8 layers were 0.85, 0.87, 0.88, and 0.89 respectively. The areas under the curve (AUC) were 0.92, 0.95, 0.95, and 0.96. As the model grows deeper, the AUC or diagnostic accuracies did not have statistically significant changes. The results of this study indicate that DCNN models have promising potential in the field of intelligent medical image diagnosis practice.

Flow Doppler imaging is widely used by clinicians to detect diseases of the valves. In particular, continuous wave (CW) Doppler mode scan is routinely done during echocardiography and shows Doppler signal traces over multiple heart cycles. Traditionally, echocardiographers have manually traced such velocity envelopes to extract measurements such as decay time and pressure gradient which are then matched to normal and abnormal values based on clinical guidelines. In this paper, we present a fully automatic approach to deriving these measurements for aortic stenosis retrospectively from echocardiography videos. Comparison of our method with measurements made by echocardiographers shows large agreement as well as identification of new cases missed by echocardiographers.

Late Gadolinium-Enhanced Cardiac MRI (LGE CMRI) is an emerging non-invasive technique to image and quantify preablation native and post-ablation atrial scarring. Previous studies have reported that enhanced image intensities of the atrial scarring in the LGE CMRI inversely correlate with the left atrial endocardial voltage invasively obtained by electro-anatomical mapping. However, the reported reproducibility of using LGE CMRI to identify and quantify atrial scarring is variable. This may be due to two reasons: first, delineation of the left atrium (LA) and pulmonary veins (PVs) anatomy generally relies on manual operation that is highly subjective, and this could substantially affect the subsequent atrial scarring segmentation; second, simple intensity based image features may not be good enough to detect subtle changes in atrial scarring. In this study, we hypothesized that texture analysis can provide reliable image features for the LGE CMRI images subject to accurate and objective delineation of the heart anatomy based on a fully-automated whole heart segmentation (WHS) method. We tested the extracted texture features to differentiate between pre-ablation and post-ablation LGE CMRI studies in longstanding persistent atrial fibrillation patients. These patients often have extensive native scarring and differentiation from post-ablation scarring can be difficult. Quantification results showed that our method is capable of solving this classification task, and we can envisage further deployment of this texture analysis based method for other clinical problems using LGE CMRI.

Screening mammography is the most effective and low-cost method to date for early cancer detection. Mammographic breast density has been shown to be highly correlated with breast cancer risk. We are developing a radiomic model for BI-RADS density categorization on digital mammography (FFDM) with a supervised machine learning approach. With IRB approval, we retrospectively collected 478 FFDMs from 478 women. As a gold standard, breast density was assessed by an MQSA radiologist based on BI-RADS categories. The raw FFDMs were used for computerized density assessment. The raw FFDM first underwent log-transform to approximate the x-ray sensitometric response, followed by multiscale processing to enhance the fibroglandular densities and parenchymal patterns. Three ROIs were automatically identified based on the keypoint distribution, where the keypoints were obtained as the extrema in the image Gaussian scale-space. A total of 73 features, including intensity and texture features that describe the density and the parenchymal pattern, were extracted from each breast. Our BI-RADS density estimator was constructed by using a random forest classifier. We used a 10-fold cross validation resampling approach to estimate the errors. With the random forest classifier, computerized density categories for 412 of the 478 cases agree with radiologist’s assessment (weighted kappa = 0.93). The machine learning method with radiomic features as predictors demonstrated a high accuracy in classifying FFDMs into BI-RADS density categories. Further work is underway to improve our system performance as well as to perform an independent testing using a large unseen FFDM set.

We proposed the neutrosophic approach for segmenting breast lesions in breast Computer Tomography (bCT) images. The neutrosophic set (NS) considers the nature and properties of neutrality (or indeterminacy), which is neither true nor false. We considered the image noise as an indeterminate component, while treating the breast lesion and other breast areas as true and false components. We first transformed the image into the NS domain. Each voxel in the image can be described as its membership in True, Indeterminate, and False sets. Operations α-mean, β-enhancement, and γ-plateau iteratively smooth and contrast-enhance the image to reduce the noise level of the true set. Once the true image no longer changes, we applied one existing algorithm for bCT images, the RGI segmentation, on the resulting image to segment the breast lesions. We compared the segmentation performance of the proposed method (named as NS-RGI) to that of the regular RGI segmentation. We used a total of 122 breast lesions (44 benign, 78 malignant) of 123 non-contrasted bCT cases. We measured the segmentation performances of the NS-RGI and the RGI using the DICE coefficient. The average DICE value of the NS-RGI was 0.82 (STD: 0.09), while that of the RGI was 0.8 (STD: 0.12). The difference between the two DICE values was statistically significant (paired t test, p-value = 0.0007). We conducted a subsequent feature analysis on the resulting segmentations. The classifier performance for the NS-RGI (AUC = 0.8) improved over that of the RGI (AUC = 0.69, p-value = 0.006).

Breast cancer is the most common cancer diagnosed among US women and the second leading cause of cancer death 1 . Breast density is an independent risk factor for breast cancer and more than 25 states mandate its reporting to patients as part of the lay mammogram report ² . Recent publications have demonstrated that breast density measured from low-dose chest CT (LDCT) correlates well with that measured from mammograms and MRIs ^3-4 , thereby providing valuable information for many women who have undergone LDCT but not recent mammograms. A fully automated framework for breast density assessment from LDCT is presented in this paper. The whole breast region is first segmented using an anatomy-orientated novel approach based on the propagation of muscle fronts for separating the fibroglandular tissue from the underlying muscles. The fibroglandular tissue regions are then identified from the segmented whole breast and the percentage density is calculated based on the volume ratio of the fibroglandular tissue to the local whole breast region. The breast region segmentation framework was validated with 1270 LDCT scans, with 96.1% satisfactory outcomes based on visual inspection. The density assessment was evaluated by comparing with BI-RADS density grades established by an experienced radiologist in 100 randomly selected LDCT scans of female subjects. The continuous breast density measurement was shown to be consistent with the reference subjective grading, with the Spearman’s rank correlation 0.91 (p-value < 0.001). After converting the continuous density to categorical grades, the automated density assessment was congruous with the radiologist’s reading in 91% cases.

We assess the feasibility of a parenchymal texture feature fusion approach, utilizing a convolutional neural network (ConvNet) architecture, to benefit breast cancer risk assessment. Hypothesizing that by capturing sparse, subtle interactions between localized motifs present in two-dimensional texture feature maps derived from mammographic images, a multitude of texture feature descriptors can be optimally reduced to five meta-features capable of serving as a basis on which a linear classifier, such as logistic regression, can efficiently assess breast cancer risk. We combine this methodology with our previously validated lattice-based strategy for parenchymal texture analysis and we evaluate the feasibility of this approach in a case-control study with 424 digital mammograms. In a randomized split-sample setting, we optimize our framework in training/validation sets (N=300) and evaluate its descriminatory performance in an independent test set (N=124). The discriminatory capacity is assessed in terms of the the area under the curve (AUC) of the receiver operator characteristic (ROC). The resulting meta-features exhibited strong classification capability in the test dataset (AUC = 0.90), outperforming conventional, non-fused, texture analysis which previously resulted in an AUC=0.85 on the same case-control dataset. Our results suggest that informative interactions between localized motifs exist and can be extracted and summarized via a fairly simple ConvNet architecture.

Understanding the key radiogenomic associations for breast cancer between DCE-MRI and micro-RNA expressions is the foundation for the discovery of radiomic features as biomarkers for assessing tumor progression and prognosis. We conducted a study to analyze the radiogenomic associations for breast cancer using the TCGA-TCIA data set. The core idea that tumor etiology is a function of the behavior of miRNAs is used to build the regression models. The associations based on regression are analyzed for three study outcomes: diagnosis, prognosis, and treatment. The diagnosis group consists of miRNAs associated with clinicopathologic features of breast cancer and significant aberration of expression in breast cancer patients. The prognosis group consists of miRNAs which are closely associated with tumor suppression and regulation of cell proliferation and differentiation. The treatment group consists of miRNAs that contribute significantly to the regulation of metastasis thereby having the potential to be part of therapeutic mechanisms. As a first step, important miRNA expressions were identified and their ability to classify the clinical phenotypes based on the study outcomes was evaluated using the area under the ROC curve (AUC) as a figure-of-merit. The key mapping between the selected miRNAs and radiomic features were determined using least absolute shrinkage and selection operator (LASSO) regression analysis within a two-loop leave-one-out cross-validation strategy. These key associations indicated a number of radiomic features from DCE-MRI to be potential biomarkers for the three study outcomes.

DCE-MRI datasets have a temporal aspect to them, resulting in multiple regions of interest (ROIs) per subject, based on contrast time points. It is unclear how the different contrast time points vary in terms of usefulness for computer-aided diagnosis tasks in conjunction with deep learning methods. We thus sought to compare the different DCE-MRI contrast time points with regard to how well their extracted features predict response to neoadjuvant chemotherapy within a deep convolutional neural network. Our dataset consisted of 561 ROIs from 64 subjects. Each subject was categorized as a non-responder or responder, determined by recurrence-free survival. First, features were extracted from each ROI using a convolutional neural network (CNN) pre-trained on non-medical images. Linear discriminant analysis classifiers were then trained on varying subsets of these features, based on their contrast time points of origin. Leave-one-out cross validation (by subject) was used to assess performance in the task of estimating probability of response to therapy, with area under the ROC curve (AUC) as the metric. The classifier trained on features from strictly the pre-contrast time point performed the best, with an AUC of 0.85 (SD = 0.033). The remaining classifiers resulted in AUCs ranging from 0.71 (SD = 0.028) to 0.82 (SD = 0.027). Overall, we found the pre-contrast time point to be the most effective at predicting response to therapy and that including additional contrast time points moderately reduces variance.

Chromoendoscopy (CH) is a gastroenterology imaging modality that involves the staining of tissues with methylene blue, which reacts with the internal walls of the gastrointestinal tract, improving the visual contrast in mucosal surfaces and thus enhancing a doctor’s ability to screen precancerous lesions or early cancer. This technique helps identify areas that can be targeted for biopsy or treatment and in this work we will focus on gastric cancer detection. Gastric chromoendoscopy for cancer detection has several taxonomies available, one of which classifies CH images into three classes (normal, metaplasia, dysplasia) based on color, shape and regularity of pit patterns. Computer-assisted diagnosis is desirable to help us improve the reliability of the tissue classification and abnormalities detection. However, traditional computer vision methodologies, mainly segmentation, do not translate well to the specific visual characteristics of a gastroenterology imaging scenario. We propose the exploitation of a first unsupervised segmentation via superpixel, which groups pixels into perceptually meaningful atomic regions, used to replace the rigid structure of the pixel grid. For each superpixel, a set of features is extracted and then fed to a random forest based classifier, which computes a model used to predict the class of each superpixel. The average general accuracy of our model is 92.05% in the pixel domain (86.62% in the superpixel domain), while detection accuracies on the normal and abnormal class are respectively 85.71% and 95%. Eventually, the whole image class can be predicted image through a majority vote on each superpixel's predicted class.

Pancreatic cancer is the most lethal cancer with an overall 5-year survival rate of 7%¹ due to the late stage at diagnosis and the ineffectiveness of current therapeutic strategies. Given the poor prognosis, early detection at a pre-cancerous stage is the best tool for preventing this disease. Intraductal papillary mucinous neoplasms (IPMN), cystic tumors of the pancreas, represent the only radiographically identifiable precursor lesion of pancreatic cancer and are known to evolve stepwise from low-to-high-grade dysplasia before progressing into an invasive carcinoma. Observation is usually recommended for low-risk (low- and intermediate-grade dysplasia) patients, while high-risk (high-grade dysplasia and invasive carcinoma) patients undergo resection; hence, patient selection is critically important in the management of pancreatic cysts.² Radiologists use standard criteria such as main pancreatic duct size, cyst size, or presence of a solid enhancing component in the cyst to optimally select patients for surgery.³ However, these findings are subject to a radiologist’s interpretation and have been shown to be inconsistent with regards to the presence of a mural nodule or solid component.⁴ We propose objective classification of risk groups based on quantitative imaging features extracted from CT scans. We apply new features that represent the solid component (i.e. areas of high intensity) within the cyst and extract standard texture features. An adaptive boost classifier⁵ achieves the best performance with area under receiver operating characteristic curve (AUC) of 0.73 and accuracy of 77.3% for texture features. The random forest classifier achieves the best performance with AUC of 0.71 and accuracy of 70.8% with the solid component features.

Magnetic Resonance Elastography (MRE) is a phase-contrast MRI technique which calculates quantitative stiffness images, called elastograms, by imaging the propagation of acoustic waves in tissues. It is used clinically to diagnose liver fibrosis. Automated analysis of MRE is difficult as the corresponding MRI magnitude images (which contain anatomical information) are affected by intensity inhomogeneity, motion artifact, and poor tissue- and edge-contrast. Additionally, areas with low wave amplitude must be excluded. An automated algorithm has already been successfully developed and validated for clinical 2D MRE. 3D MRE acquires substantially more data and, due to accelerated acquisition, has exacerbated image artifacts. Also, the current 3D MRE processing does not yield a confidence map to indicate MRE wave quality and guide ROI selection, as is the case in 2D. In this study, extension of the 2D automated method, with a simple wave-amplitude metric, was developed and validated against an expert reader in a set of 57 patient exams with both 2D and 3D MRE. The stiffness discrepancy with the expert for 3D MRE was -0.8% ± 9.45% and was better than discrepancy with the same reader for 2D MRE (-3.2% ± 10.43%), and better than the inter-reader discrepancy observed in previous studies. There were no automated processing failures in this dataset. Thus, the automated liver elasticity calculation (ALEC) algorithm is able to calculate stiffness from 3D MRE data with minimal bias and good precision, while enabling stiffness measurements to be fully reproducible and to be easily performed on the large 3D MRE datasets.

Tumor volume measured from computed tomography images is considered a biomarker for disease progression or treatment response. The estimation of the tumor volume depends on the imaging system parameters selected, as well as lesion characteristics. In this study, we examined how different image reconstruction methods affect the measurement of lesions in an anthropomorphic liver phantom with a non-uniform background. Iterative statistics-based and model-based reconstructions, as well as filtered back-projection, were evaluated and compared in this study. Statistics-based and filtered back-projection yielded similar estimation performance, while model-based yielded higher precision but lower accuracy in the case of small lesions. Iterative reconstructions exhibited higher signal-to-noise ratio but slightly lower contrast of the lesion relative to the background. A better understanding of lesion volumetry performance as a function of acquisition parameters and lesion characteristics can lead to its incorporation as a routine sizing tool.

Hepatocellular carcinoma (HCC) is the most common liver cancer and the third leading cause of cancer-related death worldwide.¹ Resection or liver transplantation may be curative in patients with early-stage HCC but early recurrence is common.^{2, 3} Microvascular invasion (MVI) is one of the most important predictors of early recurrence.³ The identification of MVI prior to surgery would optimally select patients for potentially curative resection or liver transplant. However, MVI can only be diagnosed by microscopic assessment of the resected tumor. The aim of the present study is to apply CT-based texture analysis to identify pre-operative imaging predictors of MVI in patients with HCC. Texture features are derived from CT and analyzed individually as well as in combination, to evaluate their ability to predict MVI. A two-stage classification is employed: HCC tumors are automatically categorized into uniform or heterogenous groups followed by classification into the presence or absence of MVI. We achieve an area under the receiver operating characteristic curve (AUC) of 0.76 and accuracy of 76.7% for uniform lesions and AUC of 0.79 and accuracy of 74.06% for heterogeneous tumors. These results suggest that MVI can be accurately and objectively predicted from preoperative CT scans.

Knee Osteoarthritis (OA) is the most common form of arthritis, affecting millions of people around the world. The effects of the disease have been studied using the shape and texture features of bones in PosteriorAnterior (PA) and Lateral radiographs separately. In this work we compare the utility of features from each view, and evaluate whether combining features from both is advantageous. We built a fully automated system to independently locate landmark points in both radiographic images using Random Forest Constrained Local Models. We extracted discriminative features from the two bony outlines using Appearance Models. The features were used to train Random Forest classifiers to solve three specific tasks: (i) OA classification, distinguishing patients with structural signs of OA from the others; (ii) predicting future onset of the disease and (iii) predicting which patients with no current pain will have a positive pain score later in a follow-up visit. Using a subset of the MOST dataset we show that the PA view has more discriminative features to classify and predict OA, while the lateral view contains features that achieve better performance in predicting pain, and that combining the features from both views gives a small improvement in accuracy of the classification compared to the individual views.

The effectiveness of phase contrast X-ray computed tomography (PCI-CT) in visualizing human patellar cartilage matrix has been demonstrated due to its ability to capture soft tissue contrast on a micrometer resolution scale. Recent studies have shown that off-the-shelf Convolutional Neural Network (CNN) features learned from a nonmedical data set can be used for medical image classification. In this paper, we investigate the ability of features extracted from two different CNNs for characterizing chondrocyte patterns in the cartilage matrix. We obtained features from 842 regions of interest annotated on PCI-CT images of human patellar cartilage using CaffeNet and Inception-v3 Network, which were then used in a machine learning task involving support vector machines with radial basis function kernel to classify the ROIs as healthy or osteoarthritic. Classification performance was evaluated using the area (AUC) under the Receiver Operating Characteristic (ROC) curve. The best classification performance was observed with features from Inception-v3 network (AUC = 0.95), which outperforms features extracted from CaffeNet (AUC = 0.91). These results suggest that such characterization of chondrocyte patterns using features from internal layers of CNNs can be used to distinguish between healthy and osteoarthritic tissue with high accuracy.

Few studies have analyzed the microstructural properties of bone in cases of Osteogenenis Imperfecta (OI), or ‘brittle bone disease’. Current approaches mainly focus on bone mineral density measurements as an indirect indicator of bone strength and quality. It has been shown that bone strength would depend not only on composition but also structural organization. This study aims to characterize 3D structure of the cortical bone in high-resolution micro CT images. A total of 40 bone fragments from 28 subjects (13 with OI and 15 healthy controls) were imaged using micro tomography using a synchrotron light source (SRµCT). Minkowski functionals - volume, surface, curvature, and Euler characteristics - describing the topological organization of the bone were computed from the images. The features were used in a machine learning task to classify between healthy and OI bone. The best classification performance (mean AUC – 0.96) was achieved with a combined 4-dimensional feature of all Minkowski functionals. Individually, the best feature performance was seen using curvature (mean AUC - 0.85), which characterizes the edges within a binary object. These results show that quantitative analysis of cortical bone microstructure, in a computer-aided diagnostics framework, can be used to distinguish between healthy and OI bone with high accuracy.

Melanoma is considered a major health problem since it is the deadliest form of skin cancer. The early diagnosis through periodic screening with dermoscopic images can significantly improve the survival rate as well as reduce the treatment cost and consequent suffering of patients. Dermoscopy or skin surface microscopy provides in vivo inspection of color and morphologic structures of pigmented skin lesions (PSLs), rendering higher accuracy for detecting suspicious cases than it is possible via inspecting with naked eye. However, interpretation of dermoscopic images is time consuming and subjective, even for trained dermatologists. Therefore, there is currently a great interest in the development of computeraided diagnosis (CAD) systems for automated melanoma recognition. However, the majority of the CAD systems are still in the early development stage with lack of descriptive feature generation and benchmark evaluation in ground-truth datasets. This work is focusing on by addressing the various issues related to the development of such a CAD system with effective feature extraction from Non-Subsampled Contourlet Transform (NSCT) and Eig(Hess) histogram of oriented gradients (HOG) and lesion classification with efficient Extreme Learning Machine (ELM) due to its good generalization abilities and a high learning efficiency and evaluating its effectiveness in a benchmark data set of dermoscopic images towards the goal of realistic comparison and real clinical integration. The proposed research on melanoma recognition has huge potential for offering powerful services that would significantly benefit the present Biomedical Information Systems.

The abstract is not available

A convolutional network architecture is presented to determine bounding boxes around six organs in thoraxabdomen CT scans. A single network for each orthogonal view determines the presence of lungs, kidneys, spleen and liver. We show that an architecture that takes additional slices before and after the slice of interest as an additional input outperforms an architecture that processes single slices. From the slice-based analysis, a bounding box around the structures of interest can be computed. The system uses 6 convolutional, 4 pooling and one fully connected layer and uses 333 scans for training and 110 for validation. The test set contains 110 scans. The average Dice score of the proposed method was 0.95 and 0.95 for the lungs, 0.59 and 0.58 for the kidneys, 0.83 for the liver and 0.63 for the spleen. This paper shows that automatic localization of organs using multi-label convolution neural networks is possible. This architecture can likely be used to identify other organs of interest as well.

Image-derived features of breast parenchymal texture patterns have emerged as promising risk factors for breast cancer, paving the way towards personalized recommendations regarding women’s cancer risk evaluation and screening. The main steps to extract texture features of the breast parenchyma are the selection of regions of interest (ROIs) where texture analysis is performed, the texture feature calculation and the texture feature summarization in case of multiple ROIs. In this study, we incorporate breast anatomy in these three key steps by (a) introducing breast anatomical sampling for the definition of ROIs, (b) texture feature calculation aligned with the structure of the breast and (c) weighted texture feature summarization considering the spatial position and the underlying tissue composition of each ROI. We systematically optimize this novel framework for parenchymal tissue characterization in a case-control study with digital mammograms from 424 women. We also compare the proposed approach with a conventional methodology, not considering breast anatomy, recently shown to enhance the case-control discriminatory capacity of parenchymal texture analysis. The case-control classification performance is assessed using elastic-net regression with 5-fold cross validation, where the evaluation measure is the area under the curve (AUC) of the receiver operating characteristic. Upon optimization, the proposed breast-anatomy-driven approach demonstrated a promising case-control classification performance (AUC=0.87). In the same dataset, the performance of conventional texture characterization was found to be significantly lower (AUC=0.80, DeLong's test p-value<0.05). Our results suggest that breast anatomy may further leverage the associations of parenchymal texture features with breast cancer, and may therefore be a valuable addition in pipelines aiming to elucidate quantitative mammographic phenotypes of breast cancer risk.

The application of endovascular aortic aneurysm repair has expanded over the last decade. However, the long-term performance of stent grafts, in particular durable fixation and sealing to the aortic wall, remains the main concern of this treatment. The sealing and fixation are challenged at every heartbeat due to downward and radial pulsatile forces. Yet knowledge on cardiac-induced dynamics of implanted stent grafts is sparse, as it is not measured in routine clinical follow-up. Such knowledge is particularly relevant to perform fatigue tests, to predict failure in the individual patient and to improve stent graft designs. Using a physical dynamic stent graft model in an anthropomorphic phantom, we have evaluated the performance of our previously proposed segmentation and registration algorithm to detect periodic motion of stent grafts on ECG-gated (3D+t) CT data. Abdominal aortic motion profiles were simulated in two series of Gaussian based patterns with different amplitudes and frequencies. Experiments were performed on a 64-slice CT scanner with a helical scan protocol and retrospective gating. Motion patterns as estimated by our algorithm were compared to motion patterns obtained from optical camera recordings of the physical stent graft model in motion. Absolute errors of the patterns' amplitude were smaller than 0.28 mm. Even the motion pattern with an amplitude of 0.23 mm was measured, although the amplitude of motion was overestimated by the algorithm with 43%. We conclude that the algorithm performs well for measurement of stent graft motion in the mm and sub-mm range. This ultimately is expected to aid in patient-specific risk assessment and improving stent graft designs.

Hereditary hemorrhagic telangiectasia (HHT) is an autosomic dominant disorder, which is characterized by the development of multiple arterio-venous malformations in the skin, mucous membranes, and/or visceral organs. Pulmonary Arterio-Venous Malformation (PAVM) is an abnormal connection where feeding arteries shunt directly into draining veins with no intervening capillary bed. This condition may lead to paradoxical embolism and hemorrhagic complications. PAVMs patients should systematically be screened as the spontaneous complication rate is high, reaching almost 50%. Chest enhanced contrast CT scanner is the reference screening and follow-up examination. When performed by experienced operators as the prime treatment, percutaneous embolization of PAVMs is a safe, efficient and sustained therapy. The accuracy of PAVM detection and quantification of its progression over time is the key of embolotherapy success. In this paper, we propose an automatic method for PAVM detection and quantification relying on a modeling of vessel deformation, i.e. local caliber increase, based on mathematical morphology. The pulmonary field and vessels are first segmented using geodesic operators. The vessel caliber is estimated by means of a granulometric measure and the local caliber increase is detected by using a geodesic operator, the h-maxdomes. The detection sensitivity can be tuned up according to the choice of the h value which models the irregularity of the vessel caliber along its axis and the PAVM selection is performed according to a clinical criterion of >3 mm diameter of the feeding artery of the PAVM. The developed method was tested on a 20 patient dataset. A sensitivity study allowed choosing the irregularity parameter to maximize the true positive ratio reaching 85.4% in average. A specific false positive reduction procedure targeting the vessel trunks of the arterio-venous tree near mediastinum allows a precision increase from 13% to 67% with an average number of 1.15 false positives per scan.

Artery calcification is observed commonly in elderly patients, especially in patients with chronic kidney disease, and may affect coronary, carotid and peripheral arteries. Vascular calcification has been associated with many clinical outcomes. Manual identification of calcification in CT scans requires substantial expert interaction, which makes it time-consuming and infeasible for large-scale studies. Many works have been proposed for coronary artery calcification detection in cardiac CT scans. In these works, coronary artery extraction is commonly required for calcification detection. However, there are few works about abdominal or pelvic artery calcification detection. In this work, we present a method for automatic pelvic artery calcification detection on CT scan. This method uses the recent advanced faster region-based convolutional neural network (R-CNN) to directly identify artery calcification without a need for artery extraction since pelvic artery extraction itself is challenging. Our method first generates category-independent region proposals for each slice of the input CT scan using region proposal networks (RPN). Then, each region proposal is jointly classified and refined by softmax classifier and bounding box regressor. We applied the detection method to 500 images from 20 CT scans of patients for evaluation. The detection system achieved a 77.4% average precision and a 85% sensitivity at 1 false positive per image.

4D arterial spin labeling magnetic resonance angiography (4D ASL MRA) is a non-invasive and safe modality for cerebrovascular imaging procedures. It uses the patient’s magnetically labeled blood as intrinsic contrast agent, so that no external contrast media is required. It provides important 3D structure and blood flow information but a sufficient cerebrovascular segmentation is important since it can help clinicians to analyze and diagnose vascular diseases faster, and with higher confidence as compared to simple visual rating of raw ASL MRA images. This work presents a new method for automatic cerebrovascular segmentation in 4D ASL MRA images of the brain. In this process images are denoised, corresponding image label/control image pairs of the 4D ASL MRA sequences are subtracted, and temporal intensity averaging is used to generate a static representation of the vascular system. After that, sets of vessel and background seeds are extracted and provided as input for the image foresting transform algorithm to segment the vascular system. Four 4D ASL MRA datasets of the brain arteries of healthy subjects and corresponding time-of-flight (TOF) MRA images were available for this preliminary study. For evaluation of the segmentation results of the proposed method, the cerebrovascular system was automatically segmented in the high-resolution TOF MRA images using a validated algorithm and the segmentation results were registered to the 4D ASL datasets. Corresponding segmentation pairs were compared using the Dice similarity coefficient (DSC). On average, a DSC of 0.9025 was achieved, indicating that vessels can be extracted successfully from 4D ASL MRA datasets by the proposed segmentation method.

Neurosurgeons currently base most of their treatment decisions for intracranial aneurysms (IAs) on morphological measurements made manually from 2D angiographic images. These measurements tend to be inaccurate because 2D measurements cannot capture the complex geometry of IAs and because manual measurements are variable depending on the clinician’s experience and opinion. Incorrect morphological measurements may lead to inappropriate treatment strategies. In order to improve the accuracy and consistency of morphological analysis of IAs, we have developed an image-based computational tool, AView. In this study, we quantified the accuracy of computer-assisted adjuncts of AView for aneurysmal morphologic assessment by performing measurement on spheres of known size and anatomical IA models. AView has an average morphological error of 0.56% in size and 2.1% in volume measurement. We also investigate the clinical utility of this tool on a retrospective clinical dataset and compare size and neck diameter measurement between 2D manual and 3D computer-assisted measurement. The average error was 22% and 30% in the manual measurement of size and aneurysm neck diameter, respectively. Inaccuracies due to manual measurements could therefore lead to wrong treatment decisions in 44% and inappropriate treatment strategies in 33% of the IAs. Furthermore, computer-assisted analysis of IAs improves the consistency in measurement among clinicians by 62% in size and 82% in neck diameter measurement. We conclude that AView dramatically improves accuracy for morphological analysis. These results illustrate the necessity of a computer-assisted approach for the morphological analysis of IAs.

In this paper, we propose a novel supervoxel segmentation method designed for mediastinal lymph node by embedding Hessian-based feature extraction. Starting from a popular supervoxel segmentation method, SLIC, which computes supervoxels by minimising differences of intensity and distance, we overcome this method's limitation of merging neighboring regions with similar intensity by introducing Hessian-based feature analysis into the supervoxel formation. We call this structure-oriented voxel clustering, which allows more accurate division into distinct regions having blob-, line- or sheet-like structures. This way, different tissue types in chest CT volumes can be segmented individually, even if neighboring tissues have similar intensity or are of non- spherical extent. We demonstrate the performance of the Hessian-assisted supervoxel technique by applying it to mediastinal lymph node detection in 47 chest CT volumes, resulting in false positive reductions from lymph node candidate regions. 89 % of lymph nodes whose short axis is at least 10 mm could be detected with 5.9 false positives per case using our method, compared to our previous method having 83 % of detection rate with 6.4 false positives per case.

Recent breakthroughs in training deep neural network architectures, in particular deep Convolutional Neural Networks (CNNs), made a big impact on vision research and are increasingly responsible for advances in Computer Aided Diagnosis (CAD). Since many natural scenes and medical images vary in size and are too large to feed to the networks as a whole, two stage systems are typically employed, where in the first stage, small regions of interest in the image are located and presented to the network as training and test data. These systems allow us to harness accurate region based annotations, making the problem easier to learn. However, information is processed purely locally and context is not taken into account. In this paper, we present preliminary work on the employment of a Conditional Random Field (CRF) that is trained on top the CNN to model contextual interactions such as the presence of other suspicious regions, for mammography CAD. The model can easily be extended to incorporate other sources of information, such as symmetry, temporal change and various patient covariates and is general in the sense that it can have application in other CAD problems.

We investigated whether deep learning has potential to aid in the diagnosis of breast cancer when applied to mammograms and biologic tissue composition images derived from three-compartment (3CB) imaging. The dataset contained diagnostic mammograms and 3CB images (water, lipid, and protein content) of biopsy-sampled BIRADS 4 and 5 lesions in 195 patients. In 58 patients, the lesion manifested as a mass (13 malignant vs. 45 benign), in 87 as microcalcifications (19 vs. 68), and in 56 as (focal) asymmetry or architectural distortion (11 vs. 45). Six patients had both a mass and calcifications. For each mammogram and corresponding 3CB images, a 128x128 region of interest containing the lesion was selected by an expert radiologist and used directly as input to a deep learning method pretrained on a very large independent set of non-medical images. We used a nested leave-one-out-by-case (patient) model selection and classification protocol. The area under the ROC curve (AUC) for the task of distinguishing between benign and malignant lesions was used as performance metric. For the cases with mammographic masses, the AUC increased from 0.83 (mammograms alone) to 0.89 (mammograms+3CB, p=.162). For the microcalcification and asymmetry/architectural distortion cases the AUC increased from 0.84 to 0.91 (p=.116) and from 0.61 to 0.87 (p=.006), respectively. Our results indicate great potential for the application of deep learning methods in the diagnosis of breast cancer and additional knowledge of the biologic tissue composition appeared to improve performance, especially for lesions mammographically manifesting as asymmetries or architectural distortions.

Intuitive segmentation-based CADx/radiomic features, calculated from the lesion segmentations of dynamic contrast-enhanced magnetic resonance images (DCE-MRIs) have been utilized in the task of distinguishing between malignant and benign lesions. Additionally, transfer learning with pre-trained deep convolutional neural networks (CNNs) allows for an alternative method of radiomics extraction, where the features are derived directly from the image data. However, the comparison of computer-extracted segmentation-based and CNN features in MRI breast lesion characterization has not yet been conducted. In our study, we used a DCE-MRI database of 640 breast cases – 191 benign and 449 malignant. Thirty-eight segmentation-based features were extracted automatically using our quantitative radiomics workstation. Also, 2D ROIs were selected around each lesion on the DCE-MRIs and directly input into a pre-trained CNN AlexNet, yielding CNN features. Each method was investigated separately and in combination in terms of performance in the task of distinguishing between benign and malignant lesions. Area under the ROC curve (AUC) served as the figure of merit. Both methods yielded promising classification performance with round-robin cross-validated AUC values of 0.88 (se =0.01) and 0.76 (se=0.02) for segmentationbased and deep learning methods, respectively. Combination of the two methods enhanced the performance in malignancy assessment resulting in an AUC value of 0.91 (se=0.01), a statistically significant improvement over the performance of the CNN method alone.

Diagnostic interpretation of breast MRI studies requires meticulous work and a high level of expertise. Computerized algorithms can assist radiologists by automatically characterizing the detected lesions. Deep learning approaches have shown promising results in natural image classification, but their applicability to medical imaging is limited by the shortage of large annotated training sets. In this work, we address automatic classification of breast MRI lesions using two different deep learning approaches. We propose a novel image representation for dynamic contrast enhanced (DCE) breast MRI lesions, which combines the morphological and kinetics information in a single multi-channel image. We compare two classification approaches for discriminating between benign and malignant lesions: training a designated convolutional neural network and using a pre-trained deep network to extract features for a shallow classifier. The domain-specific trained network provided higher classification accuracy, compared to the pre-trained model, with an area under the ROC curve of 0.91 versus 0.81, and an accuracy of 0.83 versus 0.71. Similar accuracy was achieved in classifying benign lesions, malignant lesions, and normal tissue images. The trained network was able to improve accuracy by using the multi-channel image representation, and was more robust to reductions in the size of the training set. A small-size convolutional neural network can learn to accurately classify findings in medical images using only a few hundred images from a few dozen patients. With sufficient data augmentation, such a network can be trained to outperform a pre-trained out-of-domain classifier. Developing domain-specific deep-learning models for medical imaging can facilitate technological advancements in computer-aided diagnosis.

Predicting the risk of occult invasive disease in ductal carcinoma in situ (DCIS) is an important task to help address the overdiagnosis and overtreatment problems associated with breast cancer. In this work, we investigated the feasibility of using computer-extracted mammographic features to predict occult invasive disease in patients with biopsy proven DCIS. We proposed a computer-vision algorithm based approach to extract mammographic features from magnification views of full field digital mammography (FFDM) for patients with DCIS. After an expert breast radiologist provided a region of interest (ROI) mask for the DCIS lesion, the proposed approach is able to segment individual microcalcifications (MCs), detect the boundary of the MC cluster (MCC), and extract 113 mammographic features from MCs and MCC within the ROI. In this study, we extracted mammographic features from 99 patients with DCIS (74 pure DCIS; 25 DCIS plus invasive disease). The predictive power of the mammographic features was demonstrated through binary classifications between pure DCIS and DCIS with invasive disease using linear discriminant analysis (LDA). Before classification, the minimum redundancy Maximum Relevance (mRMR) feature selection method was first applied to choose subsets of useful features. The generalization performance was assessed using Leave-One-Out Cross-Validation and Receiver Operating Characteristic (ROC) curve analysis. Using the computer-extracted mammographic features, the proposed model was able to distinguish DCIS with invasive disease from pure DCIS, with an average classification performance of AUC = 0.61 ± 0.05. Overall, the proposed computer-extracted mammographic features are promising for predicting occult invasive disease in DCIS.

When humans identify objects in images, context is an important cue; a cheetah is more likely to be a domestic cat when a television set is recognised in the background. Similar principles apply to the analysis of medical images. The detection of diseases that manifest unilaterally in symmetrical organs or organ pairs can in part be facilitated by a search for symmetrical discrepancies in or between the organs in question. During a mammographic exam, images are recorded of each breast and absence of a certain structure around the same location in the contralateral image will render the area under scrutiny more suspicious and conversely, the presence of similar tissue less so. In this paper, we present a fusion scheme for a deep Convolutional Neural Network (CNN) architecture with the goal to optimally capture such asymmetries. The method is applied to the domain of mammography CAD, but can be relevant to other medical image analysis tasks where symmetry is important such as lung, prostate or brain images.

Retinal vessel caliber changes are associated with several major diseases, such as diabetes and hypertension. These caliber changes can be evaluated using eye fundus images. However, the clinical assessment is tiresome and prone to errors, motivating the development of automatic methods. An automatic method based on vessel crosssection intensity profile model fitting for the estimation of vessel caliber in retinal images is herein proposed. First, vessels are segmented from the image, vessel centerlines are detected and individual segments are extracted and smoothed. Intensity profiles are extracted perpendicularly to the vessel, and the profile lengths are determined. Then, model fitting is applied to the smoothed profiles. A novel parametric model (DoG-L7) is used, consisting on a Difference-of-Gaussians multiplied by a line which is able to describe profile asymmetry. Finally, the parameters of the best-fit model are used for determining the vessel width through regression using ensembles of bagged regression trees with random sampling of the predictors (random forests). The method is evaluated on the REVIEW public dataset. A precision close to the observers is achieved, outperforming other state-of-the-art methods. The method is robust and reliable for width estimation in images with pathologies and artifacts, with performance independent of the range of diameters.