Digital subtraction angiography (DSA) is routinely used for measuring the dimensions and characteristics of cerebral aneurysms as a step in planning of interventional treatments. Incorrect sizing of the aneurysm sac puts the patient at the risk of incomplete treatment due to the use of an intrasaccular implant that is too small or too large. In this work, we propose an automatic method to segment the aneurysm sac in 2D DSA images to enable fast and accurate measurements. We use a UNet-like architecture. However, we replace the encoder arm of this network with an EfficientNet architecture, pre-trained on 300 million natural images. We show that this architecture delivers very accurate segmentation of the aneurysm sac on a dataset of 144 DSA images obtained from patients prior to implantation of an intrasaccular device to treat wide-neck bifurcation aneurysms. We report a Dice coefficient of 0.9.
Basic deep learning classifiers used for medical images often produce global labels. While annotation for localized disease detection might be costly, the knowledge of prevalence of conditions in different anatomical areas can help improve the accuracy by limiting the classifier to relevant areas. However, this improvement provided by context knowledge, is usually offset by the errors of the segmentation map used to isolate the area of interest. This paper proposes a framework for disease classification consisting of a segmentation network, a segmentation quality assessment network, and two separate classifiers on whole image and relevant segmented area. The quality assessment network controls the impact of the two disease classifiers on the final outcome, utilizing the masked image only when segmentation is acceptable. We show that in a very large dataset of chest X-ray images, this framework produces a 2% increase in the area under ROC curve for classification compared to a baseline.
Decision support systems built for radiologists need to cover a fairly wide range of image types, with the ability to route each image to the relevant algorithm. Furthermore, the training of such networks requires building large datasets with significant efforts in image curation. In situations where the DICOM tag of an image is unavailable, or unreliable, a classifier that can automatically detect the body part depicted in the image, as well as the imaging modality, is necessary. Previous work has shown the use of imaging and textual features to distinguish between imaging modalities. In this work, we present a model for the simultaneous classification of body part and imaging modality, which to our knowledge has not been done before, as part of the larger work to create a cognitive assistant for radiologists. This classification network consists of 10 classes built from a VGG network architecture using transfer learning to learn generic features. An accuracy of 94.8% is achieved.
Age prediction based on appearances of different anatomies in medical images has been clinically explored for many decades. In this paper, we used deep learning to predict a person’s age on Chest X-Rays. Specifically, we trained a CNN in regression fashion on a large publicly available dataset. Moreover, for interpretability, we explored activation maps to identify which areas of a CXR image are important for the machine (i.e. CNN) to predict a patient’s age, offering insight. Overall, amongst correctly predicted CXRs, we see areas near the clavicles, shoulders, spine and mediastinum being most activated for age prediction, as one would expect biologically. As CXR is the most commonly requested imaging exam, a potential use case for estimating age may be found in the preventative counselling of patient health status compared to their age-expected average, particularly when there is a large discrepancy between predicted age and the real patient age.
Chest X-rays are among the most common modalities in medical imaging. Technical flaws of these images, such as over- or under-exposure or wrong positioning of the patients can result in a decision to reject and repeat the scan. We propose an automatic method to detect images that are not suitable for diagnostic study. If deployed at the point of image acquisition, such a system can warn the technician, so the repeat image is acquired without the need to bring the patient back to the scanner. We use a deep neural network trained on a dataset of 3487 images labeled by two experienced radiologists to classify the images as diagnostic or non-diagnostic. The DenseNet121 architecture is used for this classification task. The trained network has an area under the receiver operator curve (AUC) of 0.93. By removing the X-rays with diagnostic quality issues, this technology could potentially provide significant cost savings for hospitals.
Chest X-rays (CXRs) are among the most commonly used medical image modalities. They are mostly used for screening, and an indication of disease typically results in subsequent tests. As this is mostly a screening test used to rule out chest abnormalities, the requesting clinicians are often interested in whether a CXR is normal or not. A machine learning algorithm that can accurately screen out even a small proportion of the “real normal” exams out of all requested CXRs would be highly beneficial in reducing the workload for radiologists. In this work, we report a deep neural network trained for classifying CXRs with the goal of identifying a large number of normal (disease-free) images without risking the discharge of sick patients. We use an ImageNet-pretrained Inception-ResNet-v2 model to provide the image features, which are further used to train a model on CXRs labelled by expert radiologists. The probability threshold for classification is optimized for 100% precision for the normal class, ensuring no sick patients are released. At this threshold we report an average recall of 50%. This means that the proposed solution has the potential to cut in half the number of disease-free CXRs examined by radiologists, without risking the discharge of sick patients.
Medical imaging datasets are limited in size due to privacy issues and the high cost of obtaining annotations. Augmentation is a widely used practice in deep learning to enrich the data in data-limited scenarios and to avoid overfitting. However, standard augmentation methods that produce new examples of data by varying lighting, field of view, and spatial rigid transformations do not capture the biological variance of medical imaging data and could result in unrealistic images. Generative adversarial networks (GANs) provide an avenue to understand the underlying structure of image data which can then be utilized to generate new realistic samples. In this work, we investigate the use of GANs for producing chest X-ray images to augment a dataset. This dataset is then used to train a convolutional neural network to classify images for cardiovascular abnormalities. We compare our augmentation strategy with traditional data augmentation and show higher accuracy for normal vs abnormal classification in chest X-rays.
Segmenting anatomical structures in the chest is a crucial step in many automatic disease detection applications. Multi-atlas based methods are developed for this task, however, due to the required deformable registration step, they are often computationally expensive and create a bottle neck in terms of processing time. In contrast, convolutional neural networks (CNNs) with 2D or 3D kernels, although slow to train, are very fast in the deployment stage and have been employed to solve segmentation tasks in medical imaging. A recent improvement in performance of neural networks in medical image segmentation was recently reported when dice similarity coefficient (DSC) was used to optimize the weights in a fully convolutional architecture called V-Net. However, in the previous work, only the DSC calculated for one foreground object is optimized, as a result the DSC based segmentation CNNs are only able to perform a binary segmentation. In this paper, we extend the V-Net binary architecture to a multi-label segmentation network and use it for segmenting multiple anatomical structures in cardiac CTA. The method uses multi-label V-Net optimized by the sum over DSC for all the anatomies, followed by a post-processing method to refine the segmented surface. Our method takes averagely less than 3 sec to segment a full CTA volume. In contrast, the fastest multi-atlas based methods published so far take around 10 mins. Our method achieves an average DSC of 76% for 16 segmented anatomies using four-fold cross validation, which is close to the state-of-the-art.
Flow Doppler imaging is widely used by clinicians to detect diseases of the valves. In particular, continuous wave (CW) Doppler mode scan is routinely done during echocardiography and shows Doppler signal traces over multiple heart cycles. Traditionally, echocardiographers have manually traced such velocity envelopes to extract measurements such as decay time and pressure gradient which are then matched to normal and abnormal values based on clinical guidelines. In this paper, we present a fully automatic approach to deriving these measurements for aortic stenosis retrospectively from echocardiography videos. Comparison of our method with measurements made by echocardiographers shows large agreement as well as identification of new cases missed by echocardiographers.
T. Syeda-Mahmood, E. Walach, D. Beymer, F. Gilboa-Solomon, M. Moradi, P. Kisilev, D. Kakrania, C. Compas, H. Wang, R. Negahdar, Y. Cao, T. Baldwin, Y. Guo, Y. Gur, D. Rajan, A. Zlotnick, S. Rabinovici-Cohen, R. Ben-Ari, Amit Guy, P. Prasanna, J. Morey, O. Boyko, S. Hashoul
Radiologists and cardiologists today have to view large amounts of imaging data relatively quickly leading to eye fatigue. Further, they have only limited access to clinical information relying mostly on their visual interpretation of imaging studies for their diagnostic decisions. In this paper, we present Medical Sieve, an automated cognitive assistant for radiologists and cardiologists designed to help in their clinical decision-making. The sieve is a clinical informatics system that collects clinical, textual and imaging data of patients from electronic health records systems. It then analyzes multimodal content to detect anomalies if any, and summarizes the patient record collecting all relevant information pertinent to a chief complaint. The results of anomaly detection are then fed into a reasoning engine which uses evidence from both patient-independent clinical knowledge and large-scale patient-driven similar patient statistics to arrive at potential differential diagnosis to help in clinical decision making. In compactly summarizing all relevant information to the clinician per chief complaint, the system still retains links to the raw data for detailed review providing holistic summaries of patient conditions. Results of clinical studies in the domains of cardiology and breast radiology have already shown the promise of the system in differential diagnosis and imaging studies summarization.
This work evaluates the performance of a multi-stage image enhancement, segmentation, and classification approach for lymphoma recognition in hematoxylin and eosin (H and E) stained histopathology slides of excised human lymph node tissue. In the first stage, the original histology slide undergoes various image enhancement and segmentation operations, creating an additional 5 images for every slide. These new images emphasize unique aspects of the original slide, including dominant staining, staining segmentations, non-cellular groupings, and cellular groupings. For the resulting 6 total images, a collection of visual features are extracted from 3 different spatial configurations. Visual features include the first fully connected layer (4096 dimensions) of the Caffe convolutional neural network trained from ImageNet data. In total, over 200 resultant visual descriptors are extracted for each slide. Non-linear SVMs are trained over each of the over 200 descriptors, which are then input to a forward stepwise ensemble selection that optimizes a late fusion sum of logistically normalized model outputs using local hill climbing. The approach is evaluated on a public NIH dataset containing 374 images representing 3 lymphoma conditions: chronic lymphocytic leukemia (CLL), follicular lymphoma (FL), and mantle cell lymphoma (MCL). Results demonstrate a 38.4% reduction in residual error over the current state-of-art on this dataset.
The combination of Dynamic Contrast Enhanced (DCE) images with diffusion MRI has shown great potential in prostate cancer detection. The parameterization of DCE images to generate cancer markers is traditionally performed based on pharmacokinetic modeling. However, pharmacokinetic models make simplistic assumptions about the tissue perfusion process, require the knowledge of contrast agent concentration in a major artery, and the modeling process is sensitive to noise and fitting instabilities. We address this issue by extracting features directly from the DCE T1-weighted time course without modeling. In this work, we employed a set of data-driven features generated by mapping the DCE T1 time course to its principal component space, along with diffusion MRI features to detect prostate cancer. The optimal set of DCE features is extracted with sparse regularized regression through a Least Absolute Shrinkage and Selection Operator (LASSO) model. We show that when our proposed features are used within the multiparametric MRI protocol to replace the pharmacokinetic parameters, the area under ROC curve is 0.91 for peripheral zone classification and 0.87 for whole gland classification. We were able to correctly classify 32 out of 35 peripheral tumor areas identified in the data when the proposed features were used with support vector machine classification. The proposed feature set was used to generate cancer likelihood maps for the prostate gland.
Prostate cancer is the most prevalently diagnosed and the second cause of cancer-related death in North American men. Several approaches have been proposed to augment detection of prostate cancer using different imaging modalities. Due to advantages of ultrasound imaging, these approaches have been the subject of several recent studies. This paper presents the results of a feasibility study on differentiating between lower and higher grade prostate cancer using ultrasound RF time series data. We also propose new spectral features of RF time series to highlight aggressive prostate cancer in small ROIs of size 1 mm × 1 mm in a cohort of 19 ex vivo specimens of human prostate tissue. In leave-one-patient-out cross-validation strategy, an area under accumulated ROC curve of 0.8 has been achieved with overall sensitivity and specificity of 81% and 80%, respectively. The current method shows promising results on differentiating between lower and higher grade of prostate cancer using ultrasound RF time series.
The common practice for biopsy guidance is through transrectal ultrasound, with the fusion of ultrasound and MRI-based targets when available. However, ultrasound is only used as a guidance modality in MR-targeted ultrasound-guided biopsy, even though previous work has shown the potential utility of ultrasound, particularly ultrasound vibro-elastography, as a tissue typing approach. We argue that multiparametric ultrasound, which includes B-mode and vibro-elastography images, could contain information that is not captured using multiparametric MRI (mpMRI) and therefore play a role in refining the biopsy and treatment strategies. In this work, we combine mpMRI with multiparametric ultrasound features from registered tissue areas to examine the potential improvement in cancer detection. All the images were acquired prior to radical prostatectomy and cancer detection was validated based on 36 whole mount histology slides. We calculated a set of 24 texture features from vibro-elastography and B-mode images, and five features from mpMRI. Then we used recursive feature elimination (RFE) and sparse regression through LASSO to find an optimal set of features to be used for tissue classification. We show that the set of these selected features increases the area under ROC curve from 0.87 with mpMRI alone to 0.94 with the selected mpMRI and multiparametric ultrasound features, when used with support vector machine classification on features extracted from peripheral zone. For features extracted from the whole-gland, the area under the curve was 0.75 and 0.82 for mpMRI and mpMRI along with ultrasound, respectively. These preliminary results provide evidence that ultrasound and ultrasound vibro-elastography could be used as modalities for improved cancer detection in combination with MRI.
Magnetic resonance imaging (MRI), particularly dynamic contrast enhanced (DCE) imaging, has shown great potential in prostate cancer diagnosis and prognosis. The time course of the DCE images provides measures of the contrast agent uptake kinetics. Also, using pharmacokinetic modelling, one can extract parameters from the DCE-MR images that characterize the tumor vascularization and can be used to detect cancer. A requirement for calculating the pharmacokinetic DCE parameters is estimating the Arterial Input Function (AIF). One needs an accurate segmentation of the cross section of the external femoral artery to obtain the AIF. In this work we report a semi-automatic method for segmentation of the cross section of the femoral artery, using circular Hough transform, in the sequence of DCE images. We also report a machine-learning framework to combine pharmacokinetic parameters with the model-free contrast agent uptake kinetic parameters extracted from the DCE time course into a nine-dimensional feature vector. This combination of features is used with random forest and with support vector machine classi cation for cancer detection. The MR data is obtained from patients prior to radical prostatectomy. After the surgery, wholemount histopathology analysis is performed and registered to the DCE-MR images as the diagnostic reference. We show that the use of a combination of pharmacokinetic parameters and the model-free empirical parameters extracted from the time course of DCE results in improved cancer detection compared to the use of each group of features separately. We also validate the proposed method for calculation of AIF based on comparison with the manual method.
KEYWORDS: Magnetic resonance imaging, Ultrasonography, Uterus, Tumors, Image registration, Cervical cancer, 3D modeling, 3D image processing, Tissues, Cancer
MRI and Computed Tomography (CT) are used in image-based solutions for guiding High Dose Rate (HDR) brachytherapy treatment of cervical cancer. MRI is costly and CT exposes the patients to ionizing radiation. Ultrasound, on the other hand, is affordable and safe. The long-term goal of our work is to enable the use of multiparametric ultrasound imaging in image-guided HDR for cervical cancer. In this paper, we report the development of enabling technology for ultrasound guidance and tissue typing. We report a system to obtain the 3D freehand transabdominal ultrasound RF signals and B-mode images of the uterus, and a method for registration of ultrasound to MRI. MRI and 3D ultrasound images of the female pelvis were registered by contouring the uterus in the two modalities, creating a surface model, followed by rigid and B-spline deformable registration. The resulting transformation was used to map the location of the tumor from the T2-weighted MRI to ultrasound images and to determine cancerous and normal areas in ultrasound. B-mode images show a contrast for cancer vs. normal tissue. Our study shows the potential and the challenges of ultrasound imaging in guiding cervical cancer treatments.
A new approach is proposed for edge detection in ultrasound. The technique examines the image intensity profile for unit roots based on the Dickey–Fuller statistical test of stationarity. The existence of the unit root is a sign of nonstationarity and a possible edge. A simple algorithm to build a segmentation method based on this edge detection approach is also proposed, which is capable of delineating the perimeter of hollow structures such as blood vessels and cysts. In this approach, the radial edge profiles originating from the center of the object of interest are scanned for the change from stationary to nonstationary status. The algorithm treats the radial intensity profiles as a time series and uses the Dickey–Fuller statistical test along the radii to find the location at which the profile becomes nonstationary. A priori criteria for edge continuity, shape, and size of the object of interest are applied to enhance the stability of the algorithm. The accuracy is demonstrated on simulated ultrasound. Further, the method is examined on two different image sets of blood vessels and validated based on contours marked by experts. The worst case distance from expert contours is 1.8±0.3 mm . The average area difference between the expert and the extracted contours is ∼6% and ∼4% of the vessel area in the two datasets. The proposed segmentation method is also compared to segmentation using active contours on ultrasound images of breast and ovarian cysts and shown to be accurate and stable.
The low diagnostic value of ultrasound in prostate cancer imaging has resulted in an effort to enhance the tumor contrast using ultrasound-based technologies that go beyond traditional B-mode imaging. Ultrasound RF time series, formed by echo samples originating from the same location over a few seconds of imaging, has been proposed and experimentally used for tissue typing with the goal of cancer detection. In this work, for the first time we report the preliminary results of in vivo clinical use of spectral parameters extracted from RF time series in prostate cancer detection. An image processing pipeline is designed to register the ultrasound data to wholemount histopathology references acquired from prostate specimens that are removed in radical prostatectomy after imaging. Support vector machine classification is used to detect cancer in 524 regions of interest of size 5×5 mm, each forming a feature vector of spectral RF time series parameters. Preliminary ROC curves acquired based on RF time series analysis for individual cases, with leave-one-patient-out cross validation, are presented and compared with B-mode texture analysis.
Time series analysis of ultrasound radio-frequency (RF) signals has been shown to be an effective tissue classification
method. Previous studies of this method for tissue differentiation at high and clinical-frequencies have been reported. In
this paper, analysis of RF time series is extended to improve tissue classification at the clinical frequencies by including
novel features extracted from the time series spectrum. The primary feature examined is the Mean Central Frequency
(MCF) computed for regions of interest (ROIs) in the tissue extending along the axial axis of the transducer. In addition,
the intercept and slope of a line fitted to the MCF-values of the RF time series as a function of depth have been included.
To evaluate the accuracy of the new features, an in vitro animal study is performed using three tissue types: bovine
muscle, bovine liver, and chicken breast, where perfect two-way classification is achieved. The results show statistically
significant improvements over the classification accuracies with previously reported features.
Markov Random Fields (MRFs) provide a tractable means for incorporating contextual information into a Bayesian framework.
This contextual information is modeled using multiple local conditional probability density functions (LCPDFs)
which the MRF framework implicitly combines into a single joint probability density function (JPDF) that describes the
entire system. However, only LCPDFs of certain functional forms are consistent, meaning they reconstitute a valid JPDF.
These forms are specified by the Gibbs-Markov equivalence theorem which indicates that the JPDF, and hence the LCPDFs,
should be representable as a product of potential functions (i.e. Gibbs distributions). Unfortunately, potential functions
are mathematical abstractions that lack intuition; and consequently, constructing LCPDFs through their selection becomes
an ad hoc procedure, usually resulting in generic and/or heuristic models. In this paper we demonstrate that under certain
conditions the LCDPFs can be formulated in terms of quantities that are both meaningful and descriptive: probability distributions.
Using probability distributions instead of potential functions enables us to construct consistent LCPDFs whose
modeling capabilities are both more intuitive and expansive than typical MRF models. As an example, we compare the efficacy
of our so-called probabilistic pairwise Markov models (PPMMs) to the prevalent Potts model by incorporating both
into a novel computer aided diagnosis (CAD) system for detecting prostate cancer in whole-mount histological sections.
Using the Potts model the CAD system is able to detection cancerous glands with a specificity of 0.82 and sensitivity of
0.71; its area under the receiver operator characteristic (AUC) curve is 0.83. If instead the PPMM model is employed the
sensitivity (specificity is held fixed) and AUC increase to 0.77 and 0.87.
We report phantom studies on a new approach to ultrasound-based tissue typing. In the proposed approach,
we continuously record RF echo signals backscattered from tissue, while the imaging probe and the tissue are
fixed in position. The continuously recorded RF data generates a time series of echoes for each spatial sample
of the RF signal. We use the spectral and fractal features of ultrasound RF time series averaged over a region
of interest, along with support vector machine classifiers, for tissue typing. In this paper, the effects of two
properties of tissue on RF time series are investigated: cell size and elasticity. We show that RF time series
acquired from agar-gelatin based tissue mimicking phantoms, with difference only in the size of cell-mimicking
glass beads, are distinguishable with statistically reliable accuracies up to 82.2%. Similar experiments using
phantoms with different elastic properties did not result in consistently high classification accuracies. The
results of this study confirm that the evident differences in microstructure of the cancerous versus normal
tissue could play a role in the success of the proposed tissue typing method in detection of prostate cancer.
The aim of this research was to investigate the performance of wavelet transform based features of ultrasound
radiofrequency (RF) time series for automated detection of prostate cancer tumors in transrectal ultrasound images.
Sequential frames of RF echo signals from 35 extracted prostate specimens were recorded in parallel planes, while the
ultrasound probe and the tissue were fixed in position in each imaging plane. The sequence of RF echo signal samples
corresponding to a particular spot in tissue imaging plane constitutes one RF time series. Each region of interest (ROI) of
ultrasound image was represented by three groups of features of its time series, namely, wavelet, spectral and fractal
features.
Wavelet transform approximation and detail sequences of each ROI were averaged and used as wavelet features. The
average value of the normalized spectrum in four quarters of the frequency range along with the intercept and slope of a
regression line fitted to the values of the spectrum versus normalized frequency plot formed six spectral features. Fractal
dimension (FD) of the RF time series were computed based on the Higuchi's approach. A support vector machine
(SVM) classifier was used to classify the ROIs.
The results indicate that combining wavelet coefficient based features with previously proposed spectral and fractal
features of RF time series data would increase the area under ROC curve from 93.1% to 95.0%, respectively.
Furthermore, the accuracy, sensitivity, and specificity increases to 91.7%, 86.6%, and 94.7%, from 85.7%, 85.2%, and
86.1%, respectively, using only spectral and fractal features.
We present the results of an animal tissue characterization study to
demonstrate the effectiveness of a novel approach in collecting and
analyzing ultrasound echo signals. In this approach, we continuously
record RF echo signals backscattered from a tissue sample, while the
imaging probe and the tissue are fixed in position. The continuously
recorded RF data generates a time series of RF signal samples. The
Higuchi fractal dimension of the resulting time series at each
spatial coordinate of the RF frame, averaged over a region of
interest, serves as our tissue characterizing feature. The proposed
feature is used along with Bayesian classifiers and feed-forward
neural networks to distinguish different types of animal tissue.
Pairwise classification of four different types of animal tissue are
performed. Accuracies are in the range of 68%-96% and are significantly higher than the natural split of the data. The promising results of this study show that analysis of RF time series as proposed here, can potentially give rise to effective measures for ultrasound-based tissue characterization.
KEYWORDS: Image registration, Ultrasonography, Magnetic resonance imaging, Medical imaging, Brain, Image segmentation, Neuroimaging, Image processing, Tissues, Signal to noise ratio
In this paper, we describe a new methodology for keypoint-based affine and deformable medical image registration. This fast and computationally efficient method is automatic and does not rely on segmentation of images. The keypoint pixels used in this technique are extreme points in the scale space and are characterized by descriptor vectors which summarize the intensity gradient profile of the surrounding pixels. For each of the keypoints in the scene image, a corresponding keypoint is identified in the model image using the feature space nearest neighbor criteria. For deformable registration, B-splines are used to extrapolate a regular deformation grid for all of the pixels in the scene image based on the relative displacement vectors of the corresponding pairs. This approach results in a fast and accurate registration in the brain MRI images (an average target registration error of less than 2mm was acquired). We have also studied the affine registration problem in the liver ultrasound and brain MRI images and have acquired acceptable registrations using a mean square solution for affine parameters based on only around 30 corresponding keypoint pairs.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.