In this article, we show how to extend the multi-view stereo technique when the object to be reconstructed is inside a transparent but refractive medium, which causes distortions in the images. We provide a theoretical formulation of the problem accounting for a general non-planar shape of the refractive interface, and then a discrete solving method. We also present a pipeline to recover precisely the geometry of the refractive interface, considered as a convex polyhedral object. It is based on the extraction of visible polyhedron vertices from silhouette images and matching across a sequence of images acquired under circular camera motion. These contributions are validated by tests on synthetic and real data.
We present a pipeline to recover precisely the geometry of a convex polyhedral object from multiple views under circular motion. It is based on the extraction of visible polyhedron vertices from silhouette images and matching across a sequence of images. Compared to standard structure-from-motion pipelines, the method is well suited to the 3D-reconstruction of low-textured and non-Lambertian materials. Experiments on synthetic and real datasets show the efficacy of the proposed framework.
PurposeApplying machine learning techniques to magnetic resonance diffusion-weighted imaging (DWI) data is challenging due to the size of individual data samples and the lack of labeled data. It is possible, though, to learn general patterns from a very limited amount of training data if we take advantage of the geometry of the DWI data. Therefore, we present a tissue classifier based on a Riemannian deep learning framework for single-shell DWI data.ApproachThe framework consists of three layers: a lifting layer that locally represents and convolves data on tangent spaces to produce a family of functions defined on the rotation groups of the tangent spaces, i.e., a (not necessarily continuous) function on a bundle of rotational functions on the manifold; a group convolution layer that convolves this function with rotation kernels to produce a family of local functions over each of the rotation groups; a projection layer using maximization to collapse this local data to form manifold based functions.ResultsExperiments show that our method achieves the performance of the same level as state-of-the-art while using way fewer parameters in the model (<10 % ). Meanwhile, we conducted a model sensitivity analysis for our method. We ran experiments using a proportion (69.2%, 53.3%, and 29.4%) of the original training set and analyzed how much data the model needs for the task. Results show that this does reduce the overall classification accuracy mildly, but it also boosts the accuracy for minority classes.ConclusionsThis work extended convolutional neural networks to Riemannian manifolds, and it shows the potential in understanding structural patterns in the brain, as well as in aiding manual data annotation.
Rheumatoid arthritis (RA) is an inflammatory disease which afflicts the joints with arthritis and periarticular bone destruction as a result. One of its central features is bone erosion, a consequence of excessive bone resorption and insufficient bone formation. High-resolution peripheral quantitative computed tomography (HR-pQCT) is a promising tool for monitoring RA. Quantification of bone erosions and detection of possible progression is essential in the management of treatment. Detection is performed manually and is a very demanding task as rheumatologists must annotate hundreds of 2D images and inspect any region of the bone structure that is suspected to be a sign of RA. We propose a 2D based method which combines an accurate segmentation of bone surface boundary and classification of patches along the surface as healthy or eroded. We use a series of classical image processing methods to segment CT volumes semi-automatically. They are used as training data for a U-Net. We train a Siamese net to learn the difference between healthy and eroded patches. The Siamese net alleviates the problem of highly imbalanced class labels by providing a base for one-shot learning of differences between patches. We trained and tested the method using 3 full HR-pQCT scans with bone erosion of various size. The proposed pipeline succeeded in classifying healthy and eroded patches with high precision and recall. The proposed algorithm is a preliminary work to demonstrate the potential of our pipeline in automating the process of detecting and locating the eroded regions of bone surfaces affected by RA.
We present a simple approach to the voxelwise classification of brain tissue acquired with diffusion weighted MRI (DWI). The approach leverages the power of spherical harmonics to summarise the diffusion information, sampled at many points over a sphere, using only a handful of coefficients. We use simple features that are invariant to the rotation of the highly orientational diffusion data. This provides a way to directly classify voxels whose diffusion characteristics are similar yet whose primary diffusion orientations differ. Subsequent application of machine-learning to the spherical harmonic coefficients therefore may permit classification of DWI voxels according to their inferred underlying fibre properties, whilst ignoring the specifics of orientation. After smoothing apparent diffusion coefficients volumes, we apply a spherical harmonic transform, which models the multi-directional diffusion data as a collection of spherical basis functions. We use the derived coefficients as voxelwise feature vectors for classification. Using a simple Gaussian mixture model, we examined the classification performance for a range of sub-classes (3-20). The results were compared against existing alternatives for tissue classification e.g. fractional anisotropy (FA) or the standard model used by Camino.1 The approach was implemented on both two publicly-available datasets: an ex-vivo pig brain and in-vivo human brain from the Human Connectome Project (HCP). We have demonstrated how a robust classification of DWI data can be performed without the need for a model reconstruction step. This avoids the potential confounds and uncertainty that such models may impose, and has the benefit of being computable directly from the DWI volumes. As such, the method could prove useful in subsequent pre-processing stages, such as model fitting, where it could inform about individual voxel complexities and improve model parameter choice.
A challenge when using current magnetic resonance (MR)-based attenuation correction in positron emission tomography/MR imaging (PET/MRI) is that the MRIs can have a signal void around the dental fillings that is segmented as artificial air-regions in the attenuation map. For artifacts connected to the background, we propose an extension to an existing active contour algorithm to delineate the outer contour using the nonattenuation corrected PET image and the original attenuation map. We propose a combination of two different methods for differentiating the artifacts within the body from the anatomical air-regions by first using a template of artifact regions, and second, representing the artifact regions with a combination of active shape models and k-nearest-neighbors. The accuracy of the combined method has been evaluated using 25 F18-fluorodeoxyglucose PET/MR patients. Results showed that the approach was able to correct an average of 97±3% of the artifact areas.
KEYWORDS: Signal attenuation, Positron emission tomography, Magnetic resonance imaging, Medical imaging, Metals, Tissues, Nose, Control systems, Shape analysis, Esophagus
In combined PET/MR, attenuation correction (AC) is performed indirectly based on the available MR image information. Metal implant-induced susceptibility artifacts and subsequent signal voids challenge MR-based AC. Several papers acknowledge the problem in PET attenuation correction when dental artifacts are ignored, but none of them attempts to solve the problem. We propose a clinically feasible correction method which combines Active Shape Models (ASM) and k- Nearest-Neighbors (kNN) into a simple approach which finds and corrects the dental artifacts within the surface boundaries of the patient anatomy. ASM is used to locate a number of landmarks in the T1-weighted MR-image of a new patient. We calculate a vector of offsets from each voxel within a signal void to each of the landmarks. We then use kNN to classify each voxel as belonging to an artifact or an actual signal void using this offset vector, and fill the artifact voxels with a value representing soft tissue. We tested the method using fourteen patients without artifacts, and eighteen patients with dental artifacts of varying sizes within the anatomical surface of the head/neck region. Though the method wrongly filled a small volume in the bottom part of a maxillary sinus in two patients without any artifacts, due to their abnormal location, it succeeded in filling all dental artifact regions in all patients. In conclusion, we propose a method, which combines ASM and kNN into a simple approach which, as the results show, succeeds to find and correct the dental artifacts within the anatomical surface.
KEYWORDS: Image segmentation, Cartilage, Magnetic resonance imaging, Medical imaging, Bone, Machine learning, Algorithm development, Scanners, 3D image processing, 3D modeling
Many classification/segmentation tasks in medical imaging are particularly challenging for machine learning algorithms
because of the huge amount of training data required to cover biological variability. Learning methods scaling badly in
the number of training data points may not be applicable. This may exclude powerful classifiers with good generalization
performance such as standard non-linear support vector machines (SVMs). Further, many medical imaging problems
have highly imbalanced class populations, because the object to be segmented has only few pixels/voxels compared to
the background. This article presents a two-stage classifier for large-scale medical imaging problems. In the first stage,
a classifier that is easily trainable on large data sets is employed. The class imbalance is exploited and the classifier is
adjusted to correctly detect background with a very high accuracy. Only the comparatively few data points not identified as
background are passed to the second stage. Here a powerful classifier with high training time complexity can be employed
for making the final decision whether a data point belongs to the object or not. We applied our method to the problem of
automatically segmenting tibial articular cartilage from knee MRI scans. We show that by using nearest neighbor (kNN)
in the first stage we can reduce the amount of data for training a non-linear SVM in the second stage. The cascaded system
achieves better results than the state-of-the-art method relying on a single kNN classifier.
Classification is widely used in the context of medical image analysis and in order to illustrate the mechanism
of a classifier, we introduce the notion of an exaggerated image stereotype based on training data and trained
classifier. The stereotype of some image class of interest should emphasize/exaggerate the characteristic patterns
in an image class and visualize the information the employed classifier relies on. This is useful for gaining insight
into the classification and serves for comparison with the biological models of disease.
In this work, we build exaggerated image stereotypes by optimizing an objective function which consists of a
discriminative term based on the classification accuracy, and a generative term based on the class distributions.
A gradient descent method based on iterated conditional modes (ICM) is employed for optimization. We use
this idea with Fisher's linear discriminant rule and assume a multivariate normal distribution for samples within
a class. The proposed framework is applied to computed tomography (CT) images of lung tissue with emphysema.
The synthesized stereotypes illustrate the exaggerated patterns of lung tissue with emphysema, which is
underpinned by three different quantitative evaluation methods.
KEYWORDS: Chemical vapor deposition, Image classification, Medical imaging, Data modeling, Breast cancer, Error analysis, X-rays, Mammography, Principal component analysis, Machine learning
The performance of the k-nearest neighborhoods (k-NN) classifier is highly dependent on the distance metric
used to identify the k nearest neighbors of the query points. The standard Euclidean distance is commonly used
in practice. This paper investigates the performance of k-NN classifier with respect to different adaptive metrics
in the context of medical imaging. We propose using adaptive metrics such that the structure of the data is
better described, introducing some unsupervised learning knowledge in k-NN.
We investigated four different metrics are estimated: a theoretical metric based on the assumption that
images are drawn from Brownian Image Model (BIM), the normalized metric based on variance of the data, the
empirical metric is based on the empirical covariance matrix of the unlabeled data, and an optimized metric
obtained by minimizing the classification error. The spectral structure of the empirical covariance also leads to
Principal Component Analysis (PCA) performed on it which results the subspace metrics.
The metrics are evaluated on two data sets: lateral X-rays of the lumbar aortic/spine region, where we use
k-NN for performing abdominal aorta calcification detection; and mammograms, where we use k-NN for breast
cancer risk assessment. The results show that appropriate choice of metric can improve classification.
Osteoarthritis (OA) is a degenerative joint disease characterized by degradation of the articular cartilage, and is a
major cause of disability. At present, there is no cure for OA and currently available treatments are directed towards
relief of symptoms. Recently it was shown that cartilage homogeneity visualized by MRI and representing the
biochemical changes undergoing in the cartilage is a potential marker for early detection of knee OA. In this paper based
on homogeneity we present an automatic technique, embedded in a variational framework, for localization of a region of
interest in the knee cartilage that best indicates where the pathology of the disease is dominant. The technique is
evaluated on 283 knee MR scans. We show that OA affects certain areas of the cartilage more distinctly, and these are
more towards the peripheral region of the cartilage. We propose that this region in the cartilage corresponds anatomically
to the area covered by the meniscus in healthy subjects. This finding may provide valuable clues in the pathology and the
etiology of OA and thereby may improve treatment efficacy. Moreover our method is generic and may be applied to
other organs as well.
This paper aims at automatically measuring the extent of calcified plaques in the lumbar aorta from standard
radiographs. Calcifications in the abdominal aorta are an important predictor for future cardiovascular morbidity
and mortality. Accurate and reproducible measurement of the amount of calcified deposit in the aorta is therefore
of great value in disease diagnosis and prognosis, treatment planning, and the study of drug effects. We propose
a two-step approach in which first the calcifications are detected by an iterative statistical pixel classification
scheme combined with aorta shape model optimization. Subsequently, the detected calcified pixels are used as the
initialization for an inpainting based segmentation. We present results on synthetic images from the inpainting
based segmentation as well as results on several X-ray images based on the two-steps approach.
In this paper we seek to improve the standard method of assessing the degree of calcification in the lumbar aorta visualized on lateral 2-D X-rays. The semiquantitative method does not take density of calcification within the individual plaques into account and is unable to measure subtle changes in the severity of calcification over time. Both of these parameters would be desirable to assess, since they are the keys to assessing important information on the impact of risk factors and candidate drugs aiming at the prevention of atherosclerosis. As a further step for solving this task, we propose a pixelwise inpainting-based refinement scheme that seeks to optimize the individual plaque shape by maximizing the signal-to-noise ratio. Contrary to previous work the algorithm developped for this study uses a sorted candidate list, which omits possible bias introduced by the choice of starting pixel. The signal-to-noise optimization scheme will be discussed in different settings using TV as well as Harmonic inpainting and comparing these with a simple averaging process.
KEYWORDS: X-rays, Image processing, Image segmentation, Signal to noise ratio, X-ray imaging, Image restoration, Spine, Data modeling, Tissues, Data processing
In this paper we seek to improve upon the standard method of assessing the degree of calcification in the lumbar aorta, which is commonly used on lateral 2-D x-rays. The necessity for improvement arises from the fact that the existing method can not measure subtle progressions in the plaque development; neither is it possible to express the density of individual plaques. Both of these qualities would be desireable to assess, since they are the key for making progression studies as well as for testing the effect of drugs in longitudinal studies. Our approach is based on inpainting, a technique used in image restoration as well as postprocessing of film. In this study we discuss the potential implications of total variation inpainting for characterizing aortic calcification.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.