Model observers have gained popularity as a surrogate approach for image quality assessment, they are often used for the optimization of the reconstruction algorithm. The most widespread model observer is the channelized Hotelling observer (CHO) that allows measuring the image quality by calculating the detectability index (or associated area under receiver operating characteristic curve). In this work we have chosen to explore different resampling methods used to estimate the CHO performance and uncertainty. In this paper, using data from the inter-laboratory comparison of the computation of CHO model observer study, we established a simulation framework to fully evaluate different resampling methods, namely, leave-one out and bootstrapping with replacement to estimate the CHO’s detectability index bias and uncertainty. For this particular study, we focus our experiments on datasets with a few data samples, 200 normal and 200 abnormal images.
Model observers (MO) are widely used in medical imaging to act as surrogates of human observers in task-based image quality evaluation, frequently towards optimization of reconstruction algorithms. In this paper, we explore the use of convolutional neural networks (CNN) to be used as MO. We will compare CNN MO to alternative MO currently being proposed and used such as the relevance vector machine based MO and channelized Hotelling observer (CHO). As the success of the CNN, and other deep learning approaches, is rooted in large data sets availability, which is rarely the case in medical imaging systems task-performance evaluation, we will evaluate CNN performance on both large and small training data sets.
Task-based medical image quality is typically measured by the degree to which a human observer can perform a diagnostic task in a psychophysical human observer study. During a typical study, an observer is asked to provide a numerical score quantifying his confidence as to whether an image contains a diagnostic marker or not. Such scores are then used to measure the observers’ diagnostic accuracy, summarized by the receiver operating characteristic (ROC) curve and the area under ROC curve. These types of human studies are difficult to arrange, costly, and time consuming. In addition, human observers involved in this type of study should be experts on the image genre to avoid inconsistent scoring through the lengthy study. In two-alternative forced choice (2AFC) studies, known to be faster, two images are compared simultaneously and a single indicator is given. Unfortunately, the 2AFC approach cannot lead to a full ROC curve or a set of image scores. The aim of this work is to propose a methodology in which multiple rounds of the 2AFC studies are used to re-estimate an image confidence score (a.k.a. rating, ranking) and generate the full ROC curve. In the proposed approach, we treat image confidence score as an unknown rating that needs to be estimated and 2AFC as a two-player match game. To achieve this, we use the ELO rating system, which is used for calculating the relative skill levels of players in competitor-versus-competitor games such as chess. The proposed methodology is not limited to ELO, and other rating methods such as TrueSkill™, Chessmetrics, or Glicko can be also used. The presented results, using simulated data, indicate that a full ROC curve can be recovered using several rounds of 2AFC studies and that the best pairing strategy starts with the first round of pairing abnormal versus normal images (as in the classical 2AFC approach) followed by a number of rounds using random pairing. In addition, the proposed method was tested in a pilot human observer study. These pilot results indicate that three to five rounds of 2AFC studies require less human observer time than a full scoring study and that the re-estimated ROC curves and associated area under ROC curve values have high statistical agreement with the full scoring study.
Proc. SPIE. 9037, Medical Imaging 2014: Image Perception, Observer Performance, and Technology Assessment
KEYWORDS: Detection and tracking algorithms, Sensors, Interference (communication), Medical imaging, Signal processing, Signal detection, Statistical modeling, Performance modeling, Process modeling, Tin
In this paper we propose the use of a machine-learning algorithm based in Gaussian Processes to estimate a human observer linear template for the detection of a signal in a noisy background. Estimating a human observer template is not novel, however the use of a multi-kernel Gaussian Processes approach is. This model provides spatial smoothing by using a sparse kernel representation. For validation purposes, we train this model observer with the ground truth and the estimated template is actually the same as the statistically optimal detector. Next, we present the human observer template estimated for the detection of a signal on a different power-low background.
In medical imaging, image quality is assessed by the degree to which a human observer can correctly perform a given diagnostic task. Therefore the image quality is typically quantified by using performance measurements from decision/detection theory like the receiver operation characteristic (ROC) curve and the area under ROC curve (AUC). In this paper we compare five different AUC estimation techniques, widely used in the literature, including parametric and non-parametric methods. We compared each method by equivalence hypothesis testing using a model observer as well as data sets from a previously published human observer study. The main conclusions of this work are 1) if a small number of images are scored, one cannot tell apart different AUC estimation methods due to large variability in AUC estimates, regardless whether image scores are reported on a continuous or quantized scale. 2) If the number of scored images is large and image scores are reported on a continuous scale, all tested AUC estimation methods are statistically equivalent.
In this paper we explore the capabilities of a graphical processing unit (GPU) for the fast calculation of a
tomographic projection operator in content-adaptive mesh models (CAMM). We explore the use of two distinct
methods, ray-tracing and mesh element projection, both implemented on classical computers (CPU) and GPU.
Both methods have already been proposed in the literature for 2D and 3D emission tomography (EM) image
reconstruction using a CAMM, however, there was no clear comparison between both methods in terms of
computational efficiency, which is an aim of this paper.
We describe and evaluate a fast implementation of a classical block-matching motion estimation algorithm for multiple graphical processing units (GPUs) using the compute unified device architecture computing engine. The implemented block-matching algorithm uses summed absolute difference error criterion and full grid search (FS) for finding optimal block displacement. In this evaluation, we compared the execution time of a GPU and CPU implementation for images of various sizes, using integer and noninteger search grids. The results show that use of a GPU card can shorten computation time by a factor of 200 times for integer and 1000 times for a noninteger search grid. The additional speedup for a noninteger search grid comes from the fact that GPU has built-in hardware for image interpolation. Further, when using multiple GPU cards, the presented evaluation shows the importance of the data splitting method across multiple cards, but an almost linear speedup with a number of cards is achievable. In addition, we compared the execution time of the proposed FS GPU implementation with two existing, highly optimized nonfull grid search CPU-based motion estimations methods, namely implementation of the Pyramidal Lucas Kanade Optical flow algorithm in OpenCV and simplified unsymmetrical multi-hexagon search in H.264/AVC standard. In these comparisons, FS GPU implementation still showed modest improvement even though the computational complexity of FS GPU implementation is substantially higher than non-FS CPU implementation. We also demonstrated that for an image sequence of 720 × 480 pixels in resolution commonly used in video surveillance, the proposed GPU implementation is sufficiently fast for real-time motion estimation at 30 frames-per-second using two NVIDIA C1060 Tesla GPU cards.
A potential drawback of image noise suppression in medical image sequence processing is a possible loss of the apparent
motion: making objects appears to move slower or less then they move in reality. For medical imaging application this
can be of critical importance, for example myocardium motion in cardiac gated single photon emission computed
tomography (SPECT) imaging can differentiate viable muscle from scar tissue. Therefore, in this work we design a set
of experiments to measure how human observers perceive apparent motion in the presence of image degradations like
noise and blur. In addition we will try to identify relevant image features, based on a visual attention model and a block
matching motion estimation method that would allow development of an accurate numerical observer capable of
predicting human observer motion perception.