We present a risk stratification methodology for predictions made by computer-assisted detection (CAD) systems.
For each positive CAD prediction, the proposed technique assigns an individualized confidence measure
as a function of the actual CAD output, the case-specific uncertainty of the prediction estimated from the
system's performance for similar cases and the value of the operating decision threshold. The study was performed using a mammographic database containing 1,337 regions of interest (ROIs) with known ground truth
(681 with masses, 656 with normal parenchyma). Two types of decision models (1) a support vector machine
(SVM) with a radial basis function kernel and (2) a back-propagation neural network (BPNN) were developed
to detect masses based on 8 morphological features automatically extracted from each ROI. The study shows
that as requirements on the minimum confidence value are being restricted, the positive predictive value (PPV)
for qualifying cases steadily improves (from PPV = 0.73 to PPV = 0.97 for the SVM, from PPV = 0.67 to
PPV = 0.95 for the BPNN). The proposed confidence metric was successfully applied for stratification of CAD
recommendations into 3 categories of different expected reliability: HIGH (PPV = 0.90), LOW (PPV = 0.30)
and MEDIUM (all remaining cases). Since radiologists often disregard accurate CAD cues, an individualized
confidence measure should improve their ability to correctly process visual cues and thus reduce the interpretation
error associated with the detection task. While keeping the clinically determined operating point satisfied,
the proposed methodology draws the CAD users' attention to cases/regions of highest risk while helping them
confidently eliminate cases with low risk.