24 June 1998 Components of variance in ROC analysis of CADx classifier performance
Author Affiliations +
We analyze the contributions to the population variance of the area under the ROC curve in assessment of CADx classifier performance and consider a number of models for this variance. The models all contain a pure term or terms in the number of training samples, a pure term in the number of test samples, plus a term or terms representing their interaction. The subset of terms containing the number of test samples also provide a model for what we call the mean Wilcoxon variance based on a single data set. By this variance we mean a nonparametric estimate of the uncertainty in the ROC area obtainable from a single experiment. The remaining terms--i.e., the pure terms in the number of training samples--are not directly estimable without drawing additional training samples. We investigate whether they may be inferred indirectly using a resampling strategy. The current study is presented within the context of our previous work on finite-sample effects on classifier performance, and is related to recent work of others on Analysis of Variance in ROC analysis.
© (1998) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Robert F. Wagner, Robert F. Wagner, Heang-Ping Chan, Heang-Ping Chan, Joseph T. Mossoba, Joseph T. Mossoba, Berkman Sahiner, Berkman Sahiner, Nicholas Petrick, Nicholas Petrick, } "Components of variance in ROC analysis of CADx classifier performance", Proc. SPIE 3338, Medical Imaging 1998: Image Processing, (24 June 1998); doi: 10.1117/12.310896; https://doi.org/10.1117/12.310896

Back to Top