17 May 2018 Paired split-plot designs of multireader multicase studies
Author Affiliations +
Abstract
The widely used multireader multicase ROC study design for comparing imaging modalities is the fully crossed (FC) design: every reader reads every case of both modalities. We investigate paired split-plot (PSP) designs that may allow for reduced cost and increased flexibility compared with the FC design. In the PSP design, case images from two modalities are read by the same readers, thereby the readings are paired across modalities. However, within each modality, not every reader reads every case. Instead, both the readers and the cases are partitioned into a fixed number of groups and each group of readers reads its own group of cases—a split-plot design. Using a U-statistic based variance analysis for AUC (i.e., area under the ROC curve), we show analytically that precision can be gained by the PSP design as compared with the FC design with the same number of readers and readings. Equivalently, we show that the PSP design can achieve the same statistical power as the FC design with a reduced number of readings. The trade-off for the increased precision in the PSP design is the cost of collecting a larger number of truth-verified patient cases than the FC design. This means that one can trade-off between different sources of cost and choose a least burdensome design. We provide a validation study to show the iMRMC software can be reliably used for analyzing data from both FC and PSP designs. Finally, we demonstrate the advantages of the PSP design with a reader study comparing full-field digital mammography with screen-film mammography.
© 2018 Society of Photo-Optical Instrumentation Engineers (SPIE)
Weijie Chen, Weijie Chen, Qi Gong, Qi Gong, Brandon D. Gallas, Brandon D. Gallas, } "Paired split-plot designs of multireader multicase studies," Journal of Medical Imaging 5(3), 031410 (17 May 2018). https://doi.org/10.1117/1.JMI.5.3.031410 . Submission: Received: 2 November 2017; Accepted: 30 April 2018
Received: 2 November 2017; Accepted: 30 April 2018; Published: 17 May 2018
JOURNAL ARTICLE
12 PAGES


SHARE
RELATED CONTENT

Re use of pilot data and interim analysis of pivotal...
Proceedings of SPIE (March 10 2017)
How can the impact of PACS on inpatient length of...
Proceedings of SPIE (May 15 1994)
Jackknife free-response ROC methodology
Proceedings of SPIE (May 04 2004)
Statistical issues in ROC curve analysis
Proceedings of SPIE (August 01 1990)

Back to Top