Paired split-plot designs of multireader multicase studies

Weijie Chen; Qi Gong; Brandon D. Gallas

doi:10.1117/1.JMI.5.3.031410

17 May 2018 Paired split-plot designs of multireader multicase studies

Weijie Chen, Qi Gong, Brandon D. Gallas

Author Affiliations +

Journal of Medical Imaging, Vol. 5, Issue 3, 031410 (May 2018). https://doi.org/10.1117/1.JMI.5.3.031410

Abstract

The widely used multireader multicase ROC study design for comparing imaging modalities is the fully crossed (FC) design: every reader reads every case of both modalities. We investigate paired split-plot (PSP) designs that may allow for reduced cost and increased flexibility compared with the FC design. In the PSP design, case images from two modalities are read by the same readers, thereby the readings are paired across modalities. However, within each modality, not every reader reads every case. Instead, both the readers and the cases are partitioned into a fixed number of groups and each group of readers reads its own group of cases—a split-plot design. Using a U-statistic based variance analysis for AUC (i.e., area under the ROC curve), we show analytically that precision can be gained by the PSP design as compared with the FC design with the same number of readers and readings. Equivalently, we show that the PSP design can achieve the same statistical power as the FC design with a reduced number of readings. The trade-off for the increased precision in the PSP design is the cost of collecting a larger number of truth-verified patient cases than the FC design. This means that one can trade-off between different sources of cost and choose a least burdensome design. We provide a validation study to show the iMRMC software can be reliably used for analyzing data from both FC and PSP designs. Finally, we demonstrate the advantages of the PSP design with a reader study comparing full-field digital mammography with screen-film mammography.

Citation Download Citation

Weijie Chen, Qi Gong, and Brandon D. Gallas "Paired split-plot designs of multireader multicase studies," Journal of Medical Imaging 5(3), 031410 (17 May 2018). https://doi.org/10.1117/1.JMI.5.3.031410

Received: 2 November 2017; Accepted: 30 April 2018; Published: 17 May 2018

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available