Multireader multicase reader studies with binary agreement data: simulation, analysis, validation, and sizing

Weijie Chen; Adam Wunderlich; Nicholas A. Petrick; Brandon D. Gallas

doi:10.1117/1.JMI.1.3.031011

4 December 2014 Multireader multicase reader studies with binary agreement data: simulation, analysis, validation, and sizing

Weijie Chen, Adam Wunderlich, Nicholas A. Petrick, Brandon D. Gallas

Author Affiliations +

Journal of Medical Imaging, Vol. 1, Issue 3, 031011 (December 2014). https://doi.org/10.1117/1.JMI.1.3.031011

Abstract

We treat multireader multicase (MRMC) reader studies for which a reader’s diagnostic assessment is converted to binary agreement (1: agree with the truth state, 0: disagree with the truth state). We present a mathematical model for simulating binary MRMC data with a desired correlation structure across readers, cases, and two modalities, assuming the expected probability of agreement is equal for the two modalities (P₁=P₂). This model can be used to validate the coverage probabilities of 95% confidence intervals (of P₁, P₂, or P₁−P₂ when P₁−P₂=0), validate the type I error of a superiority hypothesis test, and size a noninferiority hypothesis test (which assumes P₁=P₂). To illustrate the utility of our simulation model, we adapt the Obuchowski–Rockette–Hillis (ORH) method for the analysis of MRMC binary agreement data. Moreover, we use our simulation model to validate the ORH method for binary data and to illustrate sizing in a noninferiority setting. Our software package is publicly available on the Google code project hosting site for use in simulation, analysis, validation, and sizing of MRMC reader studies with binary agreement data.

Citation Download Citation

Weijie Chen, Adam Wunderlich, Nicholas A. Petrick, and Brandon D. Gallas "Multireader multicase reader studies with binary agreement data: simulation, analysis, validation, and sizing," Journal of Medical Imaging 1(3), 031011 (4 December 2014). https://doi.org/10.1117/1.JMI.1.3.031011

Published: 4 December 2014

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available