Simulation of multi-reader multi-case study data with realistic ROC performance characteristics

Xiaochen Zhu; Weijie Chen

doi:10.1117/12.2550545

16 March 2020 Simulation of multi-reader multi-case study data with realistic ROC performance characteristics

Xiaochen Zhu, Weijie Chen

Proceedings Volume 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment; 113160M (2020) https://doi.org/10.1117/12.2550545
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

Multi-reader multi-case (MRMC) studies are widely used in assessing medical imaging and computer-aided diagnosis devices to demonstrate the generalizability of diagnostic performance to both the population of patient cases and the population of physician readers. Simulation of MRMC study data plays an important role in validating MRMC data analysis methods or sizing a pivotal study based on pilot data. The popular Roe and Metz simulation model is a linear mixed-effect model that models a human reader's latent decision variable in assessing patient's likelihood of disease as a sum of fixed modality effect, random reader effect, random case effect, random interaction effects, and a random error. The fixed effect is represented by a mean parameter and each random effect is represented by the variance parameter of a zero-mean Gaussian distribution. The purpose of this paper is to develop a method to set these parameters such that the simulated data have realistic ROC performance characteristics (mean AUC, variance components of AUC, and inter-reader/inter-modality correlations). To this end, we derived quasi-closed-form expressions to express the mean AUC and its U-statistic variance components as functions of the simulation model parameters. We then developed a numerical algorithm to solve the simulation model parameters from the mean AUC and its U-statistic variance components. Since the mean AUC and its U-statistic variance components can be estimated from real-world reader study data, simulated data have similar performance characteristics with the real-world data. Simulation studies were conducted to verify our parameter transformation algorithm.

Conference Presentation

Citation Download Citation

Xiaochen Zhu and Weijie Chen "Simulation of multi-reader multi-case study data with realistic ROC performance characteristics", Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160M (16 March 2020); https://doi.org/10.1117/12.2550545

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available