Use of equivalent relative utility to evaluate artificial intelligence-based rule-out devices

Kwok Lung Fan; Yee Lam Elim Thompson; Weijie Chen; Craig K. Abbey; Frank W. Samuelson

doi:10.1117/12.3008632

29 March 2024 Use of equivalent relative utility to evaluate artificial intelligence-based rule-out devices

Kwok Lung Fan, Yee Lam Elim Thompson, Weijie Chen, Craig K. Abbey, Frank W. Samuelson

Proceedings Volume 12929, Medical Imaging 2024: Image Perception, Observer Performance, and Technology Assessment; 129290P (2024) https://doi.org/10.1117/12.3008632
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

We investigated the use of equivalent relative utility (ERU) to evaluate the effectiveness of artificial intelligence (AI)-based rule-out algorithms designed to autonomously remove non-cancer patient images from radiologist review. Two evaluation metrics are explored: positive/negative predictive values and ERU. We applied both methods to a recent US study that concluded an improved specificity by retrospectively applying their AI algorithm to analyze a large mammography dataset. The ERU values are also calculated given the recall and cancer detection rates from a European mammography screening study. Without large prospective studies, ERU may provide insights in the effectiveness of a rule-out algorithm.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Kwok Lung Fan, Yee Lam Elim Thompson, Weijie Chen, Craig K. Abbey, and Frank W. Samuelson "Use of equivalent relative utility to evaluate artificial intelligence-based rule-out devices", Proc. SPIE 12929, Medical Imaging 2024: Image Perception, Observer Performance, and Technology Assessment, 129290P (29 March 2024); https://doi.org/10.1117/12.3008632

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available