A training program to reduce reader search errors for liver metastasis detection in CT

Scott S. Hsieh; Akitoshi Inoue; Mariana Yalon; David A. Cook; Jeff L. Fidler; Hao Gong; Parvathy Sudhir Pillai; Andrew J. Vercnocke; Matthew P. Johnson; Shuai Leng; Lifeng Yu; David R. Holmes III; Rickey E. Carter; Cynthia H. McCollough; Joel G. Fletcher

doi:10.1117/12.2654007

3 April 2023 A training program to reduce reader search errors for liver metastasis detection in CT

Scott S. Hsieh, Akitoshi Inoue, Mariana Yalon, David A. Cook, Jeff L. Fidler, Hao Gong, Parvathy Sudhir Pillai, Andrew J. Vercnocke, Matthew P. Johnson, Shuai Leng, Lifeng Yu, David R. Holmes III, Rickey E. Carter, Cynthia H. McCollough, Joel G. Fletcher

Author Affiliations +

Proceedings Volume 12467, Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment; 1246704 (2023) https://doi.org/10.1117/12.2654007
Event: SPIE Medical Imaging, 2023, San Diego, California, United States

Abstract

Detection of low contrast liver metastases varies between radiologists. Training may improve performance for lower-performing readers and reduce inter-radiologist variability. We recruited 31 radiologists (15 trainees, eight non-abdominal staff, and eight abdominal staff) to participate in four separate reading sessions: pre-test, search training, classification training, and post-test. In the pre-test, each radiologist interpreted 40 liver CT exams containing 91 metastases, circumscribed suspected hepatic metastases while under eye tracker observation, and rated confidence. In search training, radiologists interpreted a separate set of 30 liver CT exams while receiving eye tracker feedback and after coaching to increase use of coronal reformations, interpretation time, and use of liver windows. In classification training, radiologists interpreted up to 100 liver CT image patches, most with benign or malignant lesions, and compared their annotations to ground truth. Post-test was identical to pre-test. Between pre- and post-test, sensitivity increased by 2.8% (p = 0.01) but AUC did not change significantly. Missed metastases were classified as search errors (<2 seconds gaze time) or classification errors (>2 seconds gaze time) using the eye tracker. Out of 2775 possible detections, search errors decreased (10.8% to 8.1%; p < 0.01) but classification errors were unchanged (5.7% vs 5.7%). When stratified by difficulty, easier metastases showed larger reductions in search errors: for metastases with average sensitivity of 0-50%, 50-90%, and 90-100%, reductions in search errors were 16%, 35%, and 58%, respectively. The training program studied here may be able to improve radiologist performance by reducing errors but not classification errors.

Conference Presentation

Citation Download Citation

Scott S. Hsieh, Akitoshi Inoue, Mariana Yalon, David A. Cook, Jeff L. Fidler, Hao Gong, Parvathy Sudhir Pillai, Andrew J. Vercnocke, Matthew P. Johnson, Shuai Leng, Lifeng Yu, David R. Holmes III, Rickey E. Carter, Cynthia H. McCollough, and Joel G. Fletcher "A training program to reduce reader search errors for liver metastasis detection in CT", Proc. SPIE 12467, Medical Imaging 2023: Image Perception, Observer Performance, and Technology Assessment, 1246704 (3 April 2023); https://doi.org/10.1117/12.2654007

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available