Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography

Abhinav K. Jha; Esther Mena M.D.; Brian S. Caffo; Saeed Ashrafinia; Arman Rahmim; Eric C. Frey; Rathan M. Subramaniam M.D.

doi:10.1117/1.JMI.4.1.011011

3 March 2017 Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography

Abhinav K. Jha, Esther Mena M.D., Brian S. Caffo, Saeed Ashrafinia, Arman Rahmim, Eric C. Frey, Rathan M. Subramaniam M.D.

Author Affiliations +

Funded by: National Institute of Health, US National Institutes of Health (NIH)

Journal of Medical Imaging, Vol. 4, Issue 1, 011011 (March 2017). https://doi.org/10.1117/1.JMI.4.1.011011

Abstract

Recently, a class of no-gold-standard (NGS) techniques have been proposed to evaluate quantitative imaging methods using patient data. These techniques provide figures of merit (FoMs) quantifying the precision of the estimated quantitative value without requiring repeated measurements and without requiring a gold standard. However, applying these techniques to patient data presents several practical difficulties including assessing the underlying assumptions, accounting for patient-sampling-related uncertainty, and assessing the reliability of the estimated FoMs. To address these issues, we propose statistical tests that provide confidence in the underlying assumptions and in the reliability of the estimated FoMs. Furthermore, the NGS technique is integrated within a bootstrap-based methodology to account for patient-sampling-related uncertainty. The developed NGS framework was applied to evaluate four methods for segmenting lesions from F-Fluoro-2-deoxyglucose positron emission tomography images of patients with head-and-neck cancer on the task of precisely measuring the metabolic tumor volume. The NGS technique consistently predicted the same segmentation method as the most precise method. The proposed framework provided confidence in these results, even when gold-standard data were not available. The bootstrap-based methodology indicated improved performance of the NGS technique with larger numbers of patient studies, as was expected, and yielded consistent results as long as data from more than 80 lesions were available for the analysis.

Citation Download Citation

Abhinav K. Jha, Esther Mena M.D., Brian S. Caffo, Saeed Ashrafinia, Arman Rahmim, Eric C. Frey, and Rathan M. Subramaniam M.D. "Practical no-gold-standard evaluation framework for quantitative imaging methods: application to lesion segmentation in positron emission tomography," Journal of Medical Imaging 4(1), 011011 (3 March 2017). https://doi.org/10.1117/1.JMI.4.1.011011

Received: 1 June 2016; Accepted: 9 February 2017; Published: 3 March 2017

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE