25 September 2001 New scientific accuracy measure for performance evaluation of human-computer diagnostic systems
Author Affiliations +
Proceedings Volume 4553, Visualization and Optimization Techniques; (2001) https://doi.org/10.1117/12.441586
Event: Multispectral Image Processing and Pattern Recognition, 2001, Wuhan, China
This paper first presents a new scientific accuracy measure (denoted by G) for assessing/evaluating the performance of computer medical diagnostic (CMD) systems by incorporating the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) of human and computer's diagnoses with respect to each other. Based on G, a formula for computing a multi-parameter sensitivity vector S(G), with the assumption that the system parameter percentage variations are small, is then proposed. For a given set of parameter percentage errors, from the expression of S(G), we can compute the error bound of G and assess the reliability of the system with human and/or computer errors being taken into consideration. It has been demonstrated that the new measure G is capable of providing consistent performance evaluation of a CMD system in general. Based on the value of G, a CMD system can be classified as having 'good', 'fair', or 'poor' performance. Even though the proposed basic accuracy measure and its sensitivity study are derived based on the diagnosis using two diagnostic categories (positive and negative) compared by two observers (a human expert and a computer system), however, its methodology can be extended to CMD systems with multiple diagnostic categories and observers. The formulas for measuring the performance of such systems are discussed and present.
© (2001) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Samuel C. Lee, Samuel C. Lee, Elisa T. Lee, Elisa T. Lee, Yiming Wang, Yiming Wang, "New scientific accuracy measure for performance evaluation of human-computer diagnostic systems", Proc. SPIE 4553, Visualization and Optimization Techniques, (25 September 2001); doi: 10.1117/12.441586; https://doi.org/10.1117/12.441586

Back to Top