19 May 2011 Redefining automatic target recognition (ATR) performance standards
Author Affiliations +
Present descriptors for Automatic Target Recognition (ATR) performance are inadequate for use in comparing algorithms that are purported to be a solution to the problem. The use of receiver operator characteristic curves (ROCs) is a defacto standard, but they do not communicate several key performance measures, including (i) intrinsic separation between classes in the input space, (ii) the efficacy of the mapping induced by the algorithm, (iii) the complexity of the algorithmic mapping, and (iv) a measure of the generalization of the proposed solution. Previous work by Sims et. al.2,5 has addressed the distortion of the evaluation sets to indicate an algorithm's capability (or lack thereof) for generalization and handling of unspecified cases. This paper addresses the rethinking of the summary statistics used for understanding the performance of a solution. We propose new approaches for solution characterization, allowing algorithm performance comparison in an equitable and insightful manner. This paper proffers some examples and suggests directions for new work from the community in this field.
© (2011) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Donald Waagen, Donald Waagen, Charles Hester, Charles Hester, Ben Schmid, Ben Schmid, Margaret Phillips, Margaret Phillips, M. Shane Thompson, M. Shane Thompson, Steven Vanstone, Steven Vanstone, Kelly Risko, Kelly Risko, } "Redefining automatic target recognition (ATR) performance standards", Proc. SPIE 8049, Automatic Target Recognition XXI, 80490Z (19 May 2011); doi: 10.1117/12.886308; https://doi.org/10.1117/12.886308

Back to Top