Probability densities and confidence intervals for target recognition performance metrics

David R. Parker; Steven C. Gustafson; Timothy D. Ross

doi:10.1117/12.602322

19 May 2005 Probability densities and confidence intervals for target recognition performance metrics

David R. Parker, Steven C. Gustafson, Timothy D. Ross

Proceedings Volume 5808, Algorithms for Synthetic Aperture Radar Imagery XII; (2005) https://doi.org/10.1117/12.602322
Event: Defense and Security, 2005, Orlando, Florida, United States

Abstract

Probability densities for target recognition performance metrics are developed. These densities assist in evaluation of systems under test (SUTs), which are systems that predict the presence of a target after examination of an input. After such examination, a SUT assigns a score that indicates the predicted likelihood that a target is present. From scores for a series of many inputs, the suitability of a SUT can be evaluated through performance metrics such as the receiver operating characteristic (ROC) and the confidence error (CE) generation curve. The ROC is a metric that describes how well the probability densities of target and clutter scores are separated, where clutter refers to the absence of target. The CE generation curve and the corresponding scalar CE is a metric that evaluates the accuracy of the score. Since only a limited number of test scores (scores for which the truth state is known by the evaluator) is typically available to evaluate a SUT, it is critical to quantify uncertainty in the performance metric results. A process for estimating such uncertainty through probability densities for the performance metrics is examined here. Once the probability densities are developed, confidence intervals are also obtained. The process that develops the densities and related confidence intervals is implemented in a fully Bayesian manner. Two approaches are examined, one which makes initial assumptions regarding the form of the underlying target and clutter densities and a second approach which avoids such assumptions. The target and clutter density approach is applicable to additional performance metrics.

Citation Download Citation

David R. Parker, Steven C. Gustafson, and Timothy D. Ross "Probability densities and confidence intervals for target recognition performance metrics", Proc. SPIE 5808, Algorithms for Synthetic Aperture Radar Imagery XII, (19 May 2005); https://doi.org/10.1117/12.602322

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available