Uncertainties of measures in speaker recognition evaluation

Jin Chu Wu; Alvin F. Martin; Craig S. Greenberg; Raghu N. Kacker

doi:10.1117/12.883252

10 May 2011 Uncertainties of measures in speaker recognition evaluation

Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker

Proceedings Volume 8040, Active and Passive Signatures II; 804008 (2011) https://doi.org/10.1117/12.883252
Event: SPIE Defense, Security, and Sensing, 2011, Orlando, Florida, United States

Abstract

The National Institute of Standards and Technology (NIST) Speaker Recognition Evaluations (SRE) are an ongoing series of projects conducted by NIST. In the NIST SRE, speaker detection performance is measured using a detection cost function, which is defined as a weighted sum of probabilities of type I error and type II error. The sampling variability can result in measurement uncertainties of the detection cost function. Hence, while evaluating and comparing the performances of speaker recognition systems, the uncertainties of measures must be taken into account. In this article, the uncertainties of detection cost functions in terms of standard errors (SE) and confidence intervals are computed using the nonparametric two-sample bootstrap methods based on our extensive bootstrap variability studies on large datasets conducted before. The data independence is assumed because the bootstrap results of SEs matched very well with the analytical results of SEs using the Mann-Whitney statistic for independent and identically distributed samples if the metric of area under a receiver operating characteristic curve is employed. Examples are provided.

Citation Download Citation

Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, and Raghu N. Kacker "Uncertainties of measures in speaker recognition evaluation", Proc. SPIE 8040, Active and Passive Signatures II, 804008 (10 May 2011); https://doi.org/10.1117/12.883252

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available