A risk-based comparison of classification systems

Steven Thorsen; Seth Wagenman; Mark E. Oxley; Kenneth W. Bauer

doi:10.1117/12.819280

11 May 2009 A risk-based comparison of classification systems

Steven Thorsen, Seth Wagenman, Mark E. Oxley, Kenneth W. Bauer

Proceedings Volume 7336, Signal Processing, Sensor Fusion, and Target Recognition XVIII; 73360T (2009) https://doi.org/10.1117/12.819280
Event: SPIE Defense, Security, and Sensing, 2009, Orlando, Florida, United States

Abstract

Performancemeasures for families of classification system families that rely upon the analysis of receiver operating characteristics (ROCs), such as area under the ROC curve (AUC), often fail to fully address the issue of risk, especially for classification systems involving more than two classes. For the general case, we denote matrices of class prevalence, costs, and class-conditional probabilities, and assume costs are subjectively fixed, acceptable estimates for expected values of class-conditional probabilities exist, and mutual independence between a variable in one such matrix and those of any other matrix. The ROC Risk Functional (RRF), valid for any finite number of classes, has an associated parameter argument, which specifies a member of a family of classification systems, and for which there is an associated classification system minimizing Bayes risk over the family. We typify joint distributions for class prevalences over standard simplices by means of uniform and beta distributions, and create a family of classification systems using actual data, testing independence assumptions under two such class prevalence distributions. Examples are given where the risk is minimized under two different sets of costs.

Citation Download Citation

Steven Thorsen, Seth Wagenman, Mark E. Oxley, and Kenneth W. Bauer "A risk-based comparison of classification systems", Proc. SPIE 7336, Signal Processing, Sensor Fusion, and Target Recognition XVIII, 73360T (11 May 2009); https://doi.org/10.1117/12.819280

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available