A bound on performance for object detection and classification for machine learning

John M. Irvine; James H. Tanis; Nazario Irizarry; Franck Olivier Ndjakou Njeunje

doi:10.1117/12.3013928

10 June 2024 A bound on performance for object detection and classification for machine learning

John M. Irvine, James H. Tanis, Nazario Irizarry, Franck Olivier Ndjakou Njeunje

Proceedings Volume 13037, Geospatial Informatics XIV; 1303702 (2024) https://doi.org/10.1117/12.3013928
Event: SPIE Defense + Commercial Sensing, 2024, National Harbor, Maryland, United States

Abstract

An objective, quantitative method for assessing image interpretability for machine learning (ML) would be a valuable tool to support sensor design, collection management, and algorithm selection. The National Imagery Interpretability Rating Scale (NIIRS) has served as a useful standard for image analysis in support of intelligence, surveillance, and reconnaissance (ISR) missions. However, NIIRS focuses on human perception and empirical studies have demonstrated a tenuous relationship, at best, between NIIRS and observed performance for ML algorithms. We propose a new approach that approximates the Bayes error for object classification to establish an upper bound on ML performance for a given set of imagery. The process starts with high fidelity signatures from the object classes of interest. Degrading these signatures through an emulation of the sensor’s image chain produces signatures consistent with observed imagery from that sensor. Various distance metrics quantify the separability between specific object classes. We demonstrate a resampling technique to approximate the Bayes error, which is the theoretical limit for performance. This approach provides a quantitative measure that is independent of any specific machine learning model or methodology.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

John M. Irvine, James H. Tanis, Nazario Irizarry, and Franck Olivier Ndjakou Njeunje "A bound on performance for object detection and classification for machine learning", Proc. SPIE 13037, Geospatial Informatics XIV, 1303702 (10 June 2024); https://doi.org/10.1117/12.3013928

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available