Performance metrics with operational context for video object detection algorithms

Yoldaş Ataseven

doi:10.1117/12.2636106

28 October 2022 Performance metrics with operational context for video object detection algorithms

Yoldaş Ataseven

Proceedings Volume 12276, Artificial Intelligence and Machine Learning in Defense Applications IV; 122760A (2022) https://doi.org/10.1117/12.2636106
Event: SPIE Security + Defence, 2022, Berlin, Germany

Abstract

Performance metrics that are used by the academia to evaluate the performance of video object detection algorithms are usually not informative for assessing whether or not the algorithms of interest are suitable (mature enough) for real-world deployment. We propose an approach to define the performance metrics that are suitable for various operational scenarios. In particular, we define four operational modes: Surveillance (alarm), situational awareness, detection and tracking. We then describe the performance metrics for the needs of each operational scenario, and explain the underlying reasoning. The metrics are compatible with the common practices for constructing in-house video datasets. We believe that these metrics provide useful insight for the usability of the algorithms. We also demonstrate that an algorithm which (at first glance) seem to have insufficient performance for deployment can be used in a real-world system (with simple post-processing) if its parameters are configured to provide high performance scores for scenario-specific metrics. We also show that the same underlying algorithm can be used for different operational scenarios if its parameters (and/or post-processing steps) are adjusted to meet the criteria based on relevant scenario-based performance metrics.

Conference Presentation

Citation Download Citation

Yoldaş Ataseven "Performance metrics with operational context for video object detection algorithms", Proc. SPIE 12276, Artificial Intelligence and Machine Learning in Defense Applications IV, 122760A (28 October 2022); https://doi.org/10.1117/12.2636106

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available