Virtual multi-modal object detection and classification with deep convolutional neural networks

Nikolaos Mitsakos; Manos Papadakis

doi:10.1117/12.2529233

9 September 2019 Virtual multi-modal object detection and classification with deep convolutional neural networks

Nikolaos Mitsakos, Manos Papadakis

Proceedings Volume 11138, Wavelets and Sparsity XVIII; 1113805 (2019) https://doi.org/10.1117/12.2529233
Event: SPIE Optical Engineering + Applications, 2019, San Diego, California, United States

Abstract

In this paper we demonstrate how the post-processing of gray-scale images with algorithms which have a singularity enhancement effect can assume the role of auxiliary modalities, as in the case where an intelligent system fuses information from multiple physical modalities. We show that as in multimodal AI-fusion, “virtual” multimodal inputs can improve the performance of object detection. We design, implement and test a novel Convolutional Neural Network architecture, based on the Faster R-CNN network for multiclass object detection and classification. Our architecture combines deep feature representations of the input images, generated by networks trained independently on physical and virtual imaging modalities. Using an analog of the ROC curve, the Average Recall over Precision curve, we show that the fusion of certain virtual modality inputs, capable of enhancing singularities and neutralizing illumination, improve recognition accuracy.

Citation Download Citation

Nikolaos Mitsakos and Manos Papadakis "Virtual multi-modal object detection and classification with deep convolutional neural networks", Proc. SPIE 11138, Wavelets and Sparsity XVIII, 1113805 (9 September 2019); https://doi.org/10.1117/12.2529233

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available