Attention guided cross-modal multispectral object detection

Jialu Xing; Ying Tao; Bin Ma

doi:10.1117/12.3008208

14 November 2023 Attention guided cross-modal multispectral object detection

Jialu Xing, Ying Tao, Bin Ma

Proceedings Volume 12934, Third International Conference on Computer Graphics, Image, and Virtualization (ICCGIV 2023); 129340N (2023) https://doi.org/10.1117/12.3008208
Event: 2023 3rd International Conference on Computer Graphics, Image and Virtualization (ICCGIV 2023), 2023, Nanjing, China

Abstract

The spatial offset between the visible and thermal images can pose challenges for accurately matching objects in both modalities, thus affecting the performance of object detection algorithms. To address this, we propose a feature alignment-based algorithm for unaligned RGB-T image object detection. Our approach admits the coarse-to-fine feature alignment strategy, incorporating an attention-guided feature offset prediction module. Additionally, a multi-headed self-attention mechanism is introduced to predict and correct the feature map offsets for visible images during the feature extraction stage. To further correct the offset between RGB-T features, a region of interest alignment module performs quadratic regression for each candidate frame in the pooling stage. Furthermore, our algorithm introduces a light-aware weighting module to adaptively adjust the contributions of different modalities by reweighting the region of interest features. Experimental results on the FLIRADAS dataset demonstrate that the proposed method achieves high accuracy and stability.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jialu Xing, Ying Tao, and Bin Ma "Attention guided cross-modal multispectral object detection", Proc. SPIE 12934, Third International Conference on Computer Graphics, Image, and Virtualization (ICCGIV 2023), 129340N (14 November 2023); https://doi.org/10.1117/12.3008208

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available