Obstacle detect based on improved neural network for fusion of radar and visual information

Chenhao Luo; Haoyu Zhang

doi:10.1117/12.2673495

14 April 2023 Obstacle detect based on improved neural network for fusion of radar and visual information

Chenhao Luo, Haoyu Zhang

Proceedings Volume 12613, International Conference on Computer Vision, Application, and Algorithm (CVAA 2022); 1261305 (2023) https://doi.org/10.1117/12.2673495
Event: International Conference on Computer Vision, Application, and Algorithm (CVAA 2022), 2022, Chongqing, China

Abstract

Image target detection technology using deep learning has been fully developed in recent years, but in harsh and complex environments such as rain, snow, and darkness, the information collected by the image sensor will have a lot of noise, and it is easy to lose information about small distant targets and fuzzy targets, resulting in false detection and missed detection. Millimeter-wave(mmWave) radar uses electromagnetic waves for obstacle detection, and the detection performance in harsh environments basically does not degrade, in order to solve the situation that the detection accuracy is significantly reduced when single sensor environmental sensing technology is applied to complex scenes. In this paper, an enhanced neural network model for feature fusion is proposed by fusing radar and camera sensor features and combining them with an improved YOLO-S neural network. Firstly, the model is designed to convert the radar point cloud into image form while applying the BiFPN structure with integrated CA attention mechanism to the YOLOX-S neural network. By connecting radar features with image features and combining radar features with spatial attention weights, the useless information in the extracted features is suppressed while the key information is enhanced, and the sensor features are complementary. The experimental results show that the fusion-enhanced network designed in this paper fully integrates the features of both sensors and provides better detection results and stronger robustness in detecting small distant objects and objects in dark and low-light environments.

Citation Download Citation

Chenhao Luo and Haoyu Zhang "Obstacle detect based on improved neural network for fusion of radar and visual information", Proc. SPIE 12613, International Conference on Computer Vision, Application, and Algorithm (CVAA 2022), 1261305 (14 April 2023); https://doi.org/10.1117/12.2673495

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Radar sensor technology

Radar

Target detection

Feature fusion

Image enhancement

Image sensors

Image fusion

Show All Keywords

Keywords/Phrases

Search In:

Publication Years