Research on fusion algorithm of infrared and visible image based on generative adversarial network and attention mechanism

Xiang Chen; Jun Dai; DaiZhong Jin; Tao Chen

doi:10.1117/12.2660014

3 February 2023 Research on fusion algorithm of infrared and visible image based on generative adversarial network and attention mechanism

Xiang Chen, Jun Dai, DaiZhong Jin, Tao Chen

Author Affiliations +

Proceedings Volume 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022); 125112M (2023) https://doi.org/10.1117/12.2660014
Event: Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 2022, Hulun Buir, China

Abstract

Generative adversarial network was proposed in 2014, and its core is a two-person zero-sum game, which improves the quality of generated images through the mutual game between the generator and the discriminator. In 2018, Ma et al. applied Generative Adversarial Networks to the field of infrared and visible image fusion. The generator generates an image that retains both the infrared intensity and visible light detail texture information. By inputting the visible image as "true data" into the discriminator, it retains more detailed texture parts in the visible image in the fusion result, and finally obtains a nice effect. However, since the model only uses one discriminator, it loses some infrared intensity information and detail information in the infrared image. The gradient of the loss function is not enough to describe the detailed texture of visible light, and information such as brightness and contrast is not considered, so the final fusion result has poor visual effect. This paper builds an infrared and visible image fusion network based on the LSGAN framework and uses dual discriminators, and introduces a convolutional attention module in the generator to make the fusion image pay more attention to infrared intensity information. It is proposed to use MS-SSIM loss function to constrain the generated image, so that the fused image has a higher structural similarity with the source image. Through qualitative and quantitative analysis, it is proved that this method retains the main infrared intensity information while retaining the detailed texture information of the visible image.

Citation Download Citation

Xiang Chen, Jun Dai, DaiZhong Jin, and Tao Chen "Research on fusion algorithm of infrared and visible image based on generative adversarial network and attention mechanism", Proc. SPIE 12511, Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 125112M (3 February 2023); https://doi.org/10.1117/12.2660014

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Infrared imaging

Convolution

Image processing

Data fusion

Image quality

Wavelets

Keywords/Phrases

Search In:

Publication Years