Image generation method based on diversified feature learning attention mechanism

Wenhao Sun; Gengchen Liu; Weiqi Xue; Qingqi Zhou; Jiaju Wang; Jiaqing Yan

doi:10.1117/12.3035290

19 July 2024 Image generation method based on diversified feature learning attention mechanism

Wenhao Sun, Gengchen Liu, Weiqi Xue, Qingqi Zhou, Jiaju Wang, Jiaqing Yan

Proceedings Volume 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024); 1321306 (2024) https://doi.org/10.1117/12.3035290
Event: International Conference on Image Processing and Artificial Intelligence (ICIPAl2024), 2024, Suzhou, China

Abstract

Although attention mechanisms have been widely applied in natural language processing (NLP) tasks, there are still limitations in their utilization within the field of computer vision. To integrate the advantages of convolutional neural networks (CNNs) with attention mechanisms, this study proposes a multimodal image generation model based on the Stable Diffusion architecture. The model incorporates two types of convolutional modules, namely ICB and TCB. By replacing the linear networks with convolutional neural networks, the model enhances its capability to process complex features. Subsequently, the low-dimensional encoding reconstruction of images is achieved by maximizing the mutual information between the input of the new model and the output of the encoder. Finally, the proposed model is validated using a publicly available dataset.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Wenhao Sun, Gengchen Liu, Weiqi Xue, Qingqi Zhou, Jiaju Wang, and Jiaqing Yan "Image generation method based on diversified feature learning attention mechanism", Proc. SPIE 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024), 1321306 (19 July 2024); https://doi.org/10.1117/12.3035290

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available