Research on image captioning based on LSTM and YOLOv5 fusion attention mechanism

Xiaolliang Zhang; Qingtao Zeng; Yeli Li; Likun Lu; Weichun Yang

doi:10.1117/12.2667667

22 February 2023 Research on image captioning based on LSTM and YOLOv5 fusion attention mechanism

Xiaolliang Zhang, Qingtao Zeng, Yeli Li, Likun Lu, Weichun Yang

Proceedings Volume 12587, Third International Seminar on Artificial Intelligence, Networking, and Information Technology (AINIT 2022); 125870Y (2023) https://doi.org/10.1117/12.2667667
Event: Third International Seminar on Artificial Intelligence, Networking, and Information Technology (AINIT 2022), 2022, Shanghai, China

Abstract

Humans can easily learn to recognize every object in life, every landscape, and describe the things around them in detail from the process of growing up, but computers cannot. How to make computers learn to describe things in pictures has become the research direction of many scholars. If this technology is mature, it will bring great boon to people with visual impairments. They can understand the things around them and the beautiful earth through hearing. Robots recognize objects and understand their surroundings. With the development of artificial intelligence, the power of convolutional neural networks is more and more comparable to that of the human brain. In recent years, many scholars have proposed different methods to seek better solutions to this problem, including generative adversarial networks. Based on the classic structure of Encoder-Decoder, this paper first compares the code implementation and results of ResNet101 as an Encoder on the COCO dataset, and then proposes a new solution that integrates YOLOv5 and LSTM, aiming to improve the model inference speed and inference accuracy.

Citation Download Citation

Xiaolliang Zhang, Qingtao Zeng, Yeli Li, Likun Lu, and Weichun Yang "Research on image captioning based on LSTM and YOLOv5 fusion attention mechanism", Proc. SPIE 12587, Third International Seminar on Artificial Intelligence, Networking, and Information Technology (AINIT 2022), 125870Y (22 February 2023); https://doi.org/10.1117/12.2667667

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Networks

Education and training

Image processing

Evolutionary algorithms

Detection and tracking algorithms

Computer programming

Data modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years