Continuous sign language recognition based on 3DCNN and BLSTM

Hengbo Zhang; Daming Liu; Nana Fu

doi:10.1117/12.2667649

28 March 2023 Continuous sign language recognition based on 3DCNN and BLSTM

Hengbo Zhang, Daming Liu, Nana Fu

Proceedings Volume 12566, Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022); 125661Z (2023) https://doi.org/10.1117/12.2667649
Event: Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022), 2022, Chongqing, China

Abstract

Sign language recognition can make the communication between deaf mutes and healthy people more convenient and fast. In recent years, with the continuous development of deep learning, the research method of deep learning has also been introduced into the field of sign language recognition. Compared with the recognition of isolated words, the recognition of continuous sign language is more time-dependent. The current research still has shortcomings in recognition accuracy. Therefore, we proposed a continuous sign language recognition method based on 3DCNN and BLSTM. Based on the spatial feature information extracted by 3DCNN and the short-term temporal relationship established, the global temporal modeling of the video information of continuous sign language is carried out by using the bidirectional semantic mining ability of BLSTM. The CTC loss function is constructed to solve the problem of time series label misalignment. At the same time, we add the calculation of auxiliary loss function and auxiliary classifier. Experiments show that the auxiliary loss function and classifier can effectively reduce the error rate of the network. The word error rate of the continuous sign language recognition algorithm proposed in this paper on the large continuous sign language dataset RWTH-PHONEIX-Weather 2014 is as low as 23.5%, which is lower than the previous algorithm.

Citation Download Citation

Hengbo Zhang, Daming Liu, and Nana Fu "Continuous sign language recognition based on 3DCNN and BLSTM", Proc. SPIE 12566, Fifth International Conference on Computer Information Science and Artificial Intelligence (CISAI 2022), 125661Z (28 March 2023); https://doi.org/10.1117/12.2667649

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Feature extraction

Video

Education and training

Convolution

3D modeling

Detection and tracking algorithms

Algorithm development

Show All Keywords

Keywords/Phrases

Search In:

Publication Years