A multimodal feature fusion model for depression diagnosis based on BiLSTM+ViT

Muyao Li; Yan Xing; Yitan He; Yuhe Wang; Zhouyang Yu; Xiaoyu Wang; Pingchuan Linghu

doi:10.1117/12.3035205

19 July 2024 A multimodal feature fusion model for depression diagnosis based on BiLSTM+ViT

Muyao Li, Yan Xing, Yitan He, Yuhe Wang, Zhouyang Yu, Xiaoyu Wang, Pingchuan Linghu

Proceedings Volume 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024); 132133O (2024) https://doi.org/10.1117/12.3035205
Event: International Conference on Image Processing and Artificial Intelligence (ICIPAl2024), 2024, Suzhou, China

Abstract

A thorough diagnosis of depression, a mental illness that affects many people worldwide, must be made in light of the patient's medical background, present symptoms, and pertinent examination findings. In recent years, scholars have increasingly favored machine learning algorithms for depression diagnosis or prediction models. However, achieving higher prediction accuracy in prediction models remains challenging when relying solely on one modality. This research suggests a multi-modal feature fusion model for predicting depression tendency that is based on bi-directional LSTM and vision transformer (ViT). The model demonstrates an accuracy of 70.00%, surpassing that of single-modal depression tendency prediction models and presenting a novel approach to depression tendency prediction.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Muyao Li, Yan Xing, Yitan He, Yuhe Wang, Zhouyang Yu, Xiaoyu Wang, and Pingchuan Linghu "A multimodal feature fusion model for depression diagnosis based on BiLSTM+ViT", Proc. SPIE 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024), 132133O (19 July 2024); https://doi.org/10.1117/12.3035205

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available