FDFST: facial structure constrained landmark detection using visual transformer

Qianyu Zhou; Yanxin Wang; Xuliang Li; Jiquan Ma

doi:10.1117/12.2640359

1 August 2022 FDFST: facial structure constrained landmark detection using visual transformer

Qianyu Zhou, Yanxin Wang, Xuliang Li, Jiquan Ma

Proceedings Volume 12257, 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022); 122571O (2022) https://doi.org/10.1117/12.2640359
Event: 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), 2022, Guangzhou, China

Abstract

Facial landmark detection is challenging with occlusion, pose or inadequate training samples. We proposed a two-branch facial landmark detection network (Facial Detection with Face Structure and Transformers: FDFST) considering face structure constraints. Existing regression-based facial landmark detection models have not fully considered the general facial structure for landmark detection, that usually lead a unstable prediction. In contrast to facial landmarks, facial structure is more likely to be accurately estimated in the real scenario. Therefore, we try to provide a facial structure guidance for the facial landmark detection by a facial structure estimation sub-network. In this way, two targets are predefined to supervise our model, one is the facial structure described by five landmarks, the other is facial landmark denoted by 96 points. To address the lack of occlusion samples, we proposed a novel data augmentation to boost the training process on the public data sets WFLW. Experiments have revealed that our FDFST network on the WFLW dataset achieved significant improvement.

Citation Download Citation

Qianyu Zhou, Yanxin Wang, Xuliang Li, and Jiquan Ma "FDFST: facial structure constrained landmark detection using visual transformer", Proc. SPIE 12257, 4th International Conference on Information Science, Electrical, and Automation Engineering (ISEAE 2022), 122571O (1 August 2022); https://doi.org/10.1117/12.2640359

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available