A robust spatio-temporal face modelling approach using 3D multimodal fusion for biometric security applications

Girija Chetty; Michael Wagner

doi:10.1117/12.778631

17 March 2008 A robust spatio-temporal face modelling approach using 3D multimodal fusion for biometric security applications

Girija Chetty, Michael Wagner

Author Affiliations +

Proceedings Volume 6944, Biometric Technology for Human Identification V; 694409 (2008) https://doi.org/10.1117/12.778631
Event: SPIE Defense and Security Symposium, 2008, Orlando, Florida, United States

Abstract

In this paper, we propose a robust spatio-temporal face modelling approach based on multilevel fusion strategy involving cascaded fusion of hybrid multimodal fusion of audio-lip-face motion, correlation and depth features for biometric security application. The proposed approach combines the information from different audio-video based modules, namely: audio-lip motion module, audio-lip correlation module, 2D+3D motion-depth fusion module, and performs a hybrid cascaded fusion in an automatic, unsupervised and adaptive manner, by adapting to the local performance of each module. This is done by taking the output-score based reliability estimates (confidence measures) of each of the module into account. The module weightings are determined automatically such that the reliability measure of the combined scores is maximised. To test the robustness of the proposed approach, the audio and visual speech (mouth) modalities are degraded to emulate various levels of train/test mismatch; employing additive white Gaussian noise for the audio and JPEG compression for the video signals. The results show improved fusion performance for a range of tested levels of audio and video degradation, compared to the individual module performances. Experiments on a 3D stereovision database AVOZES show that, at severe levels of audio and video mismatch, the audio, mouth, 3D face, and tri-module (audio-lip motion, correlation and depth) fusion EERs were 42.9%, 32%, 15%, and 7.3% respectively for biometric identity verification scenario.

Citation Download Citation

Girija Chetty and Michael Wagner "A robust spatio-temporal face modelling approach using 3D multimodal fusion for biometric security applications", Proc. SPIE 6944, Biometric Technology for Human Identification V, 694409 (17 March 2008); https://doi.org/10.1117/12.778631

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
14 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Laser induced plasma spectroscopy

Visualization

Mouth

Video

Reliability

3D modeling

Biometrics

Show All Keywords

Keywords/Phrases

Search In:

Publication Years