Due to the short duration and low intensity, the efficient feature learning is a big challenge for robust facial microexpression (ME) recognition. To achieve the diverse and spatial relation representation, this paper proposes a simple yet effective micro-expression recognition method based on multiscale convolutional fusion and capsule network (MCFCN). Firstly, the apex frame in a ME clip is located by computing the pixel difference of frames, and then the apex frame is processed by an optical flow operator. Secondly, a multi-scale fusion module is introduced to capture diverse ME related details. Then, the micro-expression features are fed into the capsule network for a good description about spatial relation. Finally, the entire ME recognition model is trained and verified on three popular benchmarks (SAMM, SMIC and CASMEII) using the associated standard evaluation protocols. Experimental results show that our method based on MCFCN is superior to the works based on pervious capsule network or other state-of-the-art CNN models.
Micro-expression (ME), which reveals the genuine feelings and motives within human beings, attracts considerable attention in the field of automatic affective recognition. The main challenges for robust micro-expression recognition (MER) are from the short ME duration, low intensity of facial muscle movements, and insufficient samples. To meet these challenges, we propose an optical flow-based deep capsule adversarial domain adaptation network (DCADAN) for MER, which leverages a deep neural network stemming from these speculations. To alleviate the negative impact of the identity related features, optical flow preprocessing is applied to encode the subtle face motion information that is highly related to facial MEs. Then, a deep capsule network is developed to determine the part–whole relationships on optical flow features. To cope with the data deficiency and enhance the generalization capability via domain adaptation, an adversarial discriminator module that enriches the available samples from macro-expression data is integrated into the capsule network to train an expeditious end-to-end deep network. Finally, a simple and yet efficient attention module is embedded to the DCADAN to adaptively aggregate optical flow convolution maps into the primary capsule layers. We evaluate the performance of the entire network on the cross-database ME benchmark (3DB) using the leave-one-subject-out cross-validation. Unweighted F1-score (UF1) and unweighted average recall (UAR) are exploited as the evaluation metrics. The MER based on DCADAN achieves a UF1 score of 0.801 and a UAR score of 0.829 in comparison with a UF1 of 0.788 and a UAR of 0.782 for the updated approach. The comprehensive experimental results show that the incorporation of adversarial domain adaption into the capsule network is feasible and effective for representing discriminative features in ME and the proposed model outperforms state-of-the-art deep learning networks for MER.
Micro-expression, revealing the true emotions and motives, attracts extraordinary attention on automatic facial microexpression recognition (MER). The main challenge of MER is large-scale datasets unavailable to support deep learning training. To this end, this paper proposes an end-to-end transfer model for facial MER based on the difference images. Compared with micro-expression dataset, macro-expression dataset has more samples and is easy to train for deep neural network. Thus, we pre-train the resnet-18 network on relatively large expression datasets to get the good initial backbone module. Then, the difference images based on adaptive key frame is applied to get MER related feature representation for the module input. Finally, the preprocessing difference images are feed into the pre-trained resent-18 network for fine-tuning. Consequently, the proposed method achieves the recognition rates of 74.39% and 76.22% on the CASME2 and SMIC databases, respectively. The experimental results show that the difference image between the onset and key frame can improve the transfer training performance on resnet-18, the proposed MER method outperforms the methods based on traditional hand-crafted features and deep neural networks.
To tackle with false recognition of the forged vasculars in vein recognition, a new vascular recognition method is proposed by photoacoustic anti-counterfeiting. An optical-resolution photoacoustic microscopy imaging system is built by use of the laser diode. Then, photoacoustic experiments were performed on the forged vasculars and isolated subcutaneous vasculars. To realize anti-counterfeiting of vasculars, the photoacoustic signal intensity is used to distinguish between forged and real vasculars. Meanwhile, the vascular recognition performance can be lifted on our established photoacoustic vascular library. The experiment results show that the proposed technique has the advantages of high anti-counterfeiting and high recognition rate, and can be applied in biometrics.
Traditional studies on micro-expression feature extraction primarily focused on global face from all frames. To improve the efficiency of feature extraction, this paper proposes a new framework based on the local region and the key frame to represent facial micro-expressions. Firstly, the face feature point detection technique is used to acquire the coordinates of the 68 key points, and the region of interest is divided by those key point coordinates and the action unit. Secondly, in order to remove redundant information in the micro-expression video sequence, structural similarity index (SSIM) is used to select key frames for each local region of interest. Finally, the dual-cross patterns (DCP) are extracted for the local regions of interest and are concatenated into a feature vector for the final classification. The experimental results show that compared with the traditional micro-expression method, the proposed method has higher recognition rate and achieves better time computation performance.
KEYWORDS: Facial recognition systems, Hyperspectral imaging, Associative arrays, Feature extraction, Prototyping, Databases, Signal to noise ratio, Detection and tracking algorithms, Principal component analysis, Information fusion
The hyperspectral imaging, adding many dimensions, has practical significance for robust face recognition. However, for hyperspectral face recognition, the main problems are small sample collection, low signal-to-noise ratio and inter band misalignment. In view of these problems, we propose a hyperspectral face recognition method based on SLRC (superposed linear representation classifier) for single sample problem. In the proposed method, one sample from each class is selected as training data, then the rest samples as test data. Since hyperspectral images have multiple bands, we average all bands as prototype dictionaries, and the difference between each band and the corresponding prototype dictionary as variation dictionary. Compared with other sparse representation classification methods, the proposed method can directly use a single sample to train hyperspectral face recognition and has no handcraft feature extraction. Experiments on the hyperspectral face database (PloyU-HSFD) validate that the proposed method can not only greatly increase the accuracy in single sample hyperspectral face recognition, but also improve the computation speed.
Recognizing microexpression serves as a vital clue for affective estimation. Fast and discriminative feature extraction has always been a critical issue for spontaneous microexpression recognition applications. A microexpression analysis framework is proposed by adaptively key frame extraction and representation. First, to remove redundant information in the microexpression video sequences, the key frame is adaptively selected on the criteria of structural similarity index between different face images, Second, robust principal component analysis is applied to obtain the sparse information in the key frame, which not only retains the expression attributes of the microexpression sequence, but also eliminates useless interference. Furthermore, we construct dual-cross patterns to get the final microexpressions representation for classification. Repeated comparison experiments were performed on the SMIC and CASME2 databases to evaluate the performance of the proposed method. Experimental results demonstrate that our proposed method gets higher recognition rates and achieves promising performance, compared with the traditional microexpression recognition.
Near infrared and visible fusion recognition is an active topic for robust face recognition. Local binary patterns (LBP) based descriptors and sparse representation based classification (SRC) become two significant techniques in face recognition. In this paper, near infrared and visible face fusion recognition based on LBP and extended SRC is proposed for single sample problem. Firstly, the local features are extracted by LBP descriptor for infrared and visible face representation. Secondly, the extend SRC (ESRC) is applied for single sample problem. Finally, to get a robust and time-efficient fusion model for unconstrained face recognition with single sample situation, the infrared and visible features fusion problem is resolved by error-level fusion based on ESRC. Experiments are performed on HITSZ LAB2 database and the experiments results show that the proposed method extracts the complementary features of near-infrared and visible-light images and improves the robustness of unconstrained face recognition with single sample situation.
Visible face recognition systems, being vulnerable to illumination, expression, and pose, can not achieve robust performance in unconstrained situations. Meanwhile, near infrared face images, being light- independent, can avoid or limit the drawbacks of face recognition in visible light, but its main challenges are low resolution and signal noise ratio (SNR). Therefore, near infrared and visible fusion face recognition has become an important direction in the field of unconstrained face recognition research. In order to extract the discriminative complementary features between near infrared and visible images, in this paper, we proposed a novel near infrared and visible face fusion recognition algorithm based on DCT and LBP features. Firstly, the effective features in near-infrared face image are extracted by the low frequency part of DCT coefficients and the partition histograms of LBP operator. Secondly, the LBP features of visible-light face image are extracted to compensate for the lacking detail features of the near-infrared face image. Then, the LBP features of visible-light face image, the DCT and LBP features of near-infrared face image are sent to each classifier for labeling. Finally, decision level fusion strategy is used to obtain the final recognition result. The visible and near infrared face recognition is tested on HITSZ Lab2 visible and near infrared face database. The experiment results show that the proposed method extracts the complementary features of near-infrared and visible face images and improves the robustness of unconstrained face recognition. Especially for the circumstance of small training samples, the recognition rate of proposed method can reach 96.13%, which has improved significantly than 92.75 % of the method based on statistical feature fusion.
Hyperspectral imaging, recording intrinsic spectral information of the skin cross different spectral bands, become an important issue for robust face recognition. However, the main challenges for hyperspectral face recognition are high data dimensionality, low signal to noise ratio and inter band misalignment. In this paper, hyperspectral face recognition based on LBP (Local binary pattern) and SWLD (Simplified Weber local descriptor) is proposed to extract discriminative local features from spatio-spectral fusion information. Firstly, the spatio-spectral fusion strategy based on statistical information is used to attain discriminative features of hyperspectral face images. Secondly, LBP is applied to extract the orientation of the fusion face edges. Thirdly, SWLD is proposed to encode the intensity information in hyperspectral images. Finally, we adopt a symmetric Kullback-Leibler distance to compute the encoded face images. The hyperspectral face recognition is tested on Hong Kong Polytechnic University Hyperspectral Face database (PolyUHSFD). Experimental results show that the proposed method has higher recognition rate (92.8%) than the state of the art hyperspectral face recognition algorithms.
Visible face recognition systems, being vulnerable to illumination, expression, and pose, can not achieve robust performance in unconstrained situations. Meanwhile, near infrared face images, being light- independent, can avoid or limit the drawbacks of face recognition in visible light, but its main challenges are low resolution and signal noise ratio (SNR). Therefore, near infrared and visible fusion face recognition has become an important direction in the field of unconstrained face recognition research. In this paper, a novel fusion algorithm in non-subsampled contourlet transform (NSCT) domain is proposed for Infrared and visible face fusion recognition. Firstly, NSCT is used respectively to process the infrared and visible face images, which exploits the image information at multiple scales, orientations, and frequency bands. Then, to exploit the effective discriminant feature and balance the power of high-low frequency band of NSCT coefficients, the local Gabor binary pattern (LGBP) and Local Binary Pattern (LBP) are applied respectively in different frequency parts to obtain the robust representation of infrared and visible face images. Finally, the score-level fusion is used to fuse the all the features for final classification. The visible and near infrared face recognition is tested on HITSZ Lab2 visible and near infrared face database. Experiments results show that the proposed method extracts the complementary features of near-infrared and visible-light images and improves the robustness of unconstrained face recognition.
With the optical sensing technology development, the hyperspectral camera has decreased their price significantly and obtained better resolution and quality. Hyperspectral imaging, recording intrinsic spectral information of the skin at different spectral bands, become a good issue for high performance face recognition. However, there are also many new challenges for hyperspectral face recognition, such as high data dimensionality, low signal to noise ratio and inter band misalignment. This paper proposes a hyperspectral face recognition method based on the covariance fusion of spatio-spectral information and local binary pattern (LBP). Firstly, a cube is slid over the hyperspectral face cube, and each cube is rearranged into a two-dimensional matrix for each overlapping window. Secondly, covariance matrix of each two-dimensional matrix is computed to fully incorporate local spatial information and spectral feature. Thirdly, the trace of each covariance matrix is calculated to replace the pixel values of the fusion image in the corresponding location respectively. Finally, LBP is applied for the fusion hyperspectral face image to get final recognition result. The hyperspectral face recognition is tested on Hong Kong Polytechnic University Hyperspectral Face database (PolyUHSFD). Experimental results show that the proposed method has higher recognition rate (90.8%) and lower computational complexity than the state of the art hyperspectral face recognition algorithms.
Drunk driving problem is a serious threat to traffic safety. Automatic drunk driver identification is vital to improve the traffic safety. This paper copes with automatic drunk driver detection using far infrared thermal images by the holistic features. To improve the robustness of drunk driver detection, instead of traditional local pixels, a holistic feature extraction method is proposed to attain compact and discriminative features for infrared face drunk identification. Discrete cosine transform (DCT) in discrete wavelet transform (DWT) domain is used to extract the useful features in infrared face images for its high speed. Then, the first six DCT coefficients are retained for drunk classification by means of “Z” scanning. Finally, SVM is applied to classify the drunk person. Experimental results illustrate that the accuracy rate of proposed infrared face drunk identification can reach 98.5% with high computation efficiency, which can be applied in real drunk driver detection system.
Infrared facial imaging, being light- independent, and not vulnerable to facial skin, expressions and posture, can avoid or limit the drawbacks of face recognition in visible light. Robust feature selection and representation is a key issue for infrared face recognition research. This paper proposes a novel infrared face recognition method based on local binary pattern (LBP). LBP can improve the robust of infrared face recognition under different environment situations. How to make full use of the discriminant ability in LBP patterns is an important problem. A search algorithm combination binary particle swarm with SVM is used to find out the best discriminative subset in LBP features. Experimental results show that the proposed method outperforms traditional LBP based infrared face recognition methods. It can significantly improve the recognition performance of infrared face recognition.
Due to low resolutions of infrared face image, the local texture features are more appreciated for infrared face feature extraction. To extract rich facial texture features, infrared face recognition based on local binary pattern (LBP) and center-symmetric local derivative pattern (CS-LDP) is proposed. Firstly, LBP is utilized to extract the first order texture from the original infrared face image; Secondly, the second order features are extracted CS-LDP. Finally, an adaptive weighted fusion algorithm based separability discriminant criterion is proposed to get final recognition features. Experimental results on our infrared faces databases demonstrate that separability oriented fusion of LBP and CS-LDP contributes complementary discriminant ability, which can improve the performance for infrared face recognition
Compact and discriminative feature extraction is a challenging task for infrared face recognition. In this paper, we propose an infrared face recognition method using Partial Least Square (PLS) regression on Discrete Cosine Transform (DCT) coefficients. With the strong ability for data de-correlation and compact energy, DCT is studied to get the compact features in infrared face. To dig out discriminative information in DCT coefficients, class-specific One-to-Rest Partial Least Squares (PLS) classifier is learned for accurate classification. The infrared data were collected by an infrared camera Thermo Vision A40 supplied by FLIR Systems Inc. The experimental results show that the recognition rate of the proposed algorithm can reach 95.8%, outperforms that of the state of art infrared face recognition methods based on Linear Discriminant Analysis (LDA) and DCT.
The conventional LBP-based feature as represented by the local binary pattern (LBP) histogram still has room for performance improvements. This paper focuses on the dimension reduction of LBP micro-patterns and proposes an improved infrared face recognition method based on LBP histogram representation. To extract the local robust features in infrared face images, LBP is chosen to get the composition of micro-patterns of sub-blocks. Based on statistical test theory, Kruskal-Wallis (KW) feature selection method is proposed to get the LBP patterns which are suitable for infrared face recognition. The experimental results show combination of LBP and KW features selection improves the performance of infrared face recognition, the proposed method outperforms the traditional methods based on LBP histogram, discrete cosine transform(DCT) or principal component analysis(PCA).
To extract the discriminative information from the sparse representation of infrared face, infrared face recognition
method combining multiwavelet transform and principal component analysis (PCA) is proposed in this paper. Firstly,
the effective information in infrared face is represented by multi-wavelet transformation. Then, to integrate more useful information to infrared face recognition, we assign the corresponding weights to different sub-bands in multi-wavelet domain. Finally, based on the weighted fusion distance, the 1-NN classifier is applied to get final recognition result. The experiment results show that the recognition performance of sparse representation based on multi-wavelet representation outperforms that of method based on usual wavelet representation; and the proposed infrared face method considering the useful information in different sub-bands of multiwavelet has better recognition performance, compared with the method based on approximate sub-band.
The traditional local binary pattern (LBP) histogram representation extracts the local micropatterns and assigns the same weight to all local micropatterns. To combine the different contributions of local micropatterns to face recognition, this paper proposes a weighted LBP histogram based on Weber's law. First, inspired by psychological Weber's law, intensity of local micropattern is defined by the ratio between two terms: one is relative intensity differences of a central pixel against its neighbors and the other is intensity of local central pixel. Second, regarding the intensity of local micropattern as its weight, the weighted LBP histogram is constructed with the defined weight. Finally, to make full use of the space location information and lessen the complexity of recognition, the partitioning and locality preserving projection are applied to get final features. The proposed method is tested on our infrared face databases and yields the recognition rate of 99.2% for same-session situation and 96.4% for elapsed-time situation compared to the 97.6 and 92.1% produced by the method based on traditional LBP.
In this paper, a novel method for infrared face recognition based on blood perfusion is proposed in this paper. Firstly,
thermal images are converted into blood perfusion domain to enlarge between-class distance and lessen within-class
distance, which makes full use of the biological feature of the human face. Based on the ratio of between-class distance
to within-class distance (Ratio of Distance (RD)) in sub-blocks, block-PCA is utilized to get the local discrimination
information, which can solve the small sample size problem (the null space problem). Finally, The FLD is applied to the
holistic features combined by the extracted coefficients from the information of all sub-blocks. The experiments illustrate
that the block-PCA+FLD doesn't discard the useful discriminant information in the holistic characters and the method
proposed in this paper has better performance compared with traditional methods.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.