Toggle Main Menu Toggle Search

Open Access padlockePrints

Spatio-temporal modelling with multi-gradient features and elongated quinary pattern descriptor for dynamic facial expression recognition

Lookup NU author(s): Emeritus Professor Satnam Dlay, Professor Jonathon Chambers


Full text for this publication is not currently held within this repository. Alternative links are provided below where available.


© 2023 Elsevier LtdWe propose a new spatio-temporal modelling approach for Dynamic Facial Expression Recognition (DFER). We first convert the domain of the spatial images in the sequence to the gradient of magnitude and angle images at different orientations. Robust gradient components are developed to deal with difficult types of illuminations, such as darkness, by forming the eight edge responses of the Gaussian mask. To describe the dynamic Facial Expression (FE) changes we extend the Elongated Quinary Pattern (EQP) descriptor to encode separately the anisotropic structure of the uniform patterns from Three Orthogonal Planes (TOP) of each gradient sequence. Then each encoded sequence is divided into a stack of block volumes in the XY, XT and YT planes. For each plane, the co-occurrence of histogram features are calculated from each block volume and concatenated together. Simple three-dimensional histogram features are generated by concatenating the histogram features of all planes. A Multi Classifier System (MCS) based on a multi-class Support Vector Machine (SVM) is adopted to combine all scores for the encoded sequences. The proposed approach is evaluated with the challenging MMI and Oulu-CASIA databases with different set-ups and advantage has been shown in terms of generalisation to different databases, together with robustness against difficult pose variations and illumination changes. In terms of Recognition Accuracy (RA), a comparison is established with DFER methods in the literature. A high recognition rate of 79.23% is attained in the case of six classes when applied to the MMI database which surpasses all the state-of-the-art results.

Publication metadata

Author(s): Al-Sumaidaee SAM, Abdullah MAM, Al-Nima RRO, Dlay SS, Chambers JA

Publication type: Article

Publication status: Published

Journal: Pattern Recognition

Year: 2023

Volume: 142

Print publication date: 01/10/2023

Online publication date: 04/05/2023

Acceptance date: 26/04/2023

ISSN (print): 0031-3203

Publisher: Elsevier Ltd


DOI: 10.1016/j.patcog.2023.109647


Altmetrics provided by Altmetric