EAAI Journal 2025 Journal Article
Lightweight binary convolutional-transformers fusion network for facial expression recognition
- Xing Jin
- Xiyin Wu
- Libo Weng
- Qiaolin Ye
Deep learning-based methods exploit local facial regions for facial expression recognition (FER) but overlook long-range dependencies of facial muscle movements. Moreover, deploying existing large-scale deep networks on mobile devices remains a huge challenge. To this end, this paper designs a mini-binary transformer (MiniBTR) for FER. Specifically, we first construct image patches by regions of interest (ROIs) which are associated with facial action units (AUs) and represent these patches by the histogram of oriented gradient (HOG) feature. Then, by integrating the complementary advantages of convolutional layers (Convs) and multi-head self-attention mechanism (MSA), an effective and efficient mini-transformer (MiniTR) architecture with 69K parameters is proposed to follow the human–machine collaborative strategy. Finally, we design the binary model MiniBTR by extending MiniTR with a binary operation. Extensive experimental results on four publicly available datasets demonstrate that the MiniBTR yields comparable results in terms of recognition accuracy, model size and inference speed while offering a more deployable yet high-performing alternative.