ECAI Conference 2025 Conference Paper
Optimizing Semantic Consistency Modeling: Task-Specific Tensor Fusion, Multi-Task Multi-Scale Joint Training, and Uncertainty-Aware Distillation
- Zeyu Wei
- Shuo Wang
- Xuemin Liu
- Xiaohui Rong
Semantic consistency evaluation faces two critical challenges: Bi-Encoder models, while efficient, struggle to capture fine-grained interactions between sentence pairs, limiting accuracy; meanwhile, Cross-Encoders and large language models (LLMs), despite superior performance, incur substantial computational costs, hindering practical deployment. This paper proposes a Multi-dimensional Consistency Evaluation Model (MCEM), designed to balance performance and efficiency, enabling precise modeling of sentence pairs and efficient inference across multiple consistency dimensions, including semantic, emotional, and logical aspects. The core innovation of MCEM lies in its integration of gated memory networks with a Multi-Task Mixture-of-Experts (MT-MoE) architecture, which enables fine-grained decomposition and dynamic recombination of tensors to disentangle shared and task-specific features. Furthermore, a multi-granular feature extraction module enhances the model’s ability to capture semantic information at the character, phrase, and sentence levels. Additionally, an uncertainty-aware knowledge distillation mechanism effectively transfers high-confidence knowledge from the Cross-Encoder branch to the lightweight path, significantly improving inference efficiency while maintaining high performance. Experimental results demonstrate that MCEM achieves state-of-the-art (SOTA) performance across multiple consistency benchmarks and exhibits strong generalization capability under both adversarial perturbations and previously unseen data structures.