Alignment-Enhanced Transformer for Constraining NMT with Pre-Specified Translations

We investigate the task of constraining NMT with prespecified translations, which has practical significance for a number of research and industrial applications. Existing works impose pre-specified translations as lexical constraints during decoding, which are based on word alignments derived from target-to-source attention weights. However, multiple recent studies have found that word alignment derived from generic attention heads in the Transformer is unreliable. We address this problem by introducing a dedicated head in the multi-head Transformer architecture to capture external supervision signals. Results on five language pairs show that our method is highly effective in constraining NMT with pre-specified translations, consistently outperforming previous methods in translation quality.

Authors

• Kai Song
• Kun Wang PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology, China
• Heng Yu The Digital Port Technologies Lab, School of Computer Science, University of Nottingham Ningbo China
• Yue Zhang Monash University
• Zhongqiang Huang College of Informatics, Huazhong Agricultural University, Wuhan, China
• Weihua Luo Alibaba International Digital Commerce
• Xiangyu Duan
• Min Zhang College of Intelligence and Computing, Tianjin University

Keywords

No keywords are indexed for this paper.