Deep Semantic Role Labeling With Self-Attention

Semantic Role Labeling (SRL) is believed to be a crucial step towards natural language understanding and has been widely studied. Recent years, end-to-end SRL with recurrent neural networks (RNN) has gained increasing attention. However, it remains a major challenge for RNNs to handle structural information and long range dependencies. In this paper, we present a simple and effective architecture for SRL which aims to address these problems. Our model is based on self-attention which can directly capture the relationships between two tokens regardless of their distance. Our single model achieves F1 = 83. 4 on the CoNLL-2005 shared task dataset and F1 = 82. 7 on the CoNLL-2012 shared task dataset, which outperforms the previous state-of-the-art results by 1. 8 and 1. 0 F1 score respectively. Besides, our model is computationally efficient, and the parsing speed is 50K tokens per second on a single Titan X GPU.

Authors

• Zhixing Tan
• Mingxuan Wang Bytedance Seed, China
• Jun Xie Google DeepMind
• Yidong Chen School of Informatics, Xiamen University, China Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China
• Xiaodong Shi Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen, China Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China

Keywords

No keywords are indexed for this paper.