Meta Multi-Task Learning for Sequence Modeling

Semantic composition functions have been playing a pivotal role in neural representation learning of text sequences. In spite of their success, most existing models suffer from the underfitting problem: they use the same shared compositional function on all the positions in the sequence, thereby lacking expressive power due to incapacity to capture the richness of compositionality. Besides, the composition functions of different tasks are independent and learned from scratch. In this paper, we propose a new sharing scheme of composition function across multiple tasks. Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the taskspecific semantic composition models. We conduct extensive experiments on two types of tasks, text classification and sequence tagging, which demonstrate the benefits of our approach. Besides, we show that the shared meta-knowledge learned by our proposed model can be regarded as off-theshelf knowledge and easily transferred to new tasks.

Authors

• Junkun Chen
• Xipeng Qiu School of Computer Science, Fudan University
• Pengfei Liu
• Xuanjing Huang School of Computer Science, Fudan University, Shanghai, China Shanghai Collaborative Innovation Center of Intelligent Visual Computing, Shanghai, China

Keywords

No keywords are indexed for this paper.