Learning Conceptual-Contextual Embeddings for Medical Text

External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks in a similar fashion to pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.

Authors

• Xiao Zhang Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
• Dejing Dou Boston Consulting Group, China
• Ji Wu Department of Electronic Engineering, Tsinghua University College of AI, Tsinghua University Beijing National Research Center for Information Science and Technology

Keywords

No keywords are indexed for this paper.