Concept Representation by Learning Explicit and Implicit Concept Couplings

Generating the precise semantic representation of a word or concept is a fundamental task in natural language processing. Recent studies which incorporate semantic knowledge into word embedding have shown their potential in improving the semantic representation of a concept. However, existing approaches only achieved limited performance improvement as they usually 1) model a word’s semantics from some explicit aspects while ignoring the intrinsic aspects of the word, 2) treat semantic knowledge as a supplement of word embeddings, and 3) consider partial relations between concepts while ignoring rich coupling relations between them, such as explicit concept co-occurrences in descriptive texts in a corpus as well as concept hyperlink relations in a knowledge network, and implicit couplings between concept co-occurrences and hyperlinks. In human consciousness, a concept is always associated with various couplings that exist within/between descriptive texts and knowledge networks, which inspires us to capture as many concept couplings as possible for building a more informative concept representation. We thus propose a neural coupled concept representation (CoupledCR) framework and its instantiation: a coupled concept embedding (CCE) model. CCE first learns two types of explicit couplings that are based on concept co-occurrences and hyperlink relations, respectively, and then learns a type of high-level implicit couplings between these two types of explicit couplings for better concept representation. Extensive experimental results on six real-world datasets show that CCE significantly outperforms eight state-of-the-art word embeddings and semantic representation methods.

Authors

• Wenpeng Lu Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences) Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science
• Yuteng Zhang
• Shoujin Wang
• Heyan Huang Beijing Institute of Technology
• Qian Liu Sea AI Lab
• Sheng Luo

Keywords

• Couplings
• Semantics
• Knowledge engineering
• Knowledge based systems
• Hypertext systems
• Learning systems
• Natural language processing
• Conceptual Representations
• Explicit Coupling
• Spearman Correlation
• Knowledge Base
• Rank Correlation
• Text Data
• Representation Learning
• Real-world Datasets
• Textual Descriptions
• Word Embedding
• Sentiment Analysis
• Semantic Knowledge
• Semantic Representations
• Large Corpus
• Word Representations
• Text Classification
• Similar Learning
• Explicit Relationship
• Occurrence Of Words
• Type Of Coupling
• Semantic Similarity
• Target Word
• Embedding Dimension
• Truncated Singular Value Decomposition
• Target Concept
• Singular Value Decomposition
• Semantic Network
• Conditional Probability Distribution
• Ablation Method
• Word Pairs
• Concept Representation
• Coupling Learning
• non-IID Learning
• Word Similarity