HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors

Graph Attention Networks (GATs) that compute node representation by its lower-order neighbors, are state-of-the-art architecture for representation learning with graphs. In practice, however, the high-order neighbors that turn out to be useful, remain largely unemployed in GATs. Efforts on this issue remain to be limited. This paper proposes a simple and effective high-order neighbor GAT (HONGAT) model to both effectively exploit informative high-order neighbors and address over-smoothing at the decision boundary of nodes. Two tightly coupled novel technologies, namely common neighbor similarity and new masking matrix, are introduced. Specifically, high-order neighbors are fully explored by generic high-order common-neighbor-based similarity; in order to prevent severe over-smoothing, typical averaging range no longer works well and a new masking mechanism is employed without any extra hyperparameter. Extensive empirical evaluation on real-world datasets clearly shows the necessity of the new algorithm in the ability of exploring high-order neighbors, which promisingly achieves significant gains over previous state-of-the-art graph attention methods.

Authors

• Heng-Kai Zhang Nanjing University
• Yi-Ge Zhang Nanjing University
• Zhi Zhou Nanjing University
• Yu-Feng Li Nanjing University

Keywords

• ML: Active Learning
• ML: Classification and Regression
• ML: Multi-class/Multi-label Learning & Extreme Classification
• ML: Semi-Supervised Learning
• ML: Transfer, Domain Adaptation, Multi-Task Learning