Author name cluster

Shiyao Yan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

AAAI Conference 2023 Conference Paper

TOT：Topology-Aware Optimal Transport for Multimodal Hate Detection

Linhao Zhang
Li Jin
Xian Sun
Guangluan Xu
Zequn Zhang
Xiaoyu Li
Nayu Liu
Qing Liu

Multimodal hate detection, which aims to identify the harmful content online such as memes, is crucial for building a wholesome internet environment. Previous work has made enlightening exploration in detecting explicit hate remarks. However, most of their approaches neglect the analysis of implicit harm, which is particularly challenging as explicit text markers and demographic visual cues are often twisted or missing. The leveraged cross-modal attention mechanisms also suffer from the distributional modality gap and lack logical interpretability. To address these semantic gap issues, we propose TOT: a topology-aware optimal transport framework to decipher the implicit harm in memes scenario, which formulates the cross-modal aligning problem as solutions for optimal transportation plans. Specifically, we leverage an optimal transport kernel method to capture complementary information from multiple modalities. The kernel embedding provides a non-linear transformation ability to reproduce a kernel Hilbert space (RKHS), which reflects significance for eliminating the distributional modality gap. Moreover, we perceive the topology information based on aligned representations to conduct bipartite graph path reasoning. The newly achieved state-of-the-art performance on two publicly available benchmark datasets, together with further visual analysis, demonstrate the superiority of TOT in capturing implicit cross-modal alignment.

PDF Details DOI

AAAI Conference 2022 Conference Paper

PolygonE: Modeling N-ary Relational Data as Gyro-Polygons in Hyperbolic Space

Shiyao Yan
Zequn Zhang
Xian Sun
Guangluan Xu
Shuchao Li
Qing Liu
Nayu Liu
Shensi Wang

N-ary relational knowledge base (KBs) embedding aims to map binary and beyond-binary facts into low-dimensional vector space simultaneously. Existing approaches typically decompose n-ary relational facts into subtuples, and they generally model n-ary relational KBs in Euclidean space. However, n-ary relational facts are semantically and structurally intact; decomposition undermines the semantical and structural integrity. Moreover, compared to the binary relational KBs, n-ary ones are characterized by more abundant and complicated hierarchy structures, which could not be well expressed in Euclidean space. To address the issues, we propose a gyro-polygon embedding framework to realize n-ary fact integrity keeping and hierarchy capturing, termed PolygonE. Specifically, n-ary relational facts are modeled as gyropolygons in the hyperbolic space, where we denote entities in facts as vertexes of gyro-polygons and relations as entity translocation operations. Importantly, we design a fact plausibility measuring strategy based on the vertex-gyrocentroid geodesic to optimize the relation-adjusted gyro-polygon. Experimental results demonstrate that PolygonE shows SOTA performance on all benchmark datasets and generalizes well on binary data. Finally, we also visualize the embedding to help comprehend PolygonE’s awareness of hierarchies.

PDF Details