Author name cluster

Hongbin Pei

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

AAAI Conference 2025 Conference Paper

Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models

Jie Ma
Zhitao Gao
Qi Chai
Wangchun Sun
Pinghui Wang
Hongbin Pei
Jing Tao
Lingyun Song

Large Language Models (LLMs) may suffer from hallucinations in real-world applications due to the lack of relevant knowledge. In contrast, knowledge graphs encompass extensive, multi-relational structures that store a vast array of symbolic facts. Consequently, integrating LLMs with knowledge graphs has been extensively explored, with Knowledge Graph Question Answering (KGQA) serving as a critical touchstone for the integration. This task requires LLMs to answer natural language questions by retrieving relevant triples from knowledge graphs. However, existing methods face two significant challenges: *excessively long reasoning paths distracting from the answer generation*, and *false-positive relations hindering the path refinement*. In this paper, we propose an iterative interactive KGQA framework that leverages the interactive learning capabilities of LLMs to perform reasoning and Debating over Graphs (DoG). Specifically, DoG employs a subgraph-focusing mechanism, allowing LLMs to perform answer trying after each reasoning step, thereby mitigating the impact of lengthy reasoning paths. On the other hand, DoG utilizes a multi-role debate team to gradually simplify complex questions, reducing the influence of false-positive relations. This debate mechanism ensures the reliability of the reasoning process. Experimental results on five public datasets demonstrate the effectiveness and superiority of our architecture. Notably, DoG outperforms the state-of-the-art method ToG by 23.7% and 9.1% in accuracy on WebQuestions and GrailQA, respectively. Furthermore, the integration experiments with various LLMs on the mentioned datasets highlight the flexibility of DoG.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Deliberation on Priors: Trustworthy Reasoning of Large Language Models on Knowledge Graphs

Jie Ma
NING QU
Zhitao Gao
Xing Rui
Jun Liu
Hongbin Pei
Jiang Xie
Lingyun Song

Knowledge graph-based retrieval-augmented generation seeks to mitigate hallucinations in Large Language Models (LLMs) caused by insufficient or outdated knowledge. However, existing methods often fail to fully exploit the prior knowledge embedded in knowledge graphs (KGs), particularly their structural information and explicit or implicit constraints. The former can enhance the faithfulness of LLMs' reasoning, while the latter can improve the reliability of response generations. Motivated by these, we propose a trustworthy reasoning framework, termed Deliberation over Priors (\texttt{DP}), which sufficiently utilizes the priors contained in KGs. Specifically, \texttt{DP} adopts a progressive knowledge distillation strategy that integrates structural priors into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky Optimization, thereby improving the faithfulness of relation path generation. Furthermore, our framework employs a reasoning-introspection strategy, which guides LLMs to perform refined reasoning verification based on extracted constraint priors, ensuring the reliability of response generation. Extensive experiments on three benchmark datasets demonstrate that \texttt{DP} achieves new state-of-the-art performance, especially a H@1 improvement of 13% on the ComplexWebQuestions dataset, and generates highly trustworthy responses. We also conduct various analyses to verify its flexibility and practicality. Code is available at https: //github. com/mira-ai-lab/Deliberation-on-Priors.

PDF Details

ICML Conference 2025 Conference Paper

Non-Stationary Predictions May Be More Informative: Exploring Pseudo-Labels with a Two-Phase Pattern of Training Dynamics

Hongbin Pei
Jingxin Hai
Yu Li
Huiqi Deng
Denghao Ma
Jie Ma 0001
Pinghui Wang
Jing Tao

Pseudo-labeling is a widely used strategy in semi-supervised learning. Existing methods typically select predicted labels with high confidence scores and high training stationarity, as pseudo-labels to augment training sets. In contrast, this paper explores the pseudo-labeling potential of predicted labels that do not exhibit these characteristics. We discover a new type of predicted labels suitable for pseudo-labeling, termed two-phase labels, which exhibit a two-phase pattern during training: they are initially predicted as one category in early training stages and switch to another category in subsequent epochs. Case studies show the two-phase labels are informative for decision boundaries. To effectively identify the two-phase labels, we design a 2- phasic metric that mathematically characterizes their spatial and temporal patterns. Furthermore, we propose a loss function tailored for two-phase pseudo-labeling learning, allowing models not only to learn correct correlations but also to eliminate false ones. Extensive experiments on eight datasets show that our proposed 2- phasic metric acts as a powerful booster for existing pseudo-labeling methods by additionally incorporating the two-phase labels, achieving an average classification accuracy gain of 1. 73% on image datasets and 1. 92% on graph datasets.

Details

EAAI Journal 2024 Journal Article

Deep click interest network for reranking hotels

Denghao Ma
Hongbin Pei
Peng Bao
Xueqiang Lv
Liang Shen
Genliang Yi
Haoxing Wen

Nowadays, e-commerce platforms of hotels have become a new trend to help people book hotels online. Interest modeling aims to automatically construct user interests that are critical for e-commerce platforms. Although interest modeling has achieved noticeable successes in product recommendations, it has not been explored in hotel search, especially in the reranking stage. This paper studies the interest modeling task for reranking hotels, and identifies two key challenges—high behavior sparsity and large interest gap. To address the challenges, we propose the deep click interest network (DCIN). High behavior sparsity keeps us from extracting rich semantics to characterize user interest preference. Accordingly, we propose the deep click concept to model the multi-view semantics of a click and then enrich the semantics of both a single click and a click sequence, so that the difficulty caused by the behavior sparsity can be addressed. To address the interest gap challenge, DCIN models user interests with two cascaded units: (i) Mutual-attention Interest Calibration Unit (ICU), which uses the candidate hotel to calibrate the embedding of every click, since the candidate hotel is selected by the ranking stage and can represent user current interests to some extent. So the gap between the calibrated embedding and user current interests can be diminished; (ii) Multi-attention Interest Aggregation Unit (IAU), which estimates the weight of a calibrated embedding from multiple perspectives, i. e. , context, user feedback and interest consistency. So the calibrated embeddings, which are relevant to the current query context and important to the user, dominate user current interests, and thus the interest gap can be further diminished. Offline experiments over two large datasets and online A/B testing over the platform of Meituan-Hotel show that DCIN significantly outperforms the baselines and improved baselines. Notably, DCIN has been deployed in Meituan-Hotel, resulting in an increase of 2. 40% in Click Through Conversion Rate (CTCVR) and that of 1. 09% in Click Through Rate (CTR).

Details DOI

AAAI Conference 2024 Conference Paper

Generalized Variational Inference via Optimal Transport

Jinjin Chi
Zhichao Zhang
Zhiyao Yang
Jihong Ouyang
Hongbin Pei

Variational Inference (VI) has gained popularity as a flexible approximate inference scheme for computing posterior distributions in Bayesian models. Original VI methods use Kullback-Leibler (KL) divergence to construct variational objectives. However, KL divergence has zero-forcing behavior and is completely agnostic to the metric of the underlying data distribution, resulting in bad approximations. To alleviate this issue, we propose a new variational objective by using Optimal Transport (OT) distance, which is a metric-aware divergence, to measure the difference between approximate posteriors and priors. The superior performance of OT distance enables us to learn more accurate approximations. We further enhance the objective by gradually including the OT term using a hyperparameter λ for over-parameterized models. We develop a Variational inference method with OT (VOT) which presents a gradient-based black-box framework for solving Bayesian models, even when the density function of approximate distribution is not available. We provide the consistency analysis of approximate posteriors and demonstrate the practical effectiveness on Bayesian neural networks and variational autoencoders.

PDF Details DOI

AAAI Conference 2024 Conference Paper

HAGO-Net: Hierarchical Geometric Message Passing for Molecular Representation Learning

Hongbin Pei
Taile Chen
Chen A
Huiqi Deng
Jing Tao
Pinghui Wang
Xiaohong Guan

Molecular representation learning has emerged as a game-changer at the intersection of AI and chemistry, with great potential in applications such as drug design and materials discovery. A substantial obstacle in successfully applying molecular representation learning is the difficulty of effectively and completely characterizing and learning molecular geometry, which has not been well addressed to date. To overcome this challenge, we propose a novel framework that features a novel geometric graph, termed HAGO-Graph, and a specifically designed geometric graph learning model, HAGO-Net. In the framework, the foundation is HAGO-Graph, which enables a complete characterization of molecular geometry in a hierarchical manner. Specifically, we leverage the concept of n-body in physics to characterize geometric patterns at multiple spatial scales. We then specifically design a message passing scheme, HAGO-MPS, and implement the scheme as a geometric graph neural network, HAGO-Net, to effectively learn the representation of HAGO-Graph by horizontal and vertical aggregation. We further prove DHAGO-Net, the derivative function of HAGO-Net, is an equivariant model. The proposed models are validated by extensive comparisons on four challenging benchmarks. Notably, the models exhibited state-of-the-art performance in molecular chirality identification and property prediction, achieving state-of-the-art performance on five properties of QM9 dataset. The models also achieved competitive results on molecular dynamics prediction task.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering

Jie Ma
Min Hu
Pinghui Wang
Wangchun Sun
Lingyun Song
Hongbin Pei
Jun Liu
Youtian Du

Audio-Visual Question Answering (AVQA) is a complex multi-modal reasoning task, demanding intelligent systems to accurately respond to natural language queries based on audio-video input pairs. Nevertheless, prevalent AVQA approaches are prone to overlearning dataset biases, resulting in poor robustness. Furthermore, current datasets may not provide a precise diagnostic for these methods. To tackle these challenges, firstly, we propose a novel dataset, MUSIC-AVQA-R, crafted in two steps: rephrasing questions within the test split of a public dataset ( MUSIC-AVQA ) and subsequently introducing distribution shifts to split questions. The former leads to a large, diverse test space, while the latter results in a comprehensive robustness evaluation on rare, frequent, and overall questions. Secondly, we propose a robust architecture that utilizes a multifaceted cycle collaborative debiasing strategy to overcome bias learning. Experimental results show that this architecture achieves state-of-the-art performance on MUSIC-AVQA-R, notably obtaining a significant improvement of 9. 32\%. Extensive ablation experiments are conducted on the two datasets mentioned to analyze the component effectiveness within the debiasing strategy. Additionally, we highlight the limited robustness of existing multi-modal QA methods through the evaluation on our dataset. We also conduct experiments combining various baselines with our proposed strategy on two datasets to verify its plug-and-play capability. Our dataset and code are available at https: //github. com/reml-group/MUSIC-AVQA-R.

PDF Details DOI

ICML Conference 2024 Conference Paper

Multi-Track Message Passing: Tackling Oversmoothing and Oversquashing in Graph Learning via Preventing Heterophily Mixing

Hongbin Pei
Yu Li
Huiqi Deng
Jingxin Hai
Pinghui Wang
Jie Ma 0001
Jing Tao
Yuheng Xiong

The advancement toward deeper graph neural networks is currently obscured by two inherent issues in message passing, oversmoothing and oversquashing. We identify the root cause of these issues as information loss due to heterophily mixing in aggregation, where messages of diverse category semantics are mixed. We propose a novel multi-track graph convolutional network to address oversmoothing and oversquashing effectively. Our basic idea is intuitive: if messages are separated and independently propagated according to their category semantics, heterophilic mixing can be prevented. Consequently, we present a novel multi-track message passing scheme capable of preventing heterophilic mixing, enhancing long-distance information flow, and improving separation condition. Empirical validations show that our model achieved state-of-the-art performance on several graph datasets and effectively tackled oversmoothing and oversquashing, setting a new benchmark of $86. 4$% accuracy on Cora.

Details

NeurIPS Conference 2020 Conference Paper

Curvature Regularization to Prevent Distortion in Graph Embedding

Hongbin Pei
Bingzhe Wei
Kevin Chang
Chunxu Zhang
Bo Yang

Recent research on graph embedding has achieved success in various applications. Most graph embedding methods preserve the proximity in a graph into a manifold in an embedding space. We argue an important but neglected problem about this proximity-preserving strategy: Graph topology patterns, while preserved well into an embedding manifold by preserving proximity, may distort in the ambient embedding Euclidean space, and hence to detect them becomes difficult for machine learning models. To address the problem, we propose curvature regularization, to enforce flatness for embedding manifolds, thereby preventing the distortion. We present a novel angle-based sectional curvature, termed ABS curvature, and accordingly three kinds of curvature regularization to induce flat embedding manifolds during graph embedding. We integrate curvature regularization into five popular proximity-preserving embedding methods, and empirical results in two applications show significant improvements on a wide range of open graph datasets.

PDF Details

ICLR Conference 2020 Conference Paper

Geom-GCN: Geometric Graph Convolutional Networks

Hongbin Pei
Bingzhe Wei
Kevin Chen-Chuan Chang
Yu Lei 0004
Bo Yang 0002

Message-passing neural networks (MPNNs) have been successfully applied in a wide variety of applications in the real world. However, two fundamental weaknesses of MPNNs' aggregators limit their ability to represent graph-structured data: losing the structural information of nodes in neighborhoods and lacking the ability to capture long-range dependencies in disassortative graphs. Few studies have noticed the weaknesses from different perspectives. From the observations on classical neural network and network geometry, we propose a novel geometric aggregation scheme for graph neural networks to overcome the two weaknesses. The behind basic idea is the aggregation on a graph can benefit from a continuous space underlying the graph. The proposed aggregation scheme is permutation-invariant and consists of three modules, node embedding, structural neighborhood, and bi-level aggregation. We also present an implementation of the scheme in graph convolutional networks, termed Geom-GCN, to perform transductive learning on graphs. Experimental results show the proposed Geom-GCN achieved state-of-the-art performance on a wide range of open datasets of graphs.

Details

AAAI Conference 2018 Conference Paper

Group Sparse Bayesian Learning for Active Surveillance on Epidemic Dynamics

Hongbin Pei
Bo Yang
Jiming Liu
Lei Dong

Predicting epidemic dynamics is of great value in understanding and controlling diffusion processes, such as infectious disease spread and information propagation. This task is intractable, especially when surveillance resources are very limited. To address the challenge, we study the problem of active surveillance, i. e. , how to identify a small portion of system components as sentinels to effect monitoring, such that the epidemic dynamics of an entire system can be readily predicted from the partial data collected by such sentinels. We propose a novel measure, the γ value, to identify the sentinels by modeling a sentinel network with row sparsity structure. We design a ﬂexible group sparse Bayesian learning algorithm to mine the sentinel network suitable for handling both linear and non-linear dynamical systems by using the expectation maximization method and variational approximation. The efﬁcacy of the proposed algorithm is theoretically analyzed and empirically validated using both synthetic and real-world data.

PDF Details