Author name cluster

Dianbo Sui

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

AAAI Conference 2025 Conference Paper

HFF-Tracker: A Hierarchical Fine-grained Fusion Tracker for Referring Multi-Object Tracking

Zeyong Zhao
Yanchao Hao
Minghao Zhang
Qingbin Liu
Bo Li
Dianbo Sui
Shizhu He
Xi Chen

Referring Multi-Object Tracking (RMOT) aims to track multiple objects based on a provided language expression. Although prior studies have sought to accomplish this by integrating an textual module into the multi-object tracker, these methods combine text and image features in a basic way, neglecting the importance of text features. In this study, we propose a Hierarchical Fine-grained text-image Fusion tracker, named HFF-Tracker, which can perform fine-grained fusion of pixel-level visual features and text features across various semantic levels. Specifically, we have devised a Hierarchical Multi-Modal Fusion (HMMF) module to merge text and image features at an early stage in a hierarchical and detailed manner. The Text-Guided Decoder (TGD) is designed to provide the query with prior semantic information during the decoding process. Additionally, we have crafted a Text-Guided Prediction Head (TGPH) that utilizes text information to enhance the performance of the prediction head. Furthermore, we have implemented an adaptive Look-Back training strategy to maximize the utilization of valuable labeled data. Extensive experiments on the Refer-KITTI dataset and the Refer-KITTI-V2 dataset demonstrate that our proposed HFF-Tracker outperforms other state-of-the-art methods with remarkable margins.

PDF Details DOI

ICML Conference 2025 Conference Paper

Maximizing Intermediate Checkpoint Value in LLM Pretraining with Bayesian Optimization

Deyuan Liu
Zecheng Wang
Bingning Wang
Weipeng Chen
Chunshan Li
Zhiying Tu
Dianhui Chu
Dianbo Sui

The rapid proliferation of large language models (LLMs), such as GPT-4 and Gemini, underscores the intense demand for resources during their training processes, posing significant challenges due to substantial computational and environmental costs. In this paper, we introduce a novel checkpoint merging strategy aimed at making efficient use of intermediate checkpoints during LLM pretraining. This method utilizes intermediate checkpoints with shared training trajectories, and is rooted in an extensive search space exploration for the best merging weight via Bayesian optimization. Through various experiments, we demonstrate that: (1) Our proposed methodology exhibits the capacity to augment pretraining, presenting an opportunity akin to obtaining substantial benefits at minimal cost; (2) Our proposed methodology, despite requiring a given held-out dataset, still demonstrates robust generalization capabilities across diverse domains, a pivotal aspect in pretraining.

Details

NeurIPS Conference 2025 Conference Paper

VPO: Reasoning Preferences Optimization Based on $\mathcal{V}$-Usable Information

Zecheng Wang
Chunshan Li
Yupeng Zhang
Han Liu
Bingning Wang
Dianhui Chu
Dianbo Sui

Direct Preference Optimization (DPO) is a widely used preference optimization algorithm in large language model (LLM) alignment, which reparameterizes the reward function in reinforcement learning with human feedback (RLHF) without requiring a separate reward model. However, during the DPO training process, when a large negative gradient is applied to low-confidence samples, LLMs with a softmax output head tend to squeeze the confidence in the model's output distribution towards the highest-confidence sentence, which may lead to a decrease in the confidence of both preference and non-preference samples, while increasing the confidence of unrelated tokens. This phenomenon becomes more complex in reasoning tasks. In this work, focusing on reasoning tasks, we propose VPO, a negative gradient constraint method for human non-preference samples based on $\mathcal{V}$-usable information. By using $\mathcal{V}$-usable information to measure the similarity between preference pairs and selectively constrain the negative gradient, VPO can alleviate the squeezing effect of DPO, enhance alignment with the generation objective, and maintain the model's ability to distinguish between preference and non-preference samples. We compare VPO with DPO and its latest variants on mathematical reasoning tasks using the LLama 3. 1 and Qwen 2. 5 series, including both Base and Instruct models. Our results demonstrate that VPO consistently and significantly outperforms existing methods. Specifically, on Qwen2. 5-7B-Base, VPO achieves 7. 80\% and 13. 25\% improvement over DPO on MATH500 and AMC23, respectively. We also conduct ablation experiments and in-depth analysis on VPO to explain its effectiveness and rationale.

PDF Details

AAAI Conference 2025 Conference Paper

VTG-LLM: Integrating Timestamp Knowledge into Video LLMs for Enhanced Video Temporal Grounding

Yongxin Guo
Jingyu Liu
Mingda Li
Dingxin Cheng
Xiaoying Tang
Dianbo Sui
Qingbin Liu
Xi Chen

Video Temporal Grounding (VTG) strives to accurately pinpoint event timestamps in a specific video using linguistic queries, significantly impacting downstream tasks like video browsing and editing. Unlike traditional task-specific models, Video Large Language Models (video LLMs) can handle multiple tasks concurrently in a zero-shot manner. Consequently, exploring the application of video LLMs for VTG tasks has become a burgeoning research area. However, despite considerable advancements in video content understanding, video LLMs often struggle to accurately pinpoint timestamps within videos, limiting their effectiveness in VTG tasks. To address this, we introduce VTG-LLM, a model designed to enhance video LLMs' timestamp localization abilities. Our approach includes: (1) effectively integrating timestamp knowledge into visual tokens; (2) incorporating absolute-time tokens to manage timestamp knowledge without concept shifts; and (3) introducing a lightweight, high-performance, slot-based token compression technique designed to accommodate the demands of a large number of frames to be sampled for VTG tasks. Additionally, we present VTG-IT-120K, a collection of publicly available VTG datasets that we have re-annotated to improve upon low-quality annotations. Our comprehensive experiments demonstrate the superior performance of VTG-LLM in comparison to other video LLM methods across a variety of VTG tasks.

PDF Details DOI

TIST Journal 2024 Journal Article

A Federated Social Recommendation Approach with Enhanced Hypergraph Neural Network

Hongliang Sun
Zhiying Tu
Dianbo Sui
Bolin Zhang
Xiaofei Xu

In recent years, the development of online social network platforms has led to increased research efforts in social recommendation systems. Unlike traditional recommendation systems, social recommendation systems utilize both user-item interactions and user-user social relations to recommend relevant items, taking into account social homophily and social influence. Graph neural network (GNN)-based social recommendation methods have been proposed to model these item interactions and social relations effectively. However, existing GNN-based methods rely on centralized training, which raises privacy concerns and faces challenges in data collection due to regulations and privacy restrictions. Federated learning has emerged as a privacy-preserving alternative. Combining federated learning with GNN-based methods for social recommendation can leverage their respective advantages, but it also introduces new challenges: (1) existing federated recommendation systems often lack the capability to process heterogeneous data, such as user-item interactions and social relations; (2) due to the sparsity of data distributed across different clients, capturing the higher-order relationship information among users becomes challenging and is often overlooked by most federated recommendation systems. To overcome these challenges, we propose a federated social recommendation approach with enhanced hypergraph neural network (HGNN). We introduce HGNN to learn user and item embeddings in federated recommendation systems, leveraging the hypergraph structure to address the heterogeneity of data. Based on carefully crafted triangular motifs, we merge user and item nodes to construct hypergraphs on local clients, capturing specific triangular relations. Multiple HGNN channels are used to encode different categories of high-order relations, and an attention mechanism is applied to aggregate the embedded information from these channels. Our experiments on real-world social recommendation datasets demonstrate the effectiveness of the proposed approach. Extensive experiment results on three publicly available datasets validate the effectiveness of the proposed method.

Details DOI