Hong Zhu

EAAI Journal 2025 Journal Article

Joint feature extraction and alignment in object tracking with vision-language model

• Hong Zhu
• Qingyang Lu
• Lei Xue
• Guanglin Yuan
• Kaihua Zhang

Vision-language tracking is a new rising research topic that focuses on locating the target object in a video sequence using its language description. The main challenge is to model the correspondence between the vision-language input references and the test image, that is, vision-language fusion learning. Nevertheless, current vision-language trackers still follow the conventional framework in vision-only tracking, overlooking the heterogeneity between vision and language. To tackle the problem, we present a novel framework for vision-language tracking by combining joint feature extraction, alignment, and interaction. Before fusing, we first perform joint feature extraction and modality alignment in a vision-language embedding space learned by our proposed Adapter-equipped model, so as to obtain semantically unified feature representation. The model, named A-CLIP, which integrates some lightweight Adapters into CLIP (Contrastive Language-Image Pre-training), is an efficient method of seamlessly transferring the large-scale foundation model to downstream tracking task. It inherits the remarkable generalization ability from the large-scale model, providing a solution to the challenge of limited training data in vision-language tracking. Further, a transformer-based deep fusion is designed to model the multi-source correlations, highlighting the localization-relevant cues for accurate reasoning. Our proposed method has been extensively evaluated on three benchmark datasets, and the experimental results provide both quantitative and qualitative analyses that demonstrate its superior performance compared to the state-of-the-arts.

EAAI Journal 2024 Journal Article

Predictive resilience assessment featuring diffusion reconstruction for road networks under rainfall disturbances

• Di Zang
• Chenguang Wei
• Jiayi Zhao
• Junqi Zhang
• Keshuang Tang
• Hong Zhu

The ability of road networks to withstand external disturbances is a crucial measure of transportation system performance, where resilience distinctly emerges as an effective perspective for its unique insights into the system’s resistance and recovery capabilities. In the face of unforeseen resilience disturbance events, predictive and accurate assessment of road network resilience is essential for better traffic regulation and emergency response management. However, existing resilience assessment methods of road networks are insufficient: they lack reliable real-time big-data analysis, do not possess predictive capabilities for guiding decision-making, and have a narrow view with single-dimensional resilience indicators. To address these issues, focusing on rainfall disturbance scenarios, this work introduces a novel resilience assessment method, which is predictive and real-time, consisting of two components: a deep learning traffic indicator prediction model and a comprehensive resilience assessment model. Firstly, we propose a two-stage traffic indicator prediction model, namely the Conditional Diffusion-Reconstruction-based Graph Neural Network (CDRGNN), which particularly enhances disturbance-scenario prediction accuracy, thereby providing reliable foresight in aid of the following assessments. Subsequently, we develop a resilience assessment model featuring structural-functional comprehensive resilience indicators established through shortest-path aggregation, and the overall resilience assessment is performed through comparative analysis using indicators obtained in real-time with historical non-disruptive resilience benchmarks. In a case study focusing on heavy rainfall disturbances on a road network in California, the United States, abundant experiments and visualizations are conducted to demonstrate the rationality of our proposed comprehensive resilience indicators as well as the precision and reliability of these predictive resilience assessment outcomes.

NeurIPS Conference 2022 Conference Paper

Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors

• Qixun Wang
• Yifei Wang
• Hong Zhu
• Yisen Wang

Deep models often fail to generalize well in test domains when the data distribution differs from that in the training domain. Among numerous approaches to address this Out-of-Distribution (OOD) generalization problem, there has been a growing surge of interest in exploiting Adversarial Training (AT) to improve OOD performance. Recent works have revealed that the robust model obtained by conducting sample-wise AT also retains transferability to biased test domains. In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance. Specifically, we find that AT can only maintain performance at smaller scales of perturbation while Universal AT (UAT) is more robust to larger-scale perturbations. This provides us with clues that adversarial perturbations with universal (low dimensional) structures can enhance the robustness against large data distribution shifts that are common in OOD scenarios. Inspired by this, we propose two AT variants with low-rank structures to train OOD-robust models. Extensive experiments on DomainBed benchmark show that our proposed approaches outperform Empirical Risk Minimization (ERM) and sample-wise AT. Our code is available at https: //github. com/NOVAglow646/NIPS22-MAT-and-LDAT-for-OOD.

AAAI Conference 2022 Conference Paper

Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms

• Runpeng Yu
• Hong Zhu
• Kaican Li
• Lanqing Hong
• Rui Zhang
• Nanyang Ye
• Shao-Lun Huang
• Xiuqiang He

Due to the poor generalization performance of traditional empirical risk minimization (ERM) in the case of distributional shift, Out-of-Distribution (OoD) generalization algorithms receive increasing attention. However, OoD generalization algorithms overlook the great variance in the quality of training data, which significantly compromises the accuracy of these methods. In this paper, we theoretically reveal the relationship between training data quality and algorithm performance and analyze the optimal regularization scheme for Lipschitz regularized invariant risk minimization. A novel algorithm is proposed based on the theoretical results to alleviate the influence of low-quality data at both the sample level and the domain level. The experiments on both the regression and classification benchmarks validate the effectiveness of our method with statistical significance.

AAAI Conference 2021 Conference Paper

Noninvasive Self-attention for Side Information Fusion in Sequential Recommendation

• Chang Liu
• Xiaoguang Li
• Guohao Cai
• Zhenhua Dong
• Hong Zhu
• Lifeng Shang

Sequential recommender systems aim to model users’ evolving interests from their historical behaviors, and hence make customized time-relevant recommendations. Compared with traditional models, deep learning approaches such as CNN and RNN have achieved remarkable advancements in recommendation tasks. Recently, the BERT framework also emerges as a promising method, benefited from its selfattention mechanism in processing sequential data. However, one limitation of the original BERT framework is that it only considers one input source of the natural language tokens. It is still an open question to leverage various types of information under the BERT framework. Nonetheless, it is intuitively appealing to utilize other side information, such as item category or tag, for more comprehensive depictions and better recommendations. In our pilot experiments, we found naive approaches, which directly fuse types of side information into the item embeddings, usually bring very little or even negative effects. Therefore, in this paper, we propose the NOninVasive self-Attention mechanism (NOVA) to leverage side information effectively under the BERT framework. NOVA makes use of side information to generate better attention distribution, rather than directly altering the item embeddings, which may cause information overwhelming. We validate the NOVA-BERT model on both public and commercial datasets, and our method can stably outperform the state-of-the-art models with negligible computational overheads.

AAAI Conference 2020 Conference Paper

Less Is Better: Unweighted Data Subsampling via Influence Function

• Zifeng Wang
• Hong Zhu
• Zhenhua Dong
• Xiuqiang He
• Shao-Lun Huang

In the time of Big Data, training complex models on largescale data sets is challenging, making it appealing to reduce data volume for saving computation resources by subsampling. Most previous works in subsampling are weighted methods designed to help the performance of subset-model approach the full-set-model, hence the weighted methods have no chance to acquire a subset-model that is better than the full-set-model. However, we question that how can we achieve better model with less data? In this work, we propose a novel Unweighted Influence Data Subsampling (UIDS) method, and prove that the subset-model acquired through our method can outperform the full-set-model. Besides, we show that overly confident on a given test set for sampling is common in Influence-based subsampling methods, which can eventually cause our subset-model’s failure in out-ofsample test. To mitigate it, we develop a probabilistic sampling scheme to control the worst-case risk over all distributions close to the empirical distribution. The experiment results demonstrate our methods superiority over existed subsampling methods in diverse tasks, such as text classification, image classification, click-through prediction, etc.

TCS Journal 2009 Journal Article

Preface: Algorithms, complexity and models of computation

• S. Barry Cooper
• Hong Zhu

TCS Journal 2006 Journal Article

Approximating the minimum weight weak vertex cover

• Yong Zhang
• Qi Ge
• Rudolf Fleischer
• Tao Jiang
• Hong Zhu

Accurate network flow measurement is important for a variety of network applications, where the “flow” over an edge in the network is intuitively the rate of data traffic. The problem of efficiently monitoring the network flow can be regarded as finding the minimum weight weak vertex cover for a given graph. In this paper, we present a ( 2 - 2 ν ( G ) ) -approximation algorithm solving for this problem, which improves previous results, where ν ( G ) is the cyclomatic number of G.

AAMAS Conference 2005 Conference Paper

Towards An Agent Oriented Programming Language with Caste and Scenario Mechanisms

• Ji Wang
• Rui Shen
• Hong Zhu

AAMAS Conference 2003 Conference Paper

A formal specification language for agent-oriented software engineering

• Hong Zhu