Author name cluster

Zhixin Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

23 papers

2 author rows

EAAI Journal 2026 Journal Article

Improving Transformer-based image captioning via deep collaborative cross-fusion

Junbo Hu
Tong Wu
Zhixin Li

Image captioning is a pivotal interdisciplinary task at the intersection of computer vision and natural language processing, aiming to generate accurate and descriptive sentences for visual content. Research indicates that enhancing the correlation between multimodal features can enhance image captioning generation’s accuracy. However, existing methods based on the encoder–decoder framework are limited to fusing multimodal features either during the encoding or decoding stage, making it difficult to effectively bridge the semantic gap. Confronted with challenges such as processing complex image features, filtering redundant information, and fusing multimodal features, this paper proposes a Deep Synergistic Cross-fusion Transformer model (DSCT), which leverages a comprehensive multi-feature fusion method during the entire process to improve generation quality. Specifically, we have designed a Sparse Cross-fusion Encoder (SCFE) that pre-fuses features and leverages sparsity to effectively eliminate visual redundant information, thereby solving the problem of processing complex image features and filtering redundant information, generating refined dual visual features. Furthermore, we introduce a Dual-feature Regulation Fusion Attention (DRFA) module, which can precisely regulate the significance and input of visual and semantic representations during the process of creating captions, facilitating deep fusion of multimodal features to address the issue of inadequate multimodal feature fusion. Experimental results on the MS COCO dataset demonstrate that the DSCT model achieves performance competitive with state-of-the-art methods, attaining a CIDEr score of 141. 1% on the Karpathy test split, which validates the effectiveness of the innovative approach proposed in this study.

EAAI Journal 2026 Journal Article

Spatial-aware feature enhancement network for fine-grained visual classification

Wenlan Kuang
Zhixin Li

Vision transformers have recently been widely used in fine-grained visual classification. Most of the current studies utilizing vision transformers to mine key region features not only ignore the spatial connection between patches, but also additionally consider the background noise. Meanwhile, these methods learn the visual concepts of images individually, without considering the cue interactions between image pairs. To address these issues, this paper proposes a new spatial-aware feature enhancement network for fine-grained visual classification, which enables us to adaptively explore spatial contextual information in fine-grained salient regions, ignore interference from irrelevant background features, and discriminate subtle differences in similar targets by means of image pair interaction learning. Specifically, the proposed method includes two main modules: fine-grained spatial relationship and pairwise feature ensemble learning. The fine-grained spatial relationship module selects better discriminative regions by learning the mutual attention weights of different embedding layers, and subsequently adds location-dependent information to adaptively learn the neighborhoods of different regions through graph propagation. The pairwise feature ensemble learning module utilizes the mutual attention weights of the integrated image pairs to reduce confusion between fine-grained image pairs by guiding the feature interactions of the pairs from the perspective of each image individually through a gated residual mechanism. Finally, the complementary information from different transformer layers is added to the cross-layer feature boosting strategy for predicting classification results. We verify the effectiveness of this method on five widely used datasets and achieve excellent classification results.

EAAI Journal 2026 Journal Article

Towards explainable visual question answering via cross-modal causal reasoning

Wei Li
Fuyun Deng
Zhixin Li

Explainable Visual Question Answering (EVQA) aims to not only predict accurate answers to visual questions but also generate human-friendly multimodal explanations that reveal the underlying reasoning process. Despite significant progress, existing EVQA methods suffer from two critical limitations: (1) they often rely on spurious cross-modal correlations (e. g. , linguistic biases or visual shortcuts) rather than genuine causal relations, leading to unreliable reasoning; (2) the consistency between predicted answers and generated explanations is compromised due to the lack of explicit modeling of their causal dependencies. To address these issues, we propose a Cross-Modal Causal Reasoning (CMCR) framework that integrates causal inference with multimodal learning to disentangle causal effects from spurious correlations and enforce answer-explanation consistency. Specifically, CMCR incorporates three key innovations: (1) Causal Intervention, which employs backdoor adjustment to eliminate linguistic biases and frontdoor adjustment to mitigate visual shortcut biases; (2) a Neural-Symbolic Explanation Generator designed to translate symbolic reasoning processes into natural language explanations, thereby enhancing process explainability; and (3) Variational Causal Inference, which enforces causal consistency between answers and explanations. Experiments on benchmark datasets demonstrate that CMCR outperforms state-of-the-art methods, achieving a 1. 19% higher accuracy, a 1. 05% higher grounding for explanation quality, and a 0. 42% higher answer-explanation consistency.

EAAI Journal 2025 Journal Article

A cross-modal collaborative guiding network for sarcasm explanation in multi-modal multi-party dialogues

Xingjie Zhuang
Zhixin Li
Canlong Zhang
Huifang Ma

Indirect forms of language, such as sarcasm, are highly prevalent in contemporary human daily communication. While the indirect nature of metaphorical language ensures that speakers can achieve certain pragmatic goals, enabling artificial intelligence to comprehend this way of human communication poses a challenging task. Despite sarcasm detection being an extensively researched area in discourse learning, for dialogue systems to truly understand the underlying meaning of a conversation and generate reasonable responses, mere detection of sarcasm is insufficient. It is crucial to interpret the potential sarcastic meanings to acquire its true essence. Therefore, we propose a cross-modal collaborative guiding network for this study. It takes multimodal (sarcastic) dialogues as input, conducts broadcast mapping on context information from different modalities, enabling cross-modal mutual guidance and understanding. Subsequently, by employing a multi-scale information fusion mechanism to model global relationships within each modality and integrate cross-modal information to complement the overall sarcastic understanding results. Finally, the results are integrated into the bidirectional and auto-regressive transformers framework to generate natural language interpretations of sarcastic dialogues. Notably, through extensive experiments conducted on public datasets, our model achieves top performance across nearly all metrics. Compared to current state-of-the-art multimodal models, our approach demonstrates an average improvement of approximately 2. 54% across various evaluation metrics, further validating the advantages of our model over leading methods.

EAAI Journal 2025 Journal Article

Clean-sample guided cross-modal retrieval with adaptive weighted contrastive learning

Shuni Jiang
Zhixin Li

Cross-modal retrieval enables efficient integration of information by linking different data modalities, such as images and text. As data volumes increase rapidly, the need for effective cross-modal interaction grows. Cross-modal hashing is favored for its low storage requirements and fast retrieval speed, but many existing methods depend on accurately labeled data, which can be subjective and expensive to obtain. To address this limitation, we propose Clean-guided Adaptive Weighted Contrastive Hashing (CAWCH), a novel framework designed to improve robustness against noisy labels. CAWCH incorporates two main components: a Gaussian Mixture Model (GMM)-based noise purifier that identifies reliable and noisy samples by modeling sample loss, and a contrastive learning strategy that selectively chooses positive samples and adaptively assigns weights based on multi-label similarity, considering both intra- and inter-modal relationships. Extensive experiments demonstrate that CAWCH significantly outperforms existing methods under noisy label conditions, highlighting its effectiveness and potential for real-world cross-modal retrieval applications.

ICLR Conference 2025 Conference Paper

ConMix: Contrastive Mixup at Representation Level for Long-tailed Deep Clustering

Zhixin Li
Yuheng Jia

Deep clustering has made remarkable progress in recent years. However, most existing deep clustering methods assume that distributions of different clusters are balanced or roughly balanced, which are not consistent with the common long-tailed distributions in reality. In nature, the datasets often follow long-tailed distributions, leading to biased models being trained with significant performance drop. Despite the widespread proposal of many long-tailed learning approaches with supervision information, research on long-tailed deep clustering remains almost uncharted. Unaware of the data distribution and sample labels, long-tailed deep clustering is highly challenging. To tackle this problem, we propose a novel contrastive mixup method for long-tailed deep clustering, named ConMix. The proposed method makes innovations to mixup representations in contrastive learning to enhance deep clustering in long-tailed scenarios. Neural networks trained with ConMix can learn more discriminative representations, thus achieve better long-tailed deep clustering performance. We theoretically prove that ConMix works through re-balancing loss for classes with different long-tailed degree. We evaluate our method on widely used benchmark datasets with different imbalance ratios, suggesting it outperforms many state-of-the-art deep clustering approaches. The code is available at https://github.com/LZX-001/ConMix.

EAAI Journal 2025 Journal Article

Dynamic window sampling strategy for image captioning

Zhixin Li
Jiahui Wei
Tiantao Xian
Canlong Zhang
Huifang Ma

The task of image captioning aims to transform the visual information from images into semantically accurate and grammatically correct textual descriptions. In this paper, we concentrate on improving the word sampling process to enhance the training effectiveness of image captioning models. We first demonstrate that using beam search to sample sentences during reinforcement learning training performs worse than the probability-based sampling method. We then find that the probability-based sampling is prone to selecting inaccurate words due to an unstable probability distribution. Since current mainstream reinforcement learning training relies on sentence-level feedback, it is difficult to perceive word-level effects. Consequently, irrelevant words can interfere with learning the entire sentence. To address this issue, we propose a dynamic window sampling strategy for image captioning. The core idea is to dynamically determine the word candidate pool, i. e. , the sampling window, based on overall prediction confidence. Compared to sampling words over the entire vocabulary, our approach maintains diversity while avoiding the sampling of irrelevant words as much as possible. Extensive experiments on benchmark datasets show that the proposed dynamic window sampling strategy can significantly improve model performance. Specifically, our method achieved CIDEr scores of 144. 9% (single model) and 147. 9% (ensemble of 4 models) on the offline test, and 142. 1% (c5) and 144. 0% (c40) on the official online test server. Overall, this paper contributes to artificial intelligence research by studying sampling strategies to improve image captioning performance. The source code is available at https: //github. com/792218/DWSS.

AAAI Conference 2025 Conference Paper

JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs

Hongyi Li
Jiawei Ye
Jie Wu
Tianjie Yan
Chu Wang
Zhixin Li

Large Language Models (LLMs) aligned with human feedback have recently garnered significant attention. However, it remains vulnerable to jailbreak attacks, where adversaries manipulate prompts to induce harmful outputs. Exploring jailbreak attacks enables us to investigate the vulnerabilities of LLMs and further guides us in enhancing their security. Unfortunately, existing techniques mainly rely on handcrafted templates or generated-based optimization, posing challenges in scalability, efficiency and universality. To address these issues, we present JailPO, a novel black-box jailbreak framework to examine LLM alignment. For scalability and universality, JailPO meticulously trains attack models to automatically generate covert jailbreak prompts. Furthermore, we introduce a preference optimization-based attack method to enhance the jailbreak effectiveness, thereby improving efficiency. To analyze model vulnerabilities, we provide three flexible jailbreak patterns. Extensive experiments demonstrate that JailPO not only automates the attack process while maintaining effectiveness but also exhibits superior performance in efficiency, universality, and robustness against defenses compared to baselines. Additionally, our analysis of the three JailPO patterns reveals that attacks based on complex templates exhibit higher attack strength, whereas covert question transformations elicit riskier responses and are more likely to bypass defense mechanisms.

PDF Details DOI

EAAI Journal 2025 Journal Article

Key region Semantic information Augmented Transformer for Image Captioning

Fuyun Deng
Wei Li
Zhixin Li

Existing image captioning models often face difficulties in capturing inter-object relationships and generating description that comprehensively understands the entire image content, either relying on object detectors that overlook contextual information or depending on grid features that fail to adequately model spatial interactions. This paper proposes two solutions to these challenges. The first is the introduction of a module for mining semantic information from key regions. Based on the spatial proximity and high co-occurrence between objects, this module identifies the public region covered by these objects as a key region, mines their semantic information, and incorporates it into the modeling process, which compensates for the limitations of grid features. Second, we improve the standard Transformer decoder’s architecture by innovatively introducing an adaptive gating mechanism that dynamically adjusts the alignment between textual and visual features, enhancing the model’s overall comprehension of the image. To validate our approach, we applied these modules to the Transformer framework and proposed a novel method for image captioning, called Key region Semantic information Augmented Transformer (KSAT) for Image Captioning. Extensive experiments on benchmark datasets show that the proposed method outperforms many models. Specifically, our method achieves a score of 139. 6% on the offline test, and 138. 4% on the official online test server on the Consensus-based Image Description Evaluation (CIDEr) metric. In qualitative evaluation, our method also outperforms other methods at generating captions for complex scenes. Overall, these results confirm the validity of our method and advance the field of artificial intelligence.

ICML Conference 2025 Conference Paper

Learning from Sample Stability for Deep Clustering

Zhixin Li
Yuheng Jia
Hui Liu 0032
Junhui Hou

Deep clustering, an unsupervised technique independent of labels, necessitates tailored supervision for model training. Prior methods explore supervision like similarity and pseudo labels, yet overlook individual sample training analysis. Our study correlates sample stability during unsupervised training with clustering accuracy and network memorization on a per-sample basis. Unstable representations across epochs often lead to mispredictions, indicating difficulty in memorization and atypicality. Leveraging these findings, we introduce supervision signals for the first time based on sample stability at the representation level. Our proposed strategy serves as a versatile tool to enhance various deep clustering techniques. Experiments across benchmark datasets showcase that incorporating sample stability into training can improve the performance of deep clustering. The code is available at https: //github. com/LZX-001/LFSS.

EAAI Journal 2025 Journal Article

Multi-scale Feature Refinement via Perspective Scaling and Adaptive Regularization for text-based person search

Sheng Xie
Canlong Zhang
Runcong Ma
Zhixin Li
Zhiwen Wang
Chunrong Wei

Text-based person search, a crucial cross-modal task, aims to retrieve matching person images from a gallery given a textual query. A key challenge in this domain lies in effectively extracting and aligning multi-granular features across image and text modalities. In this paper, we propose the Multi-scale Feature Refinement Network (MSFR) to address the cross-modal feature matching challenge in text-based person search. Our specific contributions to advancing text-based person search within artificial intelligence are as follows: First, Perspective Scaling transforms person feature maps into multi-scale global feature maps, enhancing global feature diversity through multi-scale fusion and feature space expansion. Second, Multi-scale Textual Attention captures multi-scale textual features using various convolution kernel sizes, enhancing the model’s comprehension of multi-level information in text. Third, Adaptive Regularization dynamically adjusts the weight coefficients of local feature losses for image-text pairs during model training, assigning higher importance to critical local features. The application of our MSFR in engineering is demonstrated through extensive experiments on three text-based person search datasets has consistently demonstrated superior performance.

EAAI Journal 2025 Journal Article

Recursively learning fine-grained spatial–temporal features for video-based person Re-identification

Haifei Ma
Canlong Zhang
Zhixin Li
Zhiwen Wang

Video-based person Re-identification (Re-ID) is a field within artificial intelligence that aims to match the same person under different cameras. Its core challenge lies in how to effectively reinforce the spatial–temporal feature learning. Existing methods usually perform careful spatial feature enhancement and simple temporal feature aggregation, or the opposite, thus these methods may easily miss fine-grained temporal or spatial clues due to occlusion. To address the above issues, we propose a recursive spatial–temporal feature learning framework to enhance spatial features and recursively integrate them along the timeline. Specifically, the Trigeminal Attention Fusion (TAF) model is proposed to perform spatial complementary learning through self-relation and cross-relation attention. The TAF module consists of three branches: the Convolutional Neural Network (CNN) Branch, the Keypoint Branch, and the Global Branch. The CNN Branch uses self-relation attention to extract enhanced local features. The Keypoint Branch employs cross-relation attention to capture pedestrian keypoint-based features and handle occlusion under global guidance. After that, the Temporal Attention Alignment (TAA) model is designed to recursively propagate temporal information between adjacent frames. Furthermore, we design a bottom-up and top-down training strategy to improve the feature learning ability of the model, which can mine high-quality video-level features from frame-level features via bottom-up inference and refine frame-level features under the top-down guidance of video-level semantic feedback. Extensive experiments on four public Re-ID benchmarks demonstrate that our framework outperforms several state-of-the-art methods and performs well even in occlusion cases.

EAAI Journal 2024 Journal Article

A feature-based restoration dynamic interaction network for multimodal sentiment analysis

Yufei Zeng
Zhixin Li
Zhenbin Chen
Huifang Ma

Multimodal sentiment analysis aims to infer the sentiment of video bloggers from the features of multiple input modalities. However, there are problems such as signal noise and signal loss in the input phase and inefficient utilization of features in the modality fusion phase. To address these issues, this study proposes a feature-based restoration dynamic interaction network for multimodal sentiment analysis. Firstly, the idea of resampler and integration is employed to enhance visual and textual features during the input phase. Secondly, in the modal interaction phase, a dynamic routing network is employed. The network is centered on text modality and dynamically fuses visual and audio features. Finally, in the classification phase, multimodal representations are united to provide guidance for multimodal sentiment analysis. This study conducted experiments on the datasets MOSI, MOSEI and UR-FUNNY, which have 2199, 22856 and 16514 video segments respectively. The results show that the proposed method achieves an average improvement of about 1 point for three metrics on MOSI and 0. 5 points for individual metrics on MOSEI compared to the state-of-the-art methods. Compared to other methods, the proposed approach achieve about 1 point improvement for individual metrics on UR-FUNNY dataset.

EAAI Journal 2024 Journal Article

Crucial rather than random: Attacking crucial substructure for backdoor attacks on graph neural networks

Haibin Tong
Huifang Ma
Hui Shen
Zhixin Li
Liang Chang

Backdoor attacks on Graph Neural Networks (GNNs) seek to manipulate the behavior of GNNs model by introducing a particular pattern or trigger into the input graph, misleading the GNNs model into making inaccurate predictions. Existing methods for implementing backdoor attacks exhibit two notable limitations: Firstly, predefined substructures lack flexibility and effectiveness in compelling the classifier to associate them with the predicted label. Secondly, random injection locations for these substructures lack stealth and fail to exploit vulnerabilities in the target system. To address the aforementioned limitations, we present a novel approach targeting crucial substructures for backdoor attacks with two core modules. The crucial substructure detection module focuses on identifying predictive-relevant substructures in the input graph, which not only explains the model’s predictions but also suggests which aspects to target for accurate categorization. The graph alignment module transforms the crucial substructures of the non-target class graphs into the crucial substructures for the attacker-chosen target class graphs, modifying few key edges and nodes. We validate the effectiveness of our method on four benchmark datasets, including those from bioinformatics and social networks. The experimental results demonstrate that our approach outperforms the majority of existing baseline methods, achieving an average ASR of 90. 77%. To the best of our knowledge, our work is the first to perform backdoor attacks with crucial substructure. Through the proposed methodology, we establish a pioneering direction for refining backdoor attack techniques on GNNs.

EAAI Journal 2024 Journal Article

Unsupervised cross-modal hashing retrieval via Dynamic Contrast and Optimization

Xiumin Xie
Zhixin Li
Bo Li
Canlong Zhang
Huifang Ma

Cross-modal hashing encodes multimodal data into a common binary space, which can efficiently measure correlations between cross-modal instances. However, most existing cross-modal hashing retrieval methods are difficult to handle the heterogeneity problem between different modalities, and the performance drops because the binary code cannot be learned in the process of hash binary optimization. To solve these problems, we propose a Dynamic Contrast and Optimization (DCO) method for unsupervised cross-modal hashing retrieval, which implements an adaptive hash optimizer to strengthen the consistency of each modal representation and maintain the correlations between different modalities. Specifically, we propose a novel adaptive memory optimization mechanism. It enables the memory unit to learn and optimize adaptively, memorize in dynamic learning, and learn from memory, thereby narrowing the gap between original features and binary representations. Furthermore, we combine cross-modal ranking learning and adversarial learning. This not only ensures the modal invariance of correlated binary codes, but also allows for better approximation of generating continuous values close to discrete binary codes. To verify the effectiveness of the proposed method, we conduct a series of experiments on three widely used benchmark datasets. Through experimental results, we demonstrate the superiority of the proposed method in comparison with some state-of-the-art methods.

EAAI Journal 2023 Journal Article

Causal-ViT: Robust Vision Transformer by causal intervention

Wei Li
Zhixin Li
Xiwei Yang
Huifang Ma

Artificial intelligence based on deep learning is better at improving the representation ability of models from data. However, due to the limitation of fixed receptive field, these agents are not able to provide a correct response outside the fixed receptive field. To address this problem, this paper provides a new perspective with improving the Image Recognition tasks. This study firstly constructs two extended receptive fields using structural causal model. Then, an approximate intervention method that changes the traditional likelihood prediction to predict the result of causal intervention is proposed. Finally, this study formulates the objective function to adapt the proxy training, which makes the whole model work well. Above all of these, a new Vision Transformer variant named Causal-ViT is proposed. Furthermore, rich experimental results of different tasks are reported. These results show that the proposed perspective makes a significant improvement in Image Recognition tasks. By simply plugging Causal-ViT to different sub-tasks, all of them bring the new benchmarks of themselves field, which proves our method is flexible.

EAAI Journal 2023 Journal Article

KGCL: A Knowledge-enhanced Graph Contrastive learning framework for session-based recommendation

Xiaohui Zhang
Huifang Ma
Fanyi Yang
Zhixin Li
Liang Chang

Session-based Recommendation (SBR) aims at predicting the next item based on a short-term anonymous user behavior, whose main challenge lies at the sparsity problem of user–item interactions. Graph contrastive learning, which discovers ground-truth samples by data augmentation, is a promising paradigm to tackle this problem. However, the following two insights are often overlooked by most of these contrastive learning-based models. First, item knowledge (i. e. , item attributes, which can be distilled from open knowledge graphs) provides side information to model the complex high-order relations among items. Second, effective embedding aggregating mechanism is capable of filtering noisy preference signals (i. e. , unrelated items) in sessions and retaining higher weight for the related items. These insights motivate us to construct an item attribute hypergraph to summarize associations among items that share common attributes and develop a Knowledge-enhanced Graph Contrastive Learning framework for session-based recommendation (KGCL). Technically, KGCL constructs two independent and complementary views (cross-session graph and item attribute hypergraph) in terms of user–item interactions sequence and item intrinsic attributes respectively, so as to explicitly capture both internal and external factors of items. Then, we encode item and session embeddings with a query-aware graph attention network and a hypergraph convolutional network over the above two views. Finally, we devise two contrastive learning loss — global–global contrastive learning and local–global contrastive learning — that maximize agreement between these two views and generate high-quality recommendation results. Extensive experiments conducted on three real-world datasets show KGCL has a higher expressive power that enables SBR to predict the next item.

EAAI Journal 2023 Journal Article

Mining graph-based dynamic relationships for object detection

Xiwei Yang
Zhixin Li
Xinfang Zhong
Canlong Zhang
Huifang Ma

Since the propagation of deep neural networks results in the loss of detailed feature information, the performance of most object detection methods is limited due to their tendency to learn regional features in visual space while neglecting relationships between objects. Therefore, this study proposes the Graph Relational Decision Network (GRDN), which mines relationships between objects in a dataset. The GRDN consists of a graph decision network, decision coefficient, and step-wise relation deduction module. The graph decision network comprises an edge decision network, and a node decision network, wherein a data-driven technique is employed to obtain implicit relationships between labels in a dataset. These relationships are expressed through an adaptive dynamic graph, which is subsequently recoded by means of the decision coefficient, which can enhance semantic information. In the step-wise relation deduction module, semantic information is employed as a guide to prevent distraction. A series of experiments were conducted on the MS COCO dataset. The proposed method achieves 52. 8% box AP on object detection, which is 2. 3% box AP higher than Cascade Mask R-CNN. The experimental results show that the addition of dynamic semantic information in this study can make up for the loss of detailed information and focus on key information, thereby improving the detection ability of small objects and occluded objects. In summary, this study extracts inter-object relationships to obtain more complete semantic information, which enriches the research of object detection.

YNICL Journal 2023 Journal Article

Sex differences in frontotemporal atrophy in CADASIL revealed by 7-Tesla MRI

Xiuqin Jia
Chen Ling
Yingying Li
Jinyuan Zhang
Zhixin Li
Xuejia Jia
Danny J.J. Wang
Zihao Zhang

Brain damage caused by small vessel disease (SVD) differs between males and females. We aimed to examine the pure sex-specific neuroanatomical mechanisms of SVD adjusted for voxel-based expected effects of age and sex on healthy brain volume. Thirty-one female and 32 male genetic SVD (cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, CADASIL) patients and 55 sex- and age-matched healthy controls (HCs) underwent 7-Tesla MRI examinations. Voxel-based W-score maps were calculated from volumes and deformations of brain tissues, controlling for the expected effects of age and sex in HCs. Significant cognitive declines in working memory and executive function were identified in male CADASIL patients compared to female patients. Greater gray matter (GM) atrophy was found in the bilateral orbitofrontal cortex (OFC), left anterior cingulate cortex (ACC), left entorhinal cortex (EC), and right temporooccipital cortex in male CADASIL patients than in females. Working memory was associated with volumes in the right OFC specific to female CADASIL patients, whereas visuospatial ability was associated with the right hOcl (primary visual area, BA 17) volume specific to males. The current findings indicate that sex affects the pathogenesis of CADASIL, ranging from differences in neuroanatomy to those in behavioral performance, which may facilitate the development of more effective sex-specific therapeutic strategies for CADASIL and SVD.

AAAI Conference 2023 Conference Paper

Target-Aware Tracking with Long-Term Context Attention

Kaijie He
Canlong Zhang
Sheng Xie
Zhixin Li
Zhiwen Wang

Most deep trackers still follow the guidance of the siamese paradigms and use a template that contains only the target without any contextual information, which makes it difficult for the tracker to cope with large appearance changes, rapid target movement, and attraction from similar objects. To alleviate the above problem, we propose a long-term context attention (LCA) module that can perform extensive information fusion on the target and its context from long-term frames, and calculate the target correlation while enhancing target features. The complete contextual information contains the location of the target as well as the state around the target. LCA uses the target state from the previous frame to exclude the interference of similar objects and complex backgrounds, thus accurately locating the target and enabling the tracker to obtain higher robustness and regression accuracy. By embedding the LCA module in Transformer, we build a powerful online tracker with a target-aware backbone, termed as TATrack. In addition, we propose a dynamic online update algorithm based on the classification confidence of historical information without additional calculation burden. Our tracker achieves state-of-the-art performance on multiple benchmarks, with 71.1% AUC, 89.3% NP, and 73.0% AO on LaSOT, TrackingNet, and GOT-10k. The code and trained models are available on https://github.com/hekaijie123/TATrack.

PDF Details DOI

EAAI Journal 2021 Journal Article

Matching images and texts with multi-head attention network for cross-media hashing retrieval

Zhixin Li
Xiumin Xie
Feng Ling
Huifang Ma
Zhiping Shi

The cross-media hashing retrieval generally encodes multimedia data into a common binary hash space, which can effectively measure the correlation between samples from different modalities. However, in the cross-media retrieval, supervised methods require a lot of manual labels, which leads to the problem of high labor in practical application. Simultaneously, most unsupervised methods do not achieve good results by preserving the correlation between or within modalities. To attack these problems and further improve retrieval performance, this paper proposes an unsupervised cross-media hashing retrieval method based on multi-head attention network, which contains rich semantic information to match images and texts better. Specifically, we make use of a multi-head attention network for generating binary hash code better. At the same time, an auxiliary similarity matrix is constructed to integrate the original neighborhood information from different modalities. Therefore, this method can capture the potential relationships between inter-modal and intra-modal correlations. Furthermore, the method is unsupervised and requires no additional semantic labels, so it has the potential to achieve large-scale cross-media retrieval. In addition, two strategies of batch normalization and replacing hash code generation functions are adopted to optimize the model, and two loss functions are designed to make the performance of our method exceed that of many supervised cross-media hashing retrieval methods. Experiments on three baseline datasets show that our method performs much better than many state-of-the-art methods. The results demonstrate the effectiveness and superiority of our method.

YNIMG Journal 2014 Journal Article

FMRI and fcMRI phenotypes map the genomic effect of chromosome 13 in Brown Norway and Dahl salt-sensitive rats

Zhixin Li
B. Douglas Ward
Melinda R. Dwinell
Julian H. Lombard
Christopher P. Pawela

Genes have been implicated as major contributors to many biological traits and susceptibility to specific diseases. However, the mechanisms of genotype action on central nervous system function have been elusive. It has been previously observed that inbred Brown Norway (BN) rats exhibit a number of quantitative complex traits markedly different from those of inbred Dahl salt-sensitive (SS) rats. These strains have become so important to cardiovascular research that a novel chromosome substitution approach was used to create SS and BN strains that have a single chromosome replaced by the homologous chromosome of the other strain. The present study was conducted in an effort to evaluate whether fMRI neuroimaging measures could be employed as a phenotype of genetic influence on neural biology in SS, BN, and consomic SSBN13 rat strains. Electrical forepaw stimulation evoked robust differential BOLD-fMRI activation along the thalamocortical pathway among the three strains across different stimulus frequencies. Moreover, using the fMRI-guided seeds in thalamus and somatosensory cortex for the analysis of fcMRI, we were able to characterize the strain-specific difference in secondary somatosensory cortex, temporal association cortex, and the CA3 region. We were also able to define the genetic influences of Chr-13 on the projection and integration of sensory information in consomic SS-13BN strain. We provided objective imaging evidence supporting the hypothesis that rat strain-specific fMRI and fcMRI combined with consomic strategy can be a useful tool in identifying the complex genetic divergence that is related to neural circuits. These findings prove the concept of neuroimaging-based phenotypes as a novel approach to visualize and fine-map the genetic effects onto brain biology at a systems level.

EAAI Journal 2013 Journal Article

Learning semantic concepts from image database with hybrid generative/discriminative approach

Zhixin Li
Zhongzhi Shi
Weizhong Zhao
Zhiqing Li
Zhenjun Tang

Semantic gap has become a bottleneck of content-based image retrieval in recent years. In order to bridge the gap and improve the retrieval performance, automatic image annotation has emerged as a crucial problem. In this paper, a hybrid approach is proposed to learn the semantic concepts of images automatically. Firstly, we present continuous probabilistic latent semantic analysis (PLSA) and derive its corresponding Expectation–Maximization (EM) algorithm. Continuous PLSA assumes that elements are sampled from a multivariate Gaussian distribution given a latent aspect, instead of a multinomial one in traditional PLSA. Furthermore, we propose a hybrid framework which employs continuous PLSA to model visual features of images in generative learning stage and uses ensembles of classifier chains to classify the multi-label data in discriminative learning stage. Therefore, the framework can learn the correlations between features as well as the correlations between words. Since the hybrid approach combines the advantages of generative and discriminative learning, it can predict semantic annotation precisely for unseen images. Finally, we conduct the experiments on three baseline datasets and the results show that our approach outperforms many state-of-the-art approaches.