Arrow Research search

Author name cluster

Jun Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

87 papers
2 author rows

Possible papers

87

EAAI Journal 2026 Journal Article

Dual feature masking stage-wise knowledge distillation for object detection

  • Zhourui Zhang
  • Jun Li
  • Zhijian Wu
  • Jifeng Shen
  • Jianhua Xu

In recent years, current mainstream feature masking distillation methods mainly function by reconstructing selectively masked regions of a student network from the feature maps of a teacher network. In these methods, attention mechanisms can help to identify spatially important regions and crucial object-aware channel clues, such that the reconstructed features are encoded with sufficient discriminative and representational power similar to teacher features. However, previous feature-masking distillation methods mainly address homogeneous knowledge distillation without fully taking into account the heterogeneous knowledge distillation scenario. In particular, the huge discrepancy between the teacher and the student frameworks within the heterogeneous distillation paradigm is detrimental to feature masking, leading to deteriorating reconstructed student features. In this study, a novel dual feature-masking heterogeneous distillation framework is proposed for object detection. More specifically, a stage-wise adaptation learning module is incorporated into the dual feature-masking framework, and thus the student model can be progressively adapted to the teacher models for bridging the gap between heterogeneous networks. Furthermore, a masking enhancement strategy is combined with stage-wise learning such that object-aware masking regions are adaptively strengthened to improve feature-masking reconstruction. In addition, semantic alignment is performed at each feature pyramid network layer between the teacher and the student networks for generating consistent feature distributions. Our experiments for the object detection task demonstrate the promise of our approach, suggesting that our method outperforms both the state-of-the-art heterogeneous and homogeneous distillation methods.

Details DOI

AAAI Conference 2026 Conference Paper

Gaussian Uncertainty-Driven Multi-Model Fitting with Graph Neural Network

  • Ligang Zhang
  • Jun Li
  • Qiming Li

Multi-model fitting is fundamental for robust geometric estimation in computer vision. However, recent deep learning methods enable parallel model detection but rely on simple architectures that inadequately model spatial relationships. Moreover, current methods typically generate hypotheses only through minimal solvers on randomly sampled points, thus failing to explore the full diversity of the solution space. To address these limitations, we propose a novel Jacobian-based Gaussian uncertainty modeling framework, which analytically propagates covariance through geometric transformations and enables efficient expansion of the hypothesis space with strong theoretical guarantees. We further introduce a Gaussian Hypothesis Generation Network (GHG-Net) to learn global parameter distributions, enabling the generation of diverse and geometrically valid hypotheses. Additionally, our network captures spatial relationships among observations by employing a dynamic graph neural network with a multi-head attention mechanism. This yields more accurate sample and inlier weights, significantly improving the quality of hypothesis generation. Extensive experiments on three representative geometric estimation tasks (i.e. vanishing point detection, fundamental matrix estimation, and homography estimation) demonstrate that our method achieves new state-of-the-art accuracy and stability, while maintaining high computational efficiency.

PDF Details DOI

AAAI Conference 2026 Conference Paper

MAPI-GNN: Multi-Activation Plane Interaction Graph Neural Network for Multimodal Medical Diagnosis

  • Ziwei Qin
  • Xuhui Song
  • Deqing Huang
  • Na Qin
  • Jun Li

Graph neural networks are increasingly applied to multimodal medical diagnosis for their inherent relational modeling capabilities. However, their efficacy is often compromised by the prevailing reliance on a single, static graph built from indiscriminate features, hindering the ability to model patient-specific pathological relationships. To this end, the proposed Multi-Activation Plane Interaction Graph Neural Network (MAPI-GNN) reconstructs this single-graph paradigm by learning a multifaceted graph profile from semantically disentangled feature subspaces. The framework first uncovers latent graph-aware patterns via a multi-dimensional discriminator; these patterns then guide the dynamic construction of a stack of activation graphs; and this multifaceted profile is finally aggregated and contextualized by a relational fusion engine for a robust diagnosis. Extensive experiments on two diverse tasks, comprising over 1300 patient samples, demonstrate that MAPI-GNN significantly outperforms state-of-the-art methods.

PDF Details DOI

AAAI Conference 2026 Conference Paper

RMLer: Synthesizing Novel Objects Across Diverse Categories via Reinforcement Mixing Learning

  • Jun Li
  • Zikun Chen
  • Haibo Chen
  • Shuo Chen
  • Jian Yang

Novel object synthesis by integrating distinct textual concepts from diverse categories remains a significant challenge in text-to-image generation. Existing methods often suffer from insufficient concept mixing, lack of rigorous evaluation, and suboptimal outputs, resulting in conceptual imbalance, superficial combinations, or mere juxtapositions. To address these limitations, we propose Reinforcement Mixing Learning (RMLer), a framework that formulates cross-category concept fusion as a reinforcement learning problem: mixed features serve as states, mixing strategies as actions, and visual outcomes as rewards. Specifically, we design an MLP policy network to predict dynamic coefficients for blending cross-category text embeddings. We further introduce visual rewards based on (1) semantic similarity and (2) compositional balance between the fused object and its constituent concepts, and optimize the policy via proximal policy optimization. At inference time, a selection strategy leverages these rewards to curate the highest-quality fused objects. Extensive experiments demonstrate that RMLer synthesizes coherent, high-fidelity objects from diverse categories and consistently outperforms existing methods. Our work provides a robust framework for generating novel visual concepts, with promising applications in film, gaming, and design.

PDF Details DOI

EAAI Journal 2026 Journal Article

Structural complementarity-aware molecular representation learning for medication recommendation

  • Shunpan Liang
  • Shuoqi Li
  • Shihao Su
  • Jun Li
  • Yanghao Xiao

To achieve accurate medication recommendations, recent studies have focused on extracting appropriate molecular structure embeddings to capture pharmacological mechanisms. Building upon two-dimensional (2D) modality representations, existing works incorporate three-dimensional (3D) structural information and employ contrastive learning to derive modality-invariant embeddings. In this paper, we identify two key limitations of existing multimodal molecular representation methods. First, different modalities of the same molecule contain shared features conveying common semantics and modality-specific features providing complementary structural information. Undifferentiated alignment strategies lead to the loss of modality-specific information. Second, naive concatenation of embeddings may cause modality collapse, where the richer 3D modality dominates and suppresses the 2D modality. To overcome these issues, we propose Structural Complementarity-Aware molecular representation learning for Medication recommendation (SCAMed), a framework designed to achieve precise alignment of shared information, effective extraction of structurally complementary modality-specific information, and balanced multimodal fusion. Specifically, during the representation extraction stage, we introduce an orthogonal decomposition module that separates the 2D and 3D molecular encodings into three disentangled components: 2D-specific, 3D-specific, and shared features. In the fusion stage, we design a reconstruction-based learning strategy that enforces the fused representation to accurately reconstruct the original 2D and 3D embeddings, effectively mitigating modality collapse. Finally, the fused molecular representations are integrated with patient Electronic Health Record (EHR) data for medication prediction. Comprehensive experiments on the Medical Information Mart for Intensive Care III (MIMIC-III) and Medical Information Mart for Intensive Care IV (MIMIC-IV) datasets demonstrate that our framework achieves substantial improvements over state-of-the-art baselines.

Details DOI

EAAI Journal 2025 Journal Article

A customized multi-class pavement distress segmentation method for routine repair monitoring

  • Hui Wang
  • Junyang Kang
  • Shuichao Zhang
  • Yanmao Xiang
  • Jun Li

This study investigates the efficacy of intelligent detection methods for monitoring the quality of urban road asphalt pavement repairs, and focuses on addressing the challenges of identifying overlapping target pixels and differentiating between fine-scale distress and large-scale block patches, tasks that heavily rely on spatial semantics. A segmentation dataset comprising 13 classes and 5633 labels was constructed. Five semantic segmentation models including Deeplab V3+, SCTNet (Single-Branch Convolutional Neural Network with Transformer Semantic Information), FastFCN (Fast Fully Convolutional Network), MobileNet V3, and SegNext were constructed and evaluated. Despite its relatively smaller size, SCTNet exhibited the highest processing speed, while MobileNetV3 exhibited the smallest size and lowest accuracy. The SegNeXt model demonstrated superior performance in both segmentation accuracy and model complexity, making it chosen as the baseline model. Three training strategies were explored: classifying pavement as background, employing multiscale input, and integrating a weighted loss function. The model that combined all three strategies (referred to as SegNeXt_IPWM) demonstrated the most promising results. Compared to the baseline model, SegNeXt_IPWM achieved significant enhancements, with a 2. 08 % increase in mIoU (mean Intersection over Union) and a 1. 42 % improvement in mean F-score. Notable improvements were observed across all categories except for marking loss, with particularly substantial gains in loose (+7. 75 % IoU), block patch (+6. 62 % IoU), and linear crack (+4. 87 % IoU) detection. Additionally, SegNeXt_IPWM demonstrated superior generalization capabilities, especially in scenarios involving background target interference, underscoring its potential for robust performance in applications.

Details DOI

EAAI Journal 2025 Journal Article

An edge-guided defect segmentation network for in-service aerospace engine blades

  • Xianming Yang
  • Kechen Song
  • Shaoning Liu
  • Fuqi Sun
  • Yiming Zheng
  • Jun Li
  • Yunhui Yan

Currently, 80 % of in-service aerospace engine blade defect detection relies on manual visual assessment. Operators use a borescope to capture images of the blade surface and make judgments based on their experience and expertise. However, this method is costly and time-consuming. With the widespread application of artificial intelligence across various fields, its strong capabilities in automated defect detection are becoming increasingly evident. To meet the demand for efficient defect detection in aero-engine blades, we have constructed a dataset based on videos collected from real inspection scenarios, ensuring alignment with actual defect types. Based on this dataset, we analyze existing defect detection methods for in-service aero-engine blades and propose an improved edge-guided and channel-enhanced network using the "Transformer" architecture. Our method leverages global edge information from "Segment Anything (SAM)" to guide learning, while the channel shuffling module boosts feature capture. Experimental results show an mean intersection over union (mIoU) of 88. 13 % and a detection speed of 30. 6 frames per second (FPS) on a single graphics processing unit (GPU), meeting real-world efficiency needs. The code will be publicly available at the link: https: //github. com/Newbiejy/EGCIENet_In-service-blade-defect-detection.

Details DOI

EAAI Journal 2025 Journal Article

BIGC-Net: A Body Inter-intra-parts Graph Convolutional Network for repetitive action counting

  • Jun Li
  • Jinying Wu
  • Qiming Li
  • Bangshu Xiong

With the continuous development of human pose estimation techniques, researchers have gradually applied them to the field of repetitive action counting, resulting in pose-level methods. However, the current researches on the pose-level are still limited. Therefore, this paper proposes a simple but efficient Body Inter-intra-parts Graph Convolutional Network (BIGC-Net). Specifically, two core modules are developed in BIGC-Net: the Global Inter-Part Feature Learning Module (GIFL-Module) and the Salient Intra-Part Feature Learning Module (SIFL-Module). Unlike previous pose-level methods, which only model human joints globally and ignore local details. Instead, we innovatively introduce the concept of body parts with Graph Convolutional Networks (GCN) to the repetitive action counting task. Based on the natural topology of the human body, we divide the joints into multiple inter-intra-parts, each of which is regarded as a subgraph to form the overall graph structure. The complete action is then achieved by the collaborative operation between different subgraphs, thus modelling the action execution process more accurately. Therefore, the GIFL-Module is designed to capture the global collaborative relationships between the subgraphs. However, since the body joints are segmented into multiple parts, this segmentation may ignore the variation of local detail information within the subgraphs. To address this issue, the SIFL-Module aims to capture the local interdependencies between joints within the subgraphs, and the ability to focus on the most salient features of the subgraphs as it moves. The collaboration of these two modules further enhances the feature representation capability. Finally, extensive experimental results on the challenging benchmark datasets (RepCount-pose, UCFRep-pose, and Countix-Fitness-pose) show that the proposed BIGC-Net achieves excellent performance.

Details DOI

NeurIPS Conference 2025 Conference Paper

Convex Potential Mirror Langevin Algorithm for Efficient Sampling of Energy-Based Models

  • Zitao Yang
  • Amin Ullah
  • Shuai Li
  • Fuxin Li
  • Jun Li

This paper introduces the Convex Potential Mirror Langevin Algorithm (CPMLA), a novel method to improve sampling efficiency for Energy-Based Models (EBMs). CPMLA uses mirror Langevin dynamics with a convex potential flow as a dynamic mirror map for EBM sampling. This dynamic mirror map enables targeted geometric exploration on the data manifold, accelerating convergence to the target distribution. Theoretical analysis proves that CPMLA achieves exponential convergence with vanishing bias under relaxed log-concave conditions, supporting its efficiency in adapting to complex data distributions. Experiments on benchmarks like CIFAR-10, SVHN, and CelebA demonstrate CPMLA's improved sampling quality and inference efficiency over existing techniques.

PDF Details

AAAI Conference 2025 Conference Paper

Depth-Centric Dehazing and Depth-Estimation from Real-World Hazy Driving Video

  • Junkai Fan
  • Kun Wang
  • Zhiqiang Yan
  • Xiang Chen
  • Shangbing Gao
  • Jun Li
  • Jian Yang

In this paper, we study the challenging problem of simultaneously removing haze and estimating depth from real monocular hazy videos. These tasks are inherently complementary: enhanced depth estimation improves dehazing via the atmospheric scattering model (ASM), while superior dehazing contributes to more accurate depth estimation through the brightness consistency constraint (BCC). To tackle these intertwined tasks, we propose a novel depth-centric learning framework that integrates the ASM model with the BCC constraint. Our key idea is that both ASM and BCC rely on a shared depth estimation network. This network simultaneously exploits adjacent dehazed frames to enhance depth estimation via BCC and uses the refined depth cues to more effectively remove haze through ASM. Additionally, we leverage a non-aligned clear video and its estimated depth to independently regularize the dehazing and depth estimation networks. This is achieved by designing two discriminator networks: D_MFIR enhances high-frequency details in dehazed videos, and D_MDR reduces the occurrence of black holes in low-texture regions. Extensive experiments demonstrate that the proposed method outperforms current state-of-the-art techniques in both video dehazing and depth estimation tasks, especially in real-world hazy scenes.

PDF Details DOI

ECAI Conference 2025 Conference Paper

Dual-Space Contrastive Learning with Abnormal Edge Suppression for Graph Anomaly Detection

  • Mark Junjie Li
  • Shiyang He
  • Jun Li
  • Jinren Li
  • Gen Zhao
  • Jiang Liu
  • Sunjie Huang

Graph Anomaly Detection identifies nodes in a graph that deviate from normal behavior and finds wide applications in finance, social networks, and cybersecurity. Recent studies focus on capturing rich contrastive information between positive and negative samples by constructing multi-view contrast patterns through data augmentation, achieving notable performance gains. Nevertheless, existing methods often suffer from abnormal information diffusion, where anomalies propagate along abnormal edges and contaminate neighboring nodes, ultimately compromising the semantic consistency between the target node and its positive subgraph. Furthermore, most existing approaches learn node representations solely in Euclidean space, limiting their ability to capture the hierarchical structure prevalent in real-world graphs. To address these challenges, we propose a novel Dual-space Contrastive Learning Framework with Abnormal Edge Suppression, named DC-AES. By incorporating hyperbolic space, our framework preserves the hierarchical structure of the graph, while the abnormal edge suppression module mitigates anomaly diffusion by filtering out anomalous edges. Extensive experiments on six real datasets demonstrate the effectiveness of our approach compared to existing SOTA methods, with a maximum improvement of 6. 63% in AUC.

Details

JBHI Journal 2025 Journal Article

EEG Detection and Prediction of Freezing of Gait in Parkinson's Disease Based on Spatiotemporal Coherent Modes

  • Jun Li
  • Yuzhu Guo

Objective: Freezing of gait (FOG) in Parkinson's disease has a complex neurological mechanism. Compared with other modalities, electroencephalogram (EEG) can reflect FOG-related brain activity of both motor and non-motor symptoms. However, EEG-based FOG prediction methods often extract time, spatial, frequency, time-frequency, or phase information separately, which fragments the coupling among these heterogeneous features and cannot completely characterize the brain dynamics when FOG occurs. Methods: In this study, dynamic spatiotemporal coherent modes of EEG were studied and used for FOG detection and prediction. A dynamic mode decomposition (DMD) method was first applied to extract the spatiotemporal coherent modes. Dynamic changes of the spatiotemporal modes, in both amplitude and phase of motor-related frequency bands, were evaluated with analytic common spatial patterns (ACSP) to extract the essential differences among normal, freezing, and transitional gaits. Results: The proposed method was verified in practical clinical data. Results showed that, in the detection task, the DMD-ACSP achieved an accuracy of 86. 4 $\pm$ 3. 6% and a sensitivity of 83. 5 $\pm$ 4. 3%. In the prediction task, 86. 5 $\pm$ 3. 2% accuracy and 86. 7 $\pm$ 7. 8% sensitivity were achieved. Conclusion: Comparative studies showed that the DMD-ACSP method significantly improves FOG detection and prediction performance. Moreover, the DMD-ACSP reveals the spatial patterns of dynamic brain functional connectivity, which best discriminate the different gaits. Significance: The spatiotemporal coherent modes may provide a useful indication for personalized intervention and transcranial magnetic stimulation neuromodulation in medical practices.

Details DOI

AAAI Conference 2025 Conference Paper

Efficient Self-Supervised Video Hashing with Selective State Spaces

  • Jinpeng Wang
  • Niu Lian
  • Jun Li
  • Yuting Wang
  • Yan Feng
  • Bin Chen
  • Yongbing Zhang
  • Shu-Tao Xia

Self-supervised video hashing (SSVH) is a practical task in video indexing and retrieval. Although Transformers are predominant in SSVH for their impressive temporal modeling capabilities, they often suffer from computational and memory inefficiencies. Drawing inspiration from Mamba, an advanced state-space model, we explore its potential in SSVH to achieve a better balance between efficacy and efficiency. We introduce S5VH, a Mamba-based video hashing model with an improved self-supervised learning paradigm. Specifically, we design bidirectional Mamba layers for both the encoder and decoder, which are effective and efficient in capturing temporal relationships thanks to the data-dependent selective scanning mechanism with linear complexity. In our learning strategy, we transform global semantics in the feature space into semantically consistent and discriminative hash centers, followed by a center alignment loss as a global learning signal. Our self-local-global (SLG) paradigm significantly improves learning efficiency, leading to faster and better convergence. Extensive experiments demonstrate S5VH's improvements over state-of-the-art methods, superior transferability, and scalable advantages in inference efficiency.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Enhancing Contrastive Learning with Variable Similarity

  • Haowen Cui
  • Shuo Chen
  • Jun Li
  • Jian Yang

Contrastive learning has achieved remarkable success in self-supervised learning by pretraining a generalizable feature representation based on the augmentation invariance. Most existing approaches assume that different augmented views of the same instance (i. e. , the positive pairs ) remain semantically invariant. However, the augmentation results with varying extent may introduce semantic discrepancies or even content distortion, and thus the conventional (pseudo) supervision from augmentation invariance may lead to misguided learning objectives. In this paper, we propose a novel method called Contrastive Learning with Variable Similarity (CLVS) to accurately characterize the intrinsic similarity relationships between different augmented views. Our method dynamically adjusts the similarity based on the augmentation extent, and it ensures that strongly augmented views are always assigned lower similarity scores than weakly augmented ones. We provide a theoretical analysis to guarantee the effectiveness of the variable similarity in improving model generalizability. Extensive experiments demonstrate the superiority of our approach, achieving gains of 2. 1\% on ImageNet-100 and 1. 4\% on ImageNet-1k compared with the state-of-the-art methods.

PDF Details

AAAI Conference 2025 Conference Paper

Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking

  • Xiantao Hu
  • Ying Tai
  • Xu Zhao
  • Chen Zhao
  • Zhenyu Zhang
  • Jun Li
  • Bineng Zhong
  • Jian Yang

Multimodal tracking has garnered widespread attention as a result of its ability to effectively address the inherent limitations of traditional RGB tracking. However, existing multimodal trackers mainly focus on the fusion and enhancement of spatial features or merely leverage the sparse temporal relationships between video frames. These approaches do not fully exploit the temporal correlations in multimodal videos, making it difficult to capture the dynamic changes and motion information of targets in complex scenarios. To alleviate this problem, we propose a unified multimodal spatial-temporal tracking approach named STTrack. In contrast to previous paradigms that solely relied on updating reference information, we introduced a temporal state generator (TSG) that continuously generates a sequence of tokens containing multimodal temporal information. These temporal information tokens are used to guide the localization of the target in the next time state, establish long-range contextual relationships between video frames, and capture the temporal trajectory of the target. Furthermore, at the spatial level, we introduced the mamba fusion and background suppression interactive (BSI) modules. These modules establish a dual-stage mechanism for coordinating information interaction and fusion between modalities. Extensive comparisons on five benchmark datasets illustrate that STTrack achieves state-of-the-art performance across various multimodal tracking scenarios.

PDF Details DOI

JBHI Journal 2025 Journal Article

Gestalt-Inspired Feature Integration Network with Entropy Uncertainty Modeling for Pathology Image Segmentation

  • Dawei Fan
  • Jiamei Wen
  • Heng Dong
  • Mingyue Han
  • Jun Li
  • Chengfei Cai
  • Yanping Chen
  • Changcai Yang

The accuracy and stability of pathology image segmentation have become critical factors in clinical applications such as cancer screening and tumor grading. However, the presence of complex local structures, uncertain regions, and subtle morphological variations in pathological images continues to pose significant challenges. Most existing feature fusion approaches rely on the simplistic aggregation of extracted features, neglecting the unique characteristics and relative importance of distinct feature representations, which ultimately limits their potential to enhance model performance. To address these issues, we propose a Gestalt-Inspired Feature Integration Network (GeNet), a novel architecture inspired by Gestalt theory that mirrors the human visual system's ability to derive holistic understanding from partial information. Embracing the principle that ‘the whole is greater than the sum of its parts, ’ GeNet introduces a mechanism to synergistically leverage multi-scale information, which assesses the similarity between features to achieve a more meaningful fusion of global context and local detail. Given the variability in target appearance within pathological images, we use information entropy to quantify feature uncertainty, allowing the model to prioritize uncertain regions and reduce the occurrence of ambiguous results. To explicitly eliminate multi-feature redundancy and misalignment, the refinement block utilizes parallel convolutional recalibration to fully leverage the advantages of various features. Extensive experiments on multiple pathological image segmentation datasets, including GlaS, GCaSeg, and EBHI-Seg, demonstrate that GeNet achieves high accuracy and strong robustness, offering a new perspective for joint modeling of global and local features in medical image analysis.

Details DOI

AAAI Conference 2025 Conference Paper

Guided Real Image Dehazing Using YCbCr Color Space

  • Wenxuan Fang
  • Junkai Fan
  • Yu Zheng
  • Jiangwei Weng
  • Ying Tai
  • Jun Li

Image dehazing, particularly with learning-based methods, has gained significant attention due to its importance in real-world applications. However, relying solely on the RGB color space often fall short, frequently leaving residual haze. This arises from two main issues: the difficulty in obtaining clear textural features from hazy RGB images and the complexity of acquiring real haze/clean image pairs outside controlled environments like smoke-filled scenes. To address these issues, we first propose a novel Structure Guided Dehazing Network (SGDN) that leverages the superior structural properties of YCbCr features over RGB. It comprises two key modules: Bi-Color Guidance Bridge (BGB) and Color Enhancement Module (CEM). BGB integrates a phase integration module and an interactive attention module, utilizing the rich texture features of the YCbCr space to guide the RGB space, thereby recovering clearer features in both frequency and spatial domains. To maintain tonal consistency, CEM further enhances the color perception of RGB features by aggregating YCbCr channel information. Furthermore, for effective supervised learning, we introduce a Real-World Well-Aligned Haze dataset, which includes a diverse range of scenes from various geographical regions and climate conditions. Experimental results demonstrate that our method surpasses existing state-of-the-art methods across multiple real-world smoke/haze datasets.

PDF Details DOI

AAAI Conference 2025 Conference Paper

Harmonious Music-driven Group Choreography with Trajectory-Controllable Diffusion

  • Yuqin Dai
  • Wanlu Zhu
  • Ronghui Li
  • Zeping Ren
  • Xiangzheng Zhou
  • Jixuan Ying
  • Jun Li
  • Jian Yang

Creating group choreography from music is crucial in cultural entertainment and virtual reality, with a focus on generating harmonious movements. Despite growing interest, recent approaches often struggle with two major challenges: multi-dancer collisions and single-dancer foot sliding. To address these challenges, we propose a Trajectory-Controllable Diffusion (TCDiff) framework, which leverages non-overlapping trajectories to ensure coherent and aesthetically pleasing dance movements. To mitigate collisions, we introduce a Dance-Trajectory Navigator that generates collision-free trajectories for multiple dancers, utilizing a distance-consistency loss to maintain optimal spacing. Furthermore, to reduce foot sliding, we present a footwork adaptor that adjusts trajectory displacement between frames, supported by a relative forward-kinematic loss to further reinforce the correlation between movements and trajectories. Experiments demonstrate our method's superiority.

PDF Details DOI

ECAI Conference 2025 Conference Paper

LaPNER: Label-Aware Prompt Learning with Non-Entity Clustering Regularization for Few-Shot NER

  • Mark Junjie Li
  • Tao Zou
  • Sunjie Huang
  • Yigang Lin
  • Qilong Gong
  • Jun Li

Despite the recent success of prompt-based methods in few-shot named entity recognition (FSNER), most approaches rely on manually created prompts (e. g. , templates or label words) that fail to capture sufficient semantic information. This limits generalization, particularly in cross-domain settings. Additionally, conventional FSNER methods typically assign tokens that are not meant to be recognized to a single non-entity category, even though this category encompasses both irrelevant tokens and other entities that are not needed to be recognized. Treating such a semantically diverse group as a single category introduces noise and confusion. In this paper, we propose an approach called Label-Aware Prompt learning with Non-Entity clustering Regularization for Few-Shot NER (LaPNER). We introduce a learnable prompt pool to enrich the semantic representations of label words. Additionally, we employ embedding clustering regularization to more effectively distinguish the heterogeneous tokens within the non-entity category. Comprehensive experiments on multiple benchmarks demonstrate that LaPNER consistently outperforms prior methods in various settings, highlighting its effectiveness in improving generalization across tasks.

Details

ICRA Conference 2025 Conference Paper

Learning Based MPC for Autonomous Driving Using a Low Dimensional Residual Model

  • Yaoyu Li
  • Chaosheng Huang
  • Dongsheng Yang
  • Wenbo Liu
  • Jun Li

In this paper, a learning based Model Predictive Control (MPC) using a low dimensional residual model is proposed for autonomous driving. One of the critical challenge in autonomous driving is the complexity of vehicle dynamics, which impedes the formulation of accurate vehicle model. Inaccurate vehicle model can significantly impact the performance of MPC controller. To address this issue, this paper decomposes the nominal vehicle model into invariable and variable elements. The accuracy of invariable elements are ensured by calibration, while the deviations in the variable elements are learned by a low-dimensional residual model. The features of residual model are selected as the physical variables most correlated with nominal model errors. Physical constraints among these features are formulated to explicitly define the valid region within the feature space. The formulated model and constraints are incorporated into the MPC framework and validated through both simulation and real vehicle experiments. The results indicate that the proposed method significantly enhances the model accuracy and controller performance.

Details

AAAI Conference 2025 Conference Paper

Learning Generalized Residual Exchange-Correlation-Uncertain Functional for Density Functional Theory

  • Sizhuo Jin
  • Shuo Chen
  • Jianjun Qian
  • Ying Tai
  • Jun Li

Density Functional Theory (DFT) stands as a widely used and efficient approach for addressing the many-electron Schrödinger equation across various domains such as physics, chemistry, and biology. However, a core challenge that persists over the long term pertains to refining the exchange-correlation (XC) approximation. This approximation significantly influences the triumphs and shortcomings observed in DFT applications. Nonetheless, a prevalent issue among XC approximations is the presence of systematic errors, stemming from deviations from the mathematical properties of the exact XC functional. For example, although both B3LYP and DM21 (DeepMind 21) exhibit improvements over previous benchmarks, there is still potential for further refinement. In this paper, we propose a strategy for enhancing XC approximations by estimating the neural uncertainty of the XC functional, named Residual XC-Uncertain Functional. Specifically, our approach involves training a neural network to predict both the mean and variance of the XC functional, treating it as a Gaussian distribution. To ensure stability in each sampling point, we construct the mean by combining traditional XC approximations with our neural predictions, mitigating the risk of divergence or vanishing values. It is crucial to highlight that our methodology excels particularly in cases where systematic errors are pronounced. Empirical outcomes from three benchmark tests substantiate the superiority of our approach over existing state-of-the-art methods. Our approach not only surpasses related techniques but also significantly outperforms both the popular B3LYP and the recent DM21 methods, achieving average RMSE improvements of 62% and 37%, respectively, across the three benchmarks: W4-17, G21EA, and G21IP.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

NOVA: A Benchmark for Rare Anomaly Localization and Clinical Reasoning in Brain MRI

  • Cosmin Bercea
  • Jun Li
  • Philipp Raffler
  • Evamaria O. Riedel
  • Lena Schmitzer
  • Angela Kurz
  • Felix Bitzer
  • Paula Roßmüller

In many real-world applications, deployed models encounter inputs that differ from the data seen during training. Open-world recognition ensures that such systems remain robust as ever-emerging, previously _unknown_ categories appear and must be addressed without retraining. Foundation and vision-language models are pre-trained on large and diverse datasets with the expectation of broad generalization across domains, including medical imaging. However, benchmarking these models on test sets with only a few common outlier types silently collapses the evaluation back to a closed-set problem, masking failures on rare or truly novel conditions encountered in clinical use. We therefore present NOVA, a challenging, real-life _evaluation-only_ benchmark of $\sim$900 brain MRI scans that span 281 rare pathologies and heterogeneous acquisition protocols. Each case includes rich clinical narratives and double-blinded expert bounding-box annotations. Together, these enable joint assessment of anomaly localisation, visual captioning, and diagnostic reasoning. Because NOVA is never used for training, it serves as an _extreme_ stress-test of out-of-distribution generalisation: models must bridge a distribution gap both in sample appearance and in semantic space. Baseline results with leading vision-language models (GPT-4o, Gemini 2. 0 Flash, and Qwen2. 5-VL-72B) reveal substantial performance drops, with approximately a 65\% gap in localisation compared to natural-image benchmarks and 40\% and 20\% gaps in captioning and reasoning, respectively, compared to resident radiologists. Therefore, NOVA establishes a testbed for advancing models that can detect, localize, and reason about truly unknown anomalies.

PDF Details

JBHI Journal 2025 Journal Article

Partial-Label Contrastive Representation Learning for Fine-Grained Biomarkers Prediction From Histopathology Whole Slide Images

  • Yushan Zheng
  • Kun Wu
  • Jun Li
  • Kunming Tang
  • Jun Shi
  • Haibo Wu
  • Zhiguo Jiang
  • Wei Wang

In the domain of histopathology analysis, existing representation learning methods for biomarkers prediction from whole slide images (WSIs) face challenges due to the complexity of tissue subtypes and label noise problems. This paper proposed a novel partial-label contrastive representation learning approach to enhance the discrimination of histopathology image representations for fine-grained biomarkers prediction. We designed a partial-label contrastive clustering (PLCC) module for partial-label disambiguation and a dynamic clustering algorithm to sample the most representative features of each category to the clustering queue during the contrastive learning process. We conducted comprehensive experiments on three gene mutation prediction datasets, including USTC-EGFR, BRCA-HER2, and TCGA-EGFR. The results show that our method outperforms 9 existing methods in terms of Accuracy, AUC, and F1 Score. Specifically, our method achieved an AUC of 0. 950 in EGFR mutation subtyping of TCGA-EGFR and an AUC of 0. 853 in HER2 0/1+/2+/3+ grading of BRCA-HER2, which demonstrates its superiority in fine-grained biomarkers prediction from histopathology whole slide images.

Details DOI

AAAI Conference 2025 Conference Paper

Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation

  • Xiaoqi An
  • Lin Zhao
  • Chen Gong
  • Jun Li
  • Jian Yang

With the rapid development of autonomous driving, LiDAR-based 3D Human Pose Estimation (3D HPE) is becoming a research focus. However, due to the noise and sparsity of LiDAR-captured point clouds, robust human pose estimation remains challenging. Most of the existing methods use temporal information, multi-modal fusion, or SMPL optimization to correct biased results. In this work, we try to obtain sufficient information for 3D HPE only by modeling the intrinsic properties of low-quality point clouds. Hence, a simple yet powerful method is proposed, which provides insights both on modeling and augmentation of point clouds. Specifically, we first propose a concise and effective density-aware pose transformer (DAPT) to get stable keypoint representations. By using a set of joint anchors and a carefully designed exchange module, valid information is extracted from point clouds with different densities. Then 1D heatmaps are utilized to represent the precise locations of the keypoints. Secondly, a comprehensive LiDAR human synthesis and augmentation method is proposed to pre-train the model, enabling it to acquire a better human body prior. We increase the diversity of point clouds by randomly sampling human positions and orientations and by simulating occlusions through the addition of laser-level masks. Extensive experiments have been conducted on multiple datasets, including IMU-annotated LidarHuman26M, SLOPER4D, and manually annotated Waymo Open Dataset v2.0 (Waymo), HumanM3. Our method demonstrates SOTA performance in all scenarios. In particular, compared with LPFormer on Waymo, we reduce the average MPJPE by 10.0mm. Compared with PRN on SLOPER4D, we notably reduce the average MPJPE by 20.7mm.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Reading Your Heart: Learning ECG Words and Sentences via Pre-training ECG Language Model

  • Jiarui Jin
  • Haoyu Wang
  • Hongyan Li 0002
  • Jun Li
  • Jiahui Pan
  • Shenda Hong

Electrocardiogram (ECG) is essential for the clinical diagnosis of arrhythmias and other heart diseases, but deep learning methods based on ECG often face limitations due to the need for high-quality annotations. Although previous ECG self-supervised learning (eSSL) methods have made significant progress in representation learning from unannotated ECG data, they typically treat ECG signals as ordinary time-series data, segmenting the signals using fixed-size and fixed-step time windows, which often ignore the form and rhythm characteristics and latent semantic relationships in ECG signals. In this work, we introduce a novel perspective on ECG signals, treating heartbeats as words and rhythms as sentences. Based on this perspective, we first designed the QRS-Tokenizer, which generates semantically meaningful ECG sentences from the raw ECG signals. Building on these, we then propose HeartLang, a novel self-supervised learning framework for ECG language processing, learning general representations at form and rhythm levels. Additionally, we construct the largest heartbeat-based ECG vocabulary to date, which will further advance the development of ECG language processing. We evaluated HeartLang across six public ECG datasets, where it demonstrated robust competitiveness against other eSSL methods. Our data and code are publicly available at https://github.com/PKUDigitalHealth/HeartLang.

Details

IROS Conference 2025 Conference Paper

The Parallel Pneumatic Artificial Muscle Platform Based on RBF Neural Network Compensation

  • Jun Li
  • Yuanquan Dai
  • Dongdong Zhang
  • Ruidong Yu
  • Mingkang Zi
  • Shuaicheng Liu
  • Yinhui Xie

A two-degree-of-freedom parallel mechanism control system based on an adaptive learning rate and radial basis function (RBF) neural network controller is studied in this paper. The mechanism is composed of four pneumatic artificial muscles(PAM), forming two pairs of antagonistic single-degree-of-freedom joints, which enable two-degree-of-freedom motion along the X and Y axes. The core objective of the system is to automatically output the air pressure values for the X and Y axes based on the input desired angle, driving the joints to precisely reach the specified angle. In this research, dynamic modeling of the two pairs of driving joints composed of four pneumatic muscles was conducted, analyzing the motion characteristics of the system. Subsequently, an RBF neural network was employed to approximate system modeling errors and external disturbances, combined with a PID controller to optimize the driving performance of the pneumatic muscles. The stability of the controller was proven by designing the Lyapunov function, ensuring that the system remains stable during dynamic changes. Finally, simulation experiments were conducted using MATLAB/Simulink to verify the effectiveness of the proposed algorithm. The experimental results demonstrate that the control algorithm enables the actual angle to track the desired angle in real-time, with high control accuracy and stability. This research provides a new solution for the precise control of pneumatic muscle-driven parallel joint systems, with broad application prospects, effectively addressing the limitations of traditional PAM control methods that require precise modeling and suffer from poor robustness.

Details

NeurIPS Conference 2025 Conference Paper

Unveiling Environmental Sensitivity of Individual Gains in Influence Maximization

  • Xinyan Su
  • Zhiheng Zhang
  • Jiyan Qiu
  • Zhaojuan Yue
  • Jun Li

Influence Maximization (IM) seeks a seed set to maximize information dissemination in a network. Elegant IM algorithms could naturally extend to cases where each node is equipped with a specific weight, reflecting individual gains to measure its importance. In prevailing literature, these gains are typically assumed to remain constant throughout diffusion and are solvable through explicit formulas based on node characteristics and network topology. However, this assumption is not always feasible due to two key challenges: 1) \textit{Unobservability}: The individual gains of each node are primarily evaluated by the difference between the outputs in the activated and non-activated states. In practice, we can only observe one of these states, with the other remaining unobservable post-propagation. 2) \textit{Environmental sensitivity}: Beyond nodes’ inherent properties, individual gains are also sensitive to the activation status of surrounding nodes, which change dynamically during propagation even when the network topology is fixed. To address these uncertainties, we introduce a Causal Influence Maximization (CauIM) framework, leveraging causal inference techniques to model dynamic individual gains. We propose two algorithms, G-CauIM and A-CauIM, where the latter incorporates a novel acceleration technique. Theoretically, we establish the generalized lower bound of influence spread and provide robustness analysis. Empirically, experiments on synthetic and real-world datasets validate the effectiveness and reliability of our approach.

PDF Details

NeurIPS Conference 2025 Conference Paper

V2X-Radar: A Multi-modal Dataset with 4D Radar for Cooperative Perception

  • Lei Yang
  • Xinyu Zhang
  • Jun Li
  • Chen Wang
  • Jiaqi Ma
  • Zhiying Song
  • Tong Zhao
  • Ziying Song

Modern autonomous vehicle perception systems often struggle with occlusions and limited perception range. Previous studies have demonstrated the effectiveness of cooperative perception in extending the perception range and overcoming occlusions, thereby enhancing the safety of autonomous driving. In recent years, a series of cooperative perception datasets have emerged; however, these datasets primarily focus on cameras and LiDAR, neglecting 4D Radar—a sensor used in single-vehicle autonomous driving to provide robust perception in adverse weather conditions. In this paper, to bridge the gap created by the absence of 4D Radar datasets in cooperative perception, we present V2X-Radar, the first large-scale, real-world multi-modal dataset featuring 4D Radar. V2X-Radar dataset is collected using a connected vehicle platform and an intelligent roadside unit equipped with 4D Radar, LiDAR, and multi-view cameras. The collected data encompasses sunny and rainy weather conditions, spanning daytime, dusk, and nighttime, as well as various typical challenging scenarios. The dataset consists of 20K LiDAR frames, 40K camera images, and 20K 4D Radar data, including 350K annotated boxes across five categories. To support various research domains, we have established V2X-Radar-C for cooperative perception, V2X-Radar-I for roadside perception, and V2X-Radar-V for single-vehicle perception. Furthermore, we provide comprehensive benchmarks across these three sub-datasets.

PDF Details

EAAI Journal 2024 Journal Article

A sentiment analysis method for COVID-19 network comments integrated with semantic concept

  • Jun Li
  • Lanlan Jiang
  • Guimin Huang
  • Jingwei Zhang

In recent years, the new coronavirus COVID-19 has brought great disaster and loss to the world and is still spreading around the world. The situation in China is generally well controlled, and the lockdown has been removed, but the comments and messages about the epidemic persist online. For people working and living normally in China, their attitudes and views toward COVID-19 directly reflect the current situation of the pandemic. This paper collected Chinese microblogs, forums, and online comments, identified the latest comments about COVID-19, and conducted a sentiment analysis of them. Specifically, we proposed a new sentiment analysis method that integrated the semantics of words with the text analyzed. Different from the traditional sentiment analysis method which only relied on sentiment words, the proposed method extended the semantic concepts of affective words by integrating the semantic conceptual information about the affective words from the context of the comments and thus, provided information to support the final judgment of the affective opinions. The proposed approach incorporated the part-of-speech embedding information along with word embedding and relies on semantic concepts to enhance the emotional expression of words in context. The experimental results showed that by integrating the semantics of words, the accuracy of sentiment analysis is substantially improved, and it also reflected that different semantics of the same word have different influences on sentiment analysis. On several benchmark datasets, there was a 3–6% improvement in accuracy.

Details DOI

AAAI Conference 2024 Conference Paper

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization

  • Kun Wang
  • Zhiqiang Yan
  • Huang Tian
  • Zhenyu Zhang
  • Xiang Li
  • Jun Li
  • Jian Yang

Neural Radiance Fields (NeRF) have shown promise in generating realistic novel views from sparse scene images. However, existing NeRF approaches often encounter challenges due to the lack of explicit 3D supervision and imprecise camera poses, resulting in suboptimal outcomes. To tackle these issues, we propose AltNeRF---a novel framework designed to create resilient NeRF representations using self-supervised monocular depth estimation (SMDE) from monocular videos, without relying on known camera poses. SMDE in AltNeRF masterfully learns depth and pose priors to regulate NeRF training. The depth prior enriches NeRF's capacity for precise scene geometry depiction, while the pose prior provides a robust starting point for subsequent pose refinement. Moreover, we introduce an alternating algorithm that harmoniously melds NeRF outputs into SMDE through a consistence-driven mechanism, thus enhancing the integrity of depth priors. This alternation empowers AltNeRF to progressively refine NeRF representations, yielding the synthesis of realistic novel views. Extensive experiments showcase the compelling capabilities of AltNeRF in generating high-fidelity and robust novel views that closely resemble reality.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Bi 2 Lane: Bi-Directional Temporal Refinement with Bi-Level Feature Aggregation for 3D Lane Detection

  • Chengxin Li
  • Yihui Hu
  • Zewen Zheng
  • Xiang Gao
  • Yongqiang Mou
  • Peng Nie
  • Jun Li

Monocular 3D lane detection has recently received increasing research attention in autonomous driving due to its application effectiveness and simplicity. However, depending solely on the limited semantic information from a single image makes current monocular detection methods unable to deal with complex scenarios, such as occluded, blurred, and unaligned scenes. In this study, we introduce an end-to-end framework named Bi 2 Lane which models temporal dependency in a continuous sequence. It recurrently utilizes detected lanes within historical frames as prior information to achieve robust lane detection. Additionally, Bi 2 Lane employs temporal reverse refinement together with temporal forward refinement to achieve bi-directional temporal refinement (BDTR) while maintaining a robust temporal dependency. For the refined features of different frames, we design a bi-level feature aggregation module (BLFA) to fuse them in both point-level and line-level manners, enabling a comprehensive feature representation to deal with complicated road scenes. Extensive experiments conducted on the OpenLane dataset demonstrate the superiority of Bi 2 Lane, achieving a notable F1 score of 63. 8% using a simple ResNet50 backbone, surpassing the performance of existing state-of-the-art methods.

Details

AAAI Conference 2024 Conference Paper

Compound Text-Guided Prompt Tuning via Image-Adaptive Cues

  • Hao Tan
  • Jun Li
  • Yizhuang Zhou
  • Jun Wan
  • Zhen Lei
  • Xiangyu Zhang

Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable generalization capabilities to downstream tasks. However, existing prompt tuning based frameworks need to parallelize learnable textual inputs for all categories, suffering from massive GPU memory consumption when there is a large number of categories in the target dataset. Moreover, previous works require to include category names within prompts, exhibiting subpar performance when dealing with ambiguous category names. To address these shortcomings, we propose Compound Text-Guided Prompt Tuning (TGP-T) that significantly reduces resource demand while achieving superior performance. We introduce text supervision to the optimization of prompts, which enables two benefits: 1) releasing the model reliance on the pre-defined category names during inference, thereby enabling more flexible prompt generation; 2) reducing the number of inputs to the text encoder, which decreases GPU memory consumption significantly. Specifically, we found that compound text supervisions, i.e., category-wise and content-wise, is highly effective, since they provide inter-class separability and capture intra-class variations, respectively. Moreover, we condition the prompt generation on visual features through a module called Bonder, which facilitates the alignment between prompts and visual features. Extensive experiments on few-shot recognition and domain generalization demonstrate that TGP-T achieves superior performance with consistently lower training costs. It reduces GPU memory usage by 93% and attains a 2.5% performance gain on 16-shot ImageNet. The code is available at https://github.com/EricTan7/TGP-T.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

DCDepth: Progressive Monocular Depth Estimation in Discrete Cosine Domain

  • Kun Wang
  • Zhiqiang Yan
  • Junkai Fan
  • Wanlu Zhu
  • Xiang Li
  • Jun Li
  • Jian Yang

In this paper, we introduce DCDepth, a novel framework for the long-standing monocular depth estimation task. Moving beyond conventional pixel-wise depth estimation in the spatial domain, our approach estimates the frequency coefficients of depth patches after transforming them into the discrete cosine domain. This unique formulation allows for the modeling of local depth correlations within each patch. Crucially, the frequency transformation segregates the depth information into various frequency components, with low-frequency components encapsulating the core scene structure and high-frequency components detailing the finer aspects. This decomposition forms the basis of our progressive strategy, which begins with the prediction of low-frequency components to establish a global scene context, followed by successive refinement of local details through the prediction of higher-frequency components. We conduct comprehensive experiments on NYU-Depth-V2, TOFDC, and KITTI datasets, and demonstrate the state-of-the-art performance of DCDepth. Code is available at https: //github. com/w2kun/DCDepth.

PDF Details DOI

EAAI Journal 2024 Journal Article

EAFNet: Extraction-amplification-fusion network for tiny cracks detection

  • Ziang Zhou
  • Wensong Zhao
  • Kechen Song
  • Yanyan Wang
  • Jun Li

Tiny cracks are often overlooked in the inspection process, causing huge economic losses and dangerous accidents. Therefore, tiny cracks should be detected in a timely and accurate manner to eliminate the disease at the initial stage. Inspired by the fact that humans are more likely to capture conspicuous information when observing objects, we propose a novel three-stage Extraction-Amplification-Fusion network (EAFNet). Specifically, in the extraction stage, we utilize an effective backbone network for feature extraction of tiny cracks. In the amplification stage, we design a Tiny Feature Amplification (TFA) module to amplify the extracted features. In the fusion stage, we propose a Two-Branch Fusion (TBF) module to fully fuse the feature maps at different resolutions. To make tiny crack information more ‘conspicuous’, we propose an activation function TinyReLU to enhance the contrast of the tiny cracks with the background. In addition, we construct a Tiny Crack (T-CRACK) dataset with six different backgrounds and a Cross-scale Crack (C-CRACK) dataset. On both datasets, EAFNet achieves an advantage over the existing 8 advanced networks. The two datasets are available at: https: //github. com/EAFNet/EAFNet.

Details DOI

NeurIPS Conference 2024 Conference Paper

MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space

  • Jiangwei Weng
  • Zhiqiang Yan
  • Ying Tai
  • Jianjun Qian
  • Jian Yang
  • Jun Li

Recent advances in low light image enhancement have been dominated by Retinex-based learning framework, leveraging convolutional neural networks (CNNs) and Transformers. However, the vanilla Retinex theory primarily addresses global illumination degradation and neglects local issues such as noise and blur in dark conditions. Moreover, CNNs and Transformers struggle to capture global degradation due to their limited receptive fields. While state space models (SSMs) have shown promise in the long-sequence modeling, they face challenges in combining local invariants and global context in visual data. In this paper, we introduce MambaLLIE, an implicit Retinex-aware low light enhancer featuring a global-then-local state space design. We first propose a Local-Enhanced State Space Module (LESSM) that incorporates an augmented local bias within a 2D selective scan mechanism, enhancing the original SSMs by preserving local 2D dependency. Additionally, an Implicit Retinex-aware Selective Kernel module (IRSK) dynamically selects features using spatially-varying operations, adapting to varying inputs through an adaptive kernel selection process. Our Global-then-Local State Space Block (GLSSB) integrates LESSM and IRSK with layer normalization (LN) as its core. This design enables MambaLLIE to achieve comprehensive global long-range modeling and flexible local feature aggregation. Extensive experiments demonstrate that MambaLLIE significantly outperforms state-of-the-art CNN and Transformer-based methods. Our code is available at https: //github. com/wengjiangwei/MambaLLIE.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

Novel Object Synthesis via Adaptive Text-Image Harmony

  • Zeren Xiong
  • Zedong Zhang
  • Zikun Chen
  • Shuo Chen
  • Xiang Li
  • Gan Sun
  • Jian Yang
  • Jun Li

In this paper, we study an object synthesis task that combines an object text with an object image to create a new object image. However, most diffusion models struggle with this task, \textit{i. e. }, often generating an object that predominantly reflects either the text or the image due to an imbalance between their inputs. To address this issue, we propose a simple yet effective method called Adaptive Text-Image Harmony (ATIH) to generate novel and surprising objects. First, we introduce a scale factor and an injection step to balance text and image features in cross-attention and to preserve image information in self-attention during the text-image inversion diffusion process, respectively. Second, to better integrate object text and image, we design a balanced loss function with a noise parameter, ensuring both optimal editability and fidelity of the object image. Third, to adaptively adjust these parameters, we present a novel similarity score function that not only maximizes the similarities between the generated object image and the input text/image but also balances these similarities to harmonize text and image integration. Extensive experiments demonstrate the effectiveness of our approach, showcasing remarkable object creations such as colobus-glass jar. https: //xzr52. github. io/ATIH/

PDF Details DOI

AAAI Conference 2024 Conference Paper

Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval

  • Zhihang Liu
  • Jun Li
  • Hongtao Xie
  • Pandeng Li
  • Jiannan Ge
  • Sun-Ao Liu
  • Guoqing Jin

Video Moment Retrieval (VMR) aims to retrieve temporal segments in untrimmed videos corresponding to a given language query by constructing cross-modal alignment strategies. However, these existing strategies are often sub-optimal since they ignore the modality imbalance problem, i.e., the semantic richness inherent in videos far exceeds that of a given limited-length sentence. Therefore, in pursuit of better alignment, a natural idea is enhancing the video modality to filter out query-irrelevant semantics, and enhancing the text modality to capture more segment-relevant knowledge. In this paper, we introduce Modal-Enhanced Semantic Modeling (MESM), a novel framework for more balanced alignment through enhancing features at two levels. First, we enhance the video modality at the frame-word level through word reconstruction. This strategy emphasizes the portions associated with query words in frame-level features while suppressing irrelevant parts. Therefore, the enhanced video contains less redundant semantics and is more balanced with the textual modality. Second, we enhance the textual modality at the segment-sentence level by learning complementary knowledge from context sentences and ground-truth segments. With the knowledge added to the query, the textual modality thus maintains more meaningful semantics and is more balanced with the video modality. By implementing two levels of MESM, the semantic information from both modalities is more balanced to align, thereby bridging the modality gap. Experiments on three widely used benchmarks, including the out-of-distribution settings, show that the proposed framework achieves a new start-of-the-art performance with notable generalization ability (e.g., 4.42% and 7.69% average gains of R1@0.7 on Charades-STA and Charades-CG). The code will be available at https://github.com/lntzm/MESM.

PDF Details DOI

EAAI Journal 2023 Journal Article

A novel noise-robustness and rotation-invariant LADAR point cloud target classification method

  • Shangwei Guo
  • Jun Li
  • Zhengchao Lai
  • Shaokun Han

Identifying targets of LAser Detection And Ranging (LADAR) point clouds that are affected by noise and different poses throws a significant challenge. This paper proposes a novel LADAR point cloud target classification method that is both noise-robust and rotation-invariant. Specifically, the proposed method transforms the point cloud target into a slice image that is resilient to noise and holds rotation-invariance. Next, a designed 2D Convolutional Neural Network (CNN) is utilized to classify the corresponding point cloud target based on the slice image. To overcome the challenge of lacking prior knowledge about the discriminative features of the slice image, the 2D CNN is designed with the Local Importance-based Pooling (LIP) layer. This layer extracts the discriminative feature in a data-driven manner, thereby improving the accuracy of the classification process. The proposed method is evaluated through multiple experiments on the public ModelNet40 dataset. The experimental results demonstrate that the designed LIP-CNN can better learn the discriminative features of the slice image, achieving high classification accuracy. Moreover, the proposed slice-image-based method is capable of accurately classifying the target, even in the presence of noise and different poses.

Details DOI

YNIMG Journal 2023 Journal Article

APART-QSM: An improved sub-voxel quantitative susceptibility mapping for susceptibility source separation using an iterative data fitting method

  • Zhenghao Li
  • Ruimin Feng
  • Qiangqiang Liu
  • Jie Feng
  • Guoyan Lao
  • Ming Zhang
  • Jun Li
  • Yuyao Zhang

The brain tissue phase contrast in MRI sequences reflects the spatial distributions of multiple substances, such as iron, myelin, calcium, and proteins. These substances with paramagnetic and diamagnetic susceptibilities often colocalize in one voxel in brain regions. Both opposing susceptibilities play vital roles in brain development and neurodegenerative diseases. Conventional QSM methods only provide voxel-averaged susceptibility value and cannot disentangle intravoxel susceptibilities with opposite signs. Advanced susceptibility imaging methods have been recently developed to distinguish the contributions of opposing susceptibility sources for QSM. The basic concept of separating paramagnetic and diamagnetic susceptibility proportions is to include the relaxation rate R 2 * with R 2 ′ in QSM. The magnitude decay kernel, describing the proportionality coefficient between R 2 ′ and susceptibility, is an essential reconstruction coefficient for QSM separation methods. In this study, we proposed a more comprehensive complex signal model that describes the relationship between 3D GRE signal and the contributions of paramagnetic and diamagnetic susceptibility to the frequency shift and R 2 * relaxation. The algorithm is implemented as a constrained minimization problem in which the voxel-wise magnitude decay kernel and sub-voxel susceptibilities are determined alternately in each iteration until convergence. The calculated voxel-wise magnitude decay kernel could realistically model the relationship between the R 2 ′ relaxation and the volume susceptibility. Thus, the proposed method effectively prevents the errors of the magnitude decay kernel from propagating to the final susceptibility separation reconstruction. Phantom studies, ex vivo macaque brain experiments, and in vivo human brain imaging studies were conducted to evaluate the ability of the proposed method to distinguish paramagnetic and diamagnetic susceptibility sources. The results demonstrate that the proposed method provides state-of-the-art performances for quantifying brain iron and myelin compared to previous QSM separation methods. Our results show that the proposed method has the potential to simultaneously quantify whole brain iron and myelin during brain development and aging. The proposed model was also deployed with multiple-orientation complex GRE data input measurements, resulting in high-quality QSM separation maps with more faithful tissue delineation between brain structures compared to those reconstructed by single-orientation QSM separation methods.

Details DOI

AAAI Conference 2023 Conference Paper

Curriculum Temperature for Knowledge Distillation

  • Zheng Li
  • Xiang Li
  • Lingfeng Yang
  • Borui Zhao
  • Renjie Song
  • Lei Luo
  • Jun Li
  • Jian Yang

Most existing distillation methods ignore the flexible role of the temperature in the loss function and fix it as a hyper-parameter that can be decided by an inefficient grid search. In general, the temperature controls the discrepancy between two distributions and can faithfully determine the difficulty level of the distillation task. Keeping a constant temperature, i.e., a fixed level of task difficulty, is usually sub-optimal for a growing student during its progressive learning stages. In this paper, we propose a simple curriculum-based technique, termed Curriculum Temperature for Knowledge Distillation (CTKD), which controls the task difficulty level during the student's learning career through a dynamic and learnable temperature. Specifically, following an easy-to-hard curriculum, we gradually increase the distillation loss w.r.t. the temperature, leading to increased distillation difficulty in an adversarial manner. As an easy-to-use plug-in technique, CTKD can be seamlessly integrated into existing knowledge distillation frameworks and brings general improvements at a negligible additional computation cost. Extensive experiments on CIFAR-100, ImageNet-2012, and MS-COCO demonstrate the effectiveness of our method.

PDF Details DOI

EAAI Journal 2023 Journal Article

Data-driven methods for stress field predictions in random heterogeneous materials

  • Enjamamul Hoq
  • Osama Aljarrah
  • Jun Li
  • Jing Bi
  • Alfa Heryudono
  • Wenzhen Huang

Predicting full-field stress responses is of fundamental importance to assessing materials failure and has various engineering applications in design optimization, manufacturing process control, and structural health monitoring. This article develops and evaluates different data-driven methods for efficient and accurate predictions of full stress fields in random heterogeneous materials. The first approach integrates model order reduction of proper orthogonal decomposition (POD) with classical machine learning techniques (K-nearest neighbors, random forest, and artificial neural networks) to predict full-field responses based on POD-reduced coefficients. However, this strategy shows limitations in predicting full stress fields, especially for heterogeneous material inclusions of small size or being close to the domain boundary. After that, two computer vision-based deep learning approaches were developed for full-field predictions. The first one uses a Resnet-based Convolutional Neural Network (CNN), and the second is based on a modified conditional Generative Adversarial Network (cGAN). Two representative example problems were studied: a random heterogeneous material inclusion or a void varying in size and location. In contrast to POD-based classical machine learning, almost invisible differences were found between the entire stress fields in finite element simulations and computer vision-based deep learning (CNN/cGAN) predictions, with significantly reduced mean squared error (MSE) and correlation values (R2) mostly above 0. 99. On the other hand, the proposed cGAN provides more accurate predictions than CNN with fewer epochs.

Details DOI

AAAI Conference 2023 Conference Paper

DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion

  • Zhiqiang Yan
  • Kun Wang
  • Xiang Li
  • Zhenyu Zhang
  • Jun Li
  • Jian Yang

Unsupervised depth completion aims to recover dense depth from the sparse one without using the ground-truth annotation. Although depth measurement obtained from LiDAR is usually sparse, it contains valid and real distance information, i.e., scale-consistent absolute depth values. Meanwhile, scale-agnostic counterparts seek to estimate relative depth and have achieved impressive performance. To leverage both the inherent characteristics, we thus suggest to model scale-consistent depth upon unsupervised scale-agnostic frameworks. Specifically, we propose the decomposed scale-consistent learning (DSCL) strategy, which disintegrates the absolute depth into relative depth prediction and global scale estimation, contributing to individual learning benefits. But unfortunately, most existing unsupervised scale-agnostic frameworks heavily suffer from depth holes due to the extremely sparse depth input and weak supervisory signal. To tackle this issue, we introduce the global depth guidance (GDG) module, which attentively propagates dense depth reference into the sparse target via novel dense-to-sparse attention. Extensive experiments show the superiority of our method on outdoor KITTI, ranking 1st and outperforming the best KBNet more than 12% in RMSE. Additionally, our approach achieves state-of-the-art performance on indoor NYUv2 benchmark as well.

PDF Details DOI

JBHI Journal 2023 Journal Article

Generalizable Pancreas Segmentation Modeling in CT Imaging via Meta-Learning and Latent-Space Feature Flow Generation

  • Jun Li
  • Tao Chen
  • Xiaohua Qian

Accurate pancreas segmentation is highly crucial for diagnosing and treating pancreatic diseases. Although CNN has demonstrated promising outcomes, the performance on unseen data can be significantly compromised by the wide appearance-style variations induced by different imaging factors. Thus, we propose a generalizable pancreas segmentation model based on a meta-learning strategy and latent-space feature flow generation method. Our approach enhances the generalizability by systematically reducing the interference from the cluttered background and appearance-style discrepancies through a coarse-to-fine workflow. Specifically, the integrity-preserving coarse segmentation module is designed to adaptively balance the pancreas coverage and segmentation accuracy with the meta-learning strategy for filtering out background clutter. It also enhances the generalization of the coarse model to reasonably-accurate ROIs thereby promoting the stability of fine segmentation. Subsequently, the appearance-style feature flow generation method is developed to generate a series of progressively-varying style-related intermediate representations between two latent spaces. This feature flow effectively models the distribution variations caused by appearance-style discrepancies, and thus enhances the adaptability of the fine model. Our method achieves superior performance on three pancreas datasets and outperforms state-of-the-art generalization methods. Besides, it can be easily integrated into other workflows, leading to a potential paradigm for enhancing generalization performance.

Details DOI

JBHI Journal 2023 Journal Article

Generalizable Pancreas Segmentation via a Dual Self-Supervised Learning Framework

  • Jun Li
  • Hongzhang Zhu
  • Tao Chen
  • Xiaohua Qian

Recently, numerous pancreas segmentation methods have achieved promising performance on local single-source datasets. However, these methods don't adequately account for generalizability issues, and hence typically show limited performance and low stability on test data from other sources. Considering the limited availability of distinct data sources, we seek to improve the generalization performance of a pancreas segmentation model trained with a single-source dataset, i. e. , the single-source generalization task. In particular, we propose a dual self-supervised learning model that incorporates both global and local anatomical contexts. Our model aims to fully exploit the anatomical features of the intra-pancreatic and extra-pancreatic regions, and hence enhance the characterization of the high-uncertainty regions for more robust generalization. Specifically, we first construct a global-feature contrastive self-supervised learning module that is guided by the pancreatic spatial structure. This module obtains complete and consistent pancreatic features through promoting intra-class cohesion, and also extracts more discriminative features for differentiating between pancreatic and non-pancreatic tissues through maximizing inter-class separation. It mitigates the influence of surrounding tissue on the segmentation outcomes in high-uncertainty regions. Subsequently, a local-image-restoration self-supervised learning module is introduced to further enhance the characterization of the high-uncertainty regions. In this module, informative anatomical contexts are actually learned to recover randomly-corrupted appearance patterns in those regions. The effectiveness of our method is demonstrated with state-of-the-art performance and comprehensive ablation analysis on three pancreas datasets (467 cases). The results demonstrate a great potential in providing a stable support for the diagnosis and treatment of pancreatic diseases.

Details DOI

JMLR Journal 2023 Journal Article

Online Change-Point Detection in High-Dimensional Covariance Structure with Application to Dynamic Networks

  • Lingjun Li
  • Jun Li

In this paper, we develop an online change-point detection procedure in the covariance structure of high-dimensional data. A new stopping rule is proposed to terminate the process as early as possible when a change in covariance structure occurs. The stopping rule allows spatial and temporal dependence and can be applied to non-Gaussian data. An explicit expression for the average run length is derived, so that the level of threshold in the stopping rule can be easily obtained with no need to run time-consuming Monte Carlo simulations. We also establish an upper bound for the expected detection delay, the expression of which demonstrates the impact of data dependence and magnitude of change in the covariance structure. Simulation studies are provided to confirm accuracy of the theoretical results. The practical usefulness of the proposed procedure is illustrated by detecting the change of brain’s covariance network in a resting-state fMRI data set. The implementation of the methodology is provided in the R package OnlineCOV. [abs] [ pdf ][ bib ] &copy JMLR 2023. ( edit, beta )

PDF Details

AAAI Conference 2023 Conference Paper

Recurrent Structure Attention Guidance for Depth Super-resolution

  • Jiayi Yuan
  • Haobo Jiang
  • Xiang Li
  • Jianjun Qian
  • Jun Li
  • Jian Yang

Image guidance is an effective strategy for depth super-resolution. Generally, most existing methods employ hand-crafted operators to decompose the high-frequency (HF) and low-frequency (LF) ingredients from low-resolution depth maps and guide the HF ingredients by directly concatenating them with image features. However, the hand-designed operators usually cause inferior HF maps (e.g., distorted or structurally missing) due to the diverse appearance of complex depth maps. Moreover, the direct concatenation often results in weak guidance because not all image features have a positive effect on the HF maps. In this paper, we develop a recurrent structure attention guided (RSAG) framework, consisting of two important parts. First, we introduce a deep contrastive network with multi-scale filters for adaptive frequency-domain separation, which adopts contrastive networks from large filters to small ones to calculate the pixel contrasts for adaptive high-quality HF predictions. Second, instead of the coarse concatenation guidance, we propose a recurrent structure attention block, which iteratively utilizes the latest depth estimation and the image features to jointly select clear patterns and boundaries, aiming at providing refined guidance for accurate depth recovery. In addition, we fuse the features of HF maps to enhance the edge structures in the decomposed LF maps. Extensive experiments show that our approach obtains superior performance compared with state-of-the-art depth super-resolution methods. Our code is available at: https://github.com/Yuanjiayii/DSR-RSAG.

PDF Details DOI

AAAI Conference 2023 Conference Paper

ScatterFormer: Locally-Invariant Scattering Transformer for Patient-Independent Multispectral Detection of Epileptiform Discharges

  • Ruizhe Zheng
  • Jun Li
  • Yi Wang
  • Tian Luo
  • Yuguo Yu

Patient-independent detection of epileptic activities based on visual spectral representation of continuous EEG (cEEG) has been widely used for diagnosing epilepsy. However, precise detection remains a considerable challenge due to subtle variabilities across subjects, channels and time points. Thus, capturing fine-grained, discriminative features of EEG patterns, which is associated with high-frequency textural information, is yet to be resolved. In this work, we propose Scattering Transformer (ScatterFormer), an invariant scattering transform-based hierarchical Transformer that specifically pays attention to subtle features. In particular, the disentangled frequency-aware attention (FAA) enables the Transformer to capture clinically informative high-frequency components, offering a novel clinical explainability based on visual encoding of multichannel EEG signals. Evaluations on two distinct tasks of epileptiform detection demonstrate the effectiveness our method. Our proposed model achieves median AUCROC and accuracy of 98.14%, 96.39% in patients with Rolandic epilepsy. On a neonatal seizure detection benchmark, it outperforms the state-of-the-art by 9% in terms of average AUCROC.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Structure Flow-Guided Network for Real Depth Super-resolution

  • Jiayi Yuan
  • Haobo Jiang
  • Xiang Li
  • Jianjun Qian
  • Jun Li
  • Jian Yang

Real depth super-resolution (DSR), unlike synthetic settings, is a challenging task due to the structural distortion and the edge noise caused by the natural degradation in real-world low-resolution (LR) depth maps. These defeats result in significant structure inconsistency between the depth map and the RGB guidance, which potentially confuses the RGB-structure guidance and thereby degrades the DSR quality. In this paper, we propose a novel structure flow-guided DSR framework, where a cross-modality flow map is learned to guide the RGB-structure information transferring for precise depth upsampling. Specifically, our framework consists of a cross-modality flow-guided upsampling network (CFUNet) and a flow-enhanced pyramid edge attention network (PEANet). CFUNet contains a trilateral self-attention module combining both the geometric and semantic correlations for reliable cross-modality flow learning. Then, the learned flow maps are combined with the grid-sampling mechanism for coarse high-resolution (HR) depth prediction. PEANet targets at integrating the learned flow map as the edge attention into a pyramid network to hierarchically learn the edge-focused guidance feature for depth edge refinement. Extensive experiments on real and synthetic DSR datasets verify that our approach achieves excellent performance compared to state-of-the-art methods. Our code is available at: https://github.com/Yuanjiayii/DSR-SFG.

PDF Details DOI

NeurIPS Conference 2022 Conference Paper

Learning Contrastive Embedding in Low-Dimensional Space

  • Shuo Chen
  • Chen Gong
  • Jun Li
  • Jian Yang
  • Gang Niu
  • Masashi Sugiyama

Contrastive learning (CL) pretrains feature embeddings to scatter instances in the feature space so that the training data can be well discriminated. Most existing CL techniques usually encourage learning such feature embeddings in the highdimensional space to maximize the instance discrimination. However, this practice may lead to undesired results where the scattering instances are sparsely distributed in the high-dimensional feature space, making it difficult to capture the underlying similarity between pairwise instances. To this end, we propose a novel framework called contrastive learning with low-dimensional reconstruction (CLLR), which adopts a regularized projection layer to reduce the dimensionality of the feature embedding. In CLLR, we build the sparse / low-rank regularizer to adaptively reconstruct a low-dimensional projection space while preserving the basic objective for instance discrimination, and thus successfully learning contrastive embeddings that alleviate the above issue. Theoretically, we prove a tighter error bound for CLLR; empirically, the superiority of CLLR is demonstrated across multiple domains. Both theoretical and experimental results emphasize the significance of learning low-dimensional contrastive embeddings.

PDF Details

AAAI Conference 2022 Conference Paper

Linearity-Aware Subspace Clustering

  • Yesong Xu
  • Shuo Chen
  • Jun Li
  • Jianjun Qian

Obtaining a good similarity matrix is extremely important in subspace clustering. Current state-of-the-art methods learn the similarity matrix through self-expressive strategy. However, these methods directly adopt original samples as a set of basis to represent itself linearly. It is difficult to accurately describe the linear relation between samples in the real-world applications, and thus is hard to find an ideal similarity matrix. To better represent the linear relation of samples, we present a subspace clustering model, Linearity-Aware Subspace Clustering (LASC), which can consciously learn the similarity matrix by employing a linearity-aware metric. This is a new subspace clustering method that combines metric learning and subspace clustering into a joint learning framework. In our model, we first utilize the self-expressive strategy to obtain an initial subspace structure and discover a low-dimensional representation of the original data. Subsequently, we use the proposed metric to learn an intrinsic similarity matrix with linearity-aware on the obtained subspace. Based on such a learned similarity matrix, the inter-cluster distance becomes larger than the intra-cluster distances, and thus successfully obtaining a good subspace cluster result. In addition, to enrich the similarity matrix with more consistent knowledge, we adopt a collaborative learning strategy for self-expressive subspace learning and linearity-aware subspace learning. Moreover, we provide detailed mathematical analysis to show that the metric can properly characterize the linear correlation between samples.

PDF Details

YNICL Journal 2022 Journal Article

Reduced sensitivity to delayed time and delayed reward of the post-operative insular glioma patients in delay discounting

  • Wenjin Fu
  • Zhenxing Huang
  • Jun Li
  • Qi Dong
  • Yang Li
  • Gen Li
  • Yaokai Xu
  • Bowen Xue

Previous studies have shown that the insula is closely related to addiction, and the structure's role in delay discounting can be measured by a specific task, but the specific role of the insula has been less studied. In this study, we first conducted a lesion study in which we recruited healthy controls (n = 30) and patients with unilateral insula injury (n = 16) to complete a behavioral delay discounting task. Then we conducted a functional magnetic resonance imaging (fMRI) study, and a separate group healthy volunteers (n = 51) completed a delay discounting task during the fMRI scan. The lesion study showed a significant difference between the two groups in the delay discounting task, which revealed that insula injury was associated with impaired decision making. The fMRI study revealed choice-sensitive insula activation that was modulated by delayed time and delayed reward, indicating an important role of the insula in delay discounting. Overall, our results provide evidence for a role of the insular lobe in delay discounting and suggests that this structure may be considered an important factor in the future treatment and diagnosis of addiction disorders.

Details DOI

JBHI Journal 2022 Journal Article

S2Q-Net: Mining the High-Pass Filtered Phase Data in Susceptibility Weighted Imaging for Quantitative Susceptibility Mapping

  • Zhiyang Lu
  • Jun Li
  • Chaoyue Wang
  • Rongjun Ge
  • Lili Chen
  • Hongjian He
  • Jun Shi

Susceptibility weighted imaging (SWI) is a routine magnetic resonance imaging (MRI) sequence that combines the magnitude and high-pass filtered phase images to qualitatively enhance the image contrasts related to tissue susceptibility. Tremendous amounts of the high-pass filtered phase data with low signal to noise ratio and incomplete background field removal have thus been collected under default clinical settings. Since SWI cannot quantitatively estimate the susceptibility, it is thus non-trivial to derive quantitative susceptibility mapping (QSM) directly from these redundant phase data, which effectively promotes the mining of the SWI data collected previously. To this end, a novel deep learning based SWI-to-QSM-Net (S2Q-Net) is proposed for QSM reconstruction from SWI high-pass filtered phase data. S2Q-Net firstly estimates the edge maps of QSM to integrate edge prior into features, which benefits the network to reconstruct QSM with realistic and clear tissue boundaries. Furthermore, a novel Second-order Cross Dense Block is proposed in S2Q-Net, which can capture rich inter-region interactions to provide more non-local phase information related to local tissue susceptibility. Experimental results on both simulated and in-vivo data indicate its superiority over all the compared deep learning based QSM reconstruction methods.

Details DOI

YNIMG Journal 2022 Journal Article

Subthalamic and pallidal stimulation in Parkinson's disease induce distinct brain topological reconstruction

  • Chunguang Chu
  • Naying He
  • Kristina Zeljic
  • Zhen Zhang
  • Jiang Wang
  • Jun Li
  • Yu Liu
  • Youmin Zhang

The subthalamic nucleus (STN) and globus pallidus internus (GPi) are the two most common and effective target brain areas for deep brain stimulation (DBS) treatment of advanced Parkinson's disease. Although DBS has been shown to restore functional neural circuits of this disorder, the changes in topological organization associated with active DBS of each target remain unknown. To investigate this, we acquired resting-state functional magnetic resonance imaging (fMRI) data from 34 medication-free patients with Parkinson's disease that had DBS electrodes implanted in either the subthalamic nucleus or internal globus pallidus (n = 17 each), in both ON and OFF DBS states. Sixteen age-matched healthy individuals were used as a control group. We evaluated the regional information processing capacity and transmission efficiency of brain networks with and without stimulation, and recorded how stimulation restructured the brain network topology of patients with Parkinson's disease. For both targets, the variation of local efficiency in motor brain regions was significantly correlated (p < 0.05) with improvement rate of the Uniform Parkinson's Disease Rating Scale-III scores, with comparable improvements in motor function for the two targets. However, non-motor brain regions showed changes in topological organization during active stimulation that were target-specific. Namely, targeting the STN decreased the information transmission of association, limbic and paralimbic regions, including the inferior frontal gyrus angle, insula, temporal pole, superior occipital gyri, and posterior cingulate, as evidenced by the simultaneous decrease of clustering coefficient and local efficiency. GPi-DBS had a similar effect on the caudate and lenticular nuclei, but enhanced information transmission in the cingulate gyrus. These effects were not present in the DBS-OFF state for GPi-DBS, but persisted for STN-DBS. Our results demonstrate that DBS to the STN and GPi induce distinct brain network topology reconstruction patterns, providing innovative theoretical evidence for deciphering the mechanism through which DBS affects disparate targets in the human brain.

Details DOI

JBHI Journal 2022 Journal Article

Utilizing GCN and Meta-Learning Strategy in Unsupervised Domain Adaptation for Pancreatic Cancer Segmentation

  • Jun Li
  • Chaolu Feng
  • Xiaozhu Lin
  • Xiaohua Qian

Automated pancreatic cancer segmentation is highly crucial for computer-assisted diagnosis. The general practice is to label images from selected modalities since it is expensive to label all modalities. This practice brought about a significant interest in learning the knowledge transfer from the labeled modalities to unlabeled ones. However, the imaging parameter inconsistency between modalities leads to a domain shift, limiting the transfer learning performance. Therefore, we propose an unsupervised domain adaptation segmentation framework for pancreatic cancer based on GCN and meta-learning strategy. Our model first transforms the source image into a target-like visual appearance through the synergistic collaboration between image and feature adaptation. Specifically, we employ encoders incorporating adversarial learning to separate domain-invariant features from domain-specific ones to achieve visual appearance translation. Then, the meta-learning strategy with good generalization capabilities is exploited to strike a reasonable balance in the training of the source and transformed images. Thus, the model acquires more correlated features and improve the adaptability to the target images. Moreover, a GCN is introduced to supervise the high-dimensional abstract features directly related to the segmentation outcomes, and hence ensure the integrity of key structural features. Extensive experiments on four multi-parameter pancreatic-cancer magnetic resonance imaging datasets demonstrate improved performance in all adaptation directions, confirming our model's effectiveness for unlabeled pancreatic cancer images. The results are promising for reducing the burden of annotation and improving the performance of computer-aided diagnosis of pancreatic cancer. Our source codes will be released at https://github.com/SJTUBME-QianLab/UDAseg, once this manuscript is accepted for publication.

Details DOI

AIIM Journal 2021 Journal Article

Improving prediction for medical institution with limited patient data: Leveraging hospital-specific data based on multicenter collaborative research network

  • Jin Li
  • Yu Tian
  • Runze Li
  • Tianshu Zhou
  • Jun Li
  • Kefeng Ding
  • Jingsong Li

Background and objective Clinical decision support assisted by prediction models usually faces the challenges of limited clinical data and a lack of labels when the model is developed with data from a single medical institution. Accordingly, research on multicenter clinical collaborative networks, which can provide external medical data, has received increasing attention. With the increasing availability of machine learning techniques such as transfer learning, leveraging large-scale patient data from multiple hospitals to build data-driven predictive models with clinical application potential provides an alternative solution to address the problem of limited patient data. Methods A multicenter hybrid semi-supervised transfer learning model (MHSTL) is proposed in this study on the basis of unified common data model to ensure multicenter data standardized representation. Then the hospital-specific features, along with the co-occurrence features across domains, are aligned through a representation learning architecture that is built based on deep neural networks and the newly proposed neural decision forest model. In this process, limited patient data from the target hospital, both labeled and unlabeled, are incorporated during the feature adaptation process, thereby contributing to better model performance. Without patient-level data sharing, the proposed model learning strategy which overcomes feature misalignment and distribution divergence, enables the multi-source transfer learning process in the case of insufficient and unlabeled patient data at target hospital. Results The effectiveness of the proposed transfer learning model was evaluated on a collaborative research network of colorectal cancer patients in the US and China. The results demonstrate that the proposed model can achieve much better performance for predicting target risk with limited resources on patient data than baseline models. Better discrimination and calibration ability are also observed when sufficient labeled data are not available in the target hospital for prognosis prediction tasks. Further exploratory experiments show that the proposed approach exhibits good model generalizability regardless of the data heterogeneity. With the help of the SHapley Additive exPlanations for model interpretation, the effectiveness of incorporating hospital-specific features in the transfer learning model is shown. Conclusions In this study, the proposed method can develop prediction models from multiple source hospitals and exhibit good performance by leveraging cross-domain hospital-specific feature information, therefore enhancing the model prediction when applied to single medical institution with limited patient data.

Details DOI

AIIM Journal 2020 Journal Article

A multicenter random forest model for effective prognosis prediction in collaborative clinical research network

  • Jin Li
  • Yu Tian
  • Yan Zhu
  • Tianshu Zhou
  • Jun Li
  • Kefeng Ding
  • Jingsong Li

Background The accuracy of a prognostic prediction model has become an essential aspect of the quality and reliability of the health-related decisions made by clinicians in modern medicine. Unfortunately, individual institutions often lack sufficient samples, which might not provide sufficient statistical power for models. One mitigation is to expand data collection from a single institution to multiple centers to collectively increase the sample size. However, sharing sensitive biomedical data for research involves complicated issues. Machine learning models such as random forests (RF), though they are commonly used and achieve good performances for prognostic prediction, usually suffer worse performance under multicenter privacy-preserving data mining scenarios compared to a centrally trained version. Methods and materials In this study, a multicenter random forest prognosis prediction model is proposed that enables federated clinical data mining from horizontally partitioned datasets. By using a novel data enhancement approach based on a differentially private generative adversarial network customized to clinical prognosis data, the proposed model is able to provide a multicenter RF model with performances on par with—or even better than—centrally trained RF but without the need to aggregate the raw data. Moreover, our model also incorporates an importance ranking step designed for feature selection without sharing patient-level information. Result The proposed model was evaluated on colorectal cancer datasets from the US and China. Two groups of datasets with different levels of heterogeneity within the collaborative research network were selected. First, we compare the performance of the distributed random forest model under different privacy parameters with different percentages of enhancement datasets and validate the effectiveness and plausibility of our approach. Then, we compare the discrimination and calibration ability of the proposed multicenter random forest with a centrally trained random forest model and other tree-based classifiers as well as some commonly used machine learning methods. The results show that the proposed model can provide better prediction performance in terms of discrimination and calibration ability than the centrally trained RF model or the other candidate models while following the privacy-preserving rules in both groups. Additionally, good discrimination and calibration ability are shown on the simplified model based on the feature importance ranking in the proposed approach. Conclusion The proposed random forest model exhibits ideal prediction capability using multicenter clinical data and overcomes the performance limitation arising from privacy guarantees. It can also provide feature importance ranking across institutions without pooling the data at a central site. This study offers a practical solution for building a prognosis prediction model in the collaborative clinical research network and solves practical issues in real-world applications of medical artificial intelligence.

Details DOI

YNICL Journal 2020 Journal Article

Effect of ZNF804A gene polymorphism (rs1344706) on the plasticity of the functional coupling between the right dorsolateral prefrontal cortex and the contralateral hippocampal formation

  • Wan Zhao
  • Xiongying Chen
  • Qiumei Zhang
  • Boqi Du
  • Xiaoxiang Deng
  • Feng Ji
  • Yu-Tao Xiang
  • Chuanyue Wang

ZNF804A has now been recognized as a schizophrenia risk gene by multiple genome-wide association studies with its intronic polymorphism rs1344706 being reported as the first genome-wide significant risk variant for schizophrenia. Although the functional impact of this gene is still unknown, rs1344706's contribution to the functional coupling between the right dorsolateral prefrontal cortex (DLPFC) and the contralateral hippocampal formation (HF) has been reported by several studies. The current study tested whether the right DLPFC-left HF functional coupling showed plasticity during cognitive training (Study I) and whether rs1344706 affected the plasticity (Study II). In Study I, we conducted a randomized controlled trial with 30 subjects receiving 20 sessions of adaptive training on a memory span task (the training group) and 30 subjects practicing on a non-adaptive easy version of the same memory span task for 20 sessions (the control group). All subjects were scanned using fMRI before and after the training. Analyses of resting-state and task-state fMRI data consistently showed that the adaptive memory span training significantly strengthened the right DLPFC-left HF functional coupling. In Study II, we conducted a genetic association study with 101 subjects (combining the data from the training group in Study I with those from an additional subsequent sample of 71 subjects who received the same training and fMRI scans). Results showed that rs1344706 was significantly associated with training-induced changes in functional coupling. Subjects carrying the non-risk allele (C) of rs1344706 showed greater training-induced plasticity than the risk allele (A) homozygotes. These findings expanded our current understanding of the functional impact of the schizophrenia risk variant of ZNF804A gene and suggested that the ZNF804A gene could be used as a prospective target for future antipsychotic drugs and clinical research.

Details DOI

NeurIPS Conference 2020 Conference Paper

Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection

  • Xiang Li
  • Wenhai Wang
  • Lijun Wu
  • Shuo Chen
  • Xiaolin Hu
  • Jun Li
  • Jinhui Tang
  • Jian Yang

One-stage detector basically formulates object detection as dense classification and localization (i. e. , bounding box regression). The classification is usually optimized by Focal Loss and the box location is commonly learned under Dirac delta distribution. A recent trend for one-stage detectors is to introduce an \emph{individual} prediction branch to estimate the quality of localization, where the predicted quality facilitates the classification to improve detection performance. This paper delves into the \emph{representations} of the above three fundamental elements: quality estimation, classification and localization. Two problems are discovered in existing practices, including (1) the inconsistent usage of the quality estimation and classification between training and inference, and (2) the inflexible Dirac delta distribution for localization. To address the problems, we design new representations for these elements. Specifically, we merge the quality estimation into the class prediction vector to form a joint representation, and use a vector to represent arbitrary distribution of box locations. The improved representations eliminate the inconsistency risk and accurately depict the flexible distribution in real data, but contain \emph{continuous} labels, which is beyond the scope of Focal Loss. We then propose Generalized Focal Loss (GFL) that generalizes Focal Loss from its discrete form to the \emph{continuous} version for successful optimization. On COCO {\tt test-dev}, GFL achieves 45. 0\% AP using ResNet-101 backbone, surpassing state-of-the-art SAPD (43. 5\%) and ATSS (43. 6\%) with higher or comparable inference speed.

PDF Details

AAAI Conference 2020 Conference Paper

Learning Part Generation and Assembly for Structure-Aware Shape Synthesis

  • Jun Li
  • Chengjie Niu
  • Kai Xu

Learning powerful deep generative models for 3D shape synthesis is largely hindered by the difficulty in ensuring plausibility encompassing correct topology and reasonable geometry. Indeed, learning the distribution of plausible 3D shapes seems a daunting task for the holistic approaches, given the significant topological variations of 3D objects even within the same category. Enlightened by the fact that 3D shape structure is characterized as part composition and placement, we propose to model 3D shape variations with a part-aware deep generative network, coined as PAGENet. The network is composed of an array of per-part VAE-GANs, generating semantic parts composing a complete shape, followed by a part assembly module that estimates a transformation for each part to correlate and assemble them into a plausible structure. Through delegating the learning of part composition and part placement into separate networks, the difficulty of modeling structural variations of 3D shapes is greatly reduced. We demonstrate through both qualitative and quantitative evaluations that PAGENet generates 3D shapes with plausible, diverse and detailed structure, and show two applications, i. e. , semantic shape segmentation and part-based shape editing.

PDF Details

AAAI Conference 2020 Conference Paper

Lifelong Spectral Clustering

  • Gan Sun
  • Yang Cong
  • Qianqian Wang
  • Jun Li
  • Yun Fu

In the past decades, spectral clustering (SC) has become one of the most effective clustering algorithms. However, most previous studies focus on spectral clustering tasks with a fixed task set, which cannot incorporate with a new spectral clustering task without accessing to previously learned tasks. In this paper, we aim to explore the problem of spectral clustering in a lifelong machine learning framework, i. e. , Lifelong Spectral Clustering (L2 SC). Its goal is to efficiently learn a model for a new spectral clustering task by selectively transferring previously accumulated experience from knowledge library. Specifically, the knowledge library of L2 SC contains two components: 1) orthogonal basis library: capturing latent cluster centers among the clusters in each pair of tasks; 2) feature embedding library: embedding the feature manifold information shared among multiple related tasks. As a new spectral clustering task arrives, L2 SC firstly transfers knowledge from both basis library and feature library to obtain encoding matrix, and further redefines the library base over time to maximize performance across all the clustering tasks. Meanwhile, a general online update formulation is derived to alternatively update the basis library and feature library. Finally, the empirical experiments on several real-world benchmark datasets demonstrate that our L2 SC model can effectively improve the clustering performance when comparing with other state-of-the-art spectral clustering algorithms.

PDF Details

AAAI Conference 2020 Conference Paper

Robust Low-Rank Discovery of Data-Driven Partial Differential Equations

  • Jun Li
  • Gan Sun
  • Guoshuai Zhao
  • Li-wei H. Lehman

Partial differential equations (PDEs) are essential foundations to model dynamic processes in natural sciences. Discovering the underlying PDEs of complex data collected from real world is key to understanding the dynamic processes of natural laws or behaviors. However, both the collected data and their partial derivatives are often corrupted by noise, especially from sparse outlying entries, due to measurement/process noise in the real-world applications. Our work is motivated by the observation that the underlying data modeled by PDEs are in fact often low rank. We thus develop a robust low-rank discovery framework to recover both the low-rank data and the sparse outlying entries by integrating double low-rank and sparse recoveries with a (group) sparse regression method, which is implemented as a minimization problem using mixed nuclear norms with ℓ1 and ℓ0 norms. We propose a low-rank sequential (grouped) threshold ridge regression algorithm to solve the minimization problem. Results from several experiments on seven canonical models (i.e., four PDEs and three parametric PDEs) verify that our framework outperforms the state-of-art sparse and group sparse regression methods. Code is available at https://github.com/junli2019/Robust-Discovery-of-PDEs

PDF Details

IJCAI Conference 2019 Conference Paper

Adversarial Graph Embedding for Ensemble Clustering

  • Zhiqiang Tao
  • Hongfu Liu
  • Jun Li
  • Zhaowen Wang
  • Yun Fu

Ensemble clustering generally integrates basic partitions into a consensus one through a graph partitioning method, which, however, has two limitations: 1) it neglects to reuse original features; 2) obtaining consensus partition with learnable graph representations is still under-explored. In this paper, we propose a novel Adversarial Graph Auto-Encoders (AGAE) model to incorporate ensemble clustering into a deep graph embedding process. Specifically, graph convolutional network is adopted as probabilistic encoder to jointly integrate the information from feature content and consensus graph, and a simple inner product layer is used as decoder to reconstruct graph with the encoded latent variables (i. e. , embedding representations). Moreover, we develop an adversarial regularizer to guide the network training with an adaptive partition-dependent prior. Experiments on eight real-world datasets are presented to show the effectiveness of AGAE over several state-of-the-art deep embedding and ensemble clustering methods.

PDF Details

NeurIPS Conference 2019 Conference Paper

Curvilinear Distance Metric Learning

  • Shuo Chen
  • Lei Luo
  • Jian Yang
  • Chen Gong
  • Jun Li
  • Heng Huang

Distance Metric Learning aims to learn an appropriate metric that faithfully measures the distance between two data points. Traditional metric learning methods usually calculate the pairwise distance with fixed distance functions (\emph{e. g. ,}\ Euclidean distance) in the projected feature spaces. However, they fail to learn the underlying geometries of the sample space, and thus cannot exactly predict the intrinsic distances between data points. To address this issue, we first reveal that the traditional linear distance metric is equivalent to the cumulative arc length between the data pair's nearest points on the learned straight measurer lines. After that, by extending such straight lines to general curved forms, we propose a Curvilinear Distance Metric Learning (CDML) method, which adaptively learns the nonlinear geometries of the training data. By virtue of Weierstrass theorem, the proposed CDML is equivalently parameterized with a 3-order tensor, and the optimization algorithm is designed to learn the tensor parameter. Theoretical analysis is derived to guarantee the effectiveness and soundness of CDML. Extensive experiments on the synthetic and real-world datasets validate the superiority of our method over the state-of-the-art metric learning models.

PDF Details

AAAI Conference 2019 Conference Paper

Data-Adaptive Metric Learning with Scale Alignment

  • Shuo Chen
  • Chen Gong
  • Jian Yang
  • Ying Tai
  • Le Hui
  • Jun Li

The central problem for most existing metric learning methods is to find a suitable projection matrix on the differences of all pairs of data points. However, a single unified projection matrix can hardly characterize all data similarities accurately as the practical data are usually very complicated, and simply adopting one global projection matrix might ignore important local patterns hidden in the dataset. To address this issue, this paper proposes a novel method dubbed “Data-Adaptive Metric Learning” (DAML), which constructs a data-adaptive projection matrix for each data pair by selectively combining a set of learned candidate matrices. As a result, every data pair can obtain a specific projection matrix, enabling the proposed DAML to flexibly fit the training data and produce discriminative projection results. The model of DAML is formulated as an optimization problem which jointly learns candidate projection matrices and their sparse combination for every data pair. Nevertheless, the over-fitting problem may occur due to the large amount of parameters to be learned. To tackle this issue, we adopt the Total Variation (TV) regularizer to align the scales of data embedding produced by all candidate projection matrices, and thus the generated metrics of these learned candidates are generally comparable. Furthermore, we extend the basic linear DAML model to the kernerlized version (denoted “KDAML”) to handle the non-linear cases, and the Iterative Shrinkage-Thresholding Algorithm (ISTA) is employed to solve the optimization model. Intensive experimental results on various applications including retrieval, classification, and verification clearly demonstrate the superiority of our algorithm to other state-of-the-art metric learning methodologies.

PDF Details

IJCAI Conference 2019 Conference Paper

Dense Transformer Networks for Brain Electron Microscopy Image Segmentation

  • Jun Li
  • Yongjun Chen
  • Lei Cai
  • Ian Davidson
  • Shuiwang Ji

The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by network architecture instead of learned from data. In this work, we propose the dense transformer networks, which can learn the shapes and sizes of patches from data. The dense transformer networks employ an encoder-decoder architecture, and a pair of dense transformer modules are inserted into each of the encoder and decoder paths. The novelty of this work is that we provide technical solutions for learning the shapes and sizes of patches from data and efficiently restoring the spatial correspondence required for dense prediction. The proposed dense transformer modules are differentiable, thus the entire network can be trained. We apply the proposed networks on biological image segmentation tasks and show superior performance is achieved in comparison to baseline methods.

PDF Details

AAAI Conference 2019 Conference Paper

One-Class Adversarial Nets for Fraud Detection

  • Panpan Zheng
  • Shuhan Yuan
  • Xintao Wu
  • Jun Li
  • Aidong Lu

Many online applications, such as online social networks or knowledge bases, are often attacked by malicious users who commit different types of actions such as vandalism on Wikipedia or fraudulent reviews on eBay. Currently, most of the fraud detection approaches require a training dataset that contains records of both benign and malicious users. However, in practice, there are often no or very few records of malicious users. In this paper, we develop one-class adversarial nets (OCAN) for fraud detection with only benign users as training data. OCAN first uses LSTM-Autoencoder to learn the representations of benign users from their sequences of online activities. It then detects malicious users by training a discriminator of a complementary GAN model that is different from the regular GAN model. Experimental results show that our OCAN outperforms the state-of-the-art oneclass classification models and achieves comparable performance with the latest multi-source LSTM model that requires both benign and malicious users in the training phase.

PDF Details

YNIMG Journal 2019 Journal Article

Photoacoustic microscopy of obesity-induced cerebrovascular alterations

  • Rui Cao
  • Jun Li
  • Chenchu Zhang
  • Zhiyi Zuo
  • Song Hu

Cerebral small vessel disease has been linked to cognitive, psychiatric and physical disabilities, especially in the elderly. However, the underlying pathophysiology remains incompletely understood, largely due to the limited accessibility of these small vessels in the live brain. Here, we report an intravital imaging and analysis platform for high-resolution, quantitative and comprehensive characterization of pathological alterations in the mouse cerebral microvasculature. By exploiting multi-parametric photoacoustic microscopy (PAM), microvascular structure, blood perfusion, oxygenation and flow were imaged in the awake brain. With the aid of vessel segmentation, these structural and functional parameters were extracted at the single-microvessel level, from which vascular density, tortuosity, wall shear stress, resistance and associated cerebral oxygen extraction fraction and metabolism were also quantified. With the use of vasodilatory stimulus, multifaceted cerebrovascular reactivity (CVR) was characterized in vivo. By extending the classic Evans blue assay to in vivo, permeability of the blood-brain barrier (BBB) was dynamically evaluated. The utility of this enabling technique was examined by studying cerebrovascular alterations in an established mouse model of high-fat diet-induced obesity. Our results revealed increased vascular density, reduced arterial flow, enhanced oxygen extraction, impaired BBB integrity, and increased multifaceted CVR in the obese brain. Interestingly, the ‘counterintuitive’ increase of CVR was supported by the elevated active endothelial nitric oxide synthase in the obese mouse. Providing comprehensive and quantitative insights into cerebral microvessels and their responses under pathological conditions, this technique opens a new door to mechanistic studies of the cerebral small vessel disease and its implications in neurodegeneration and stroke.

Details DOI

IJCAI Conference 2018 Conference Paper

Adversarial Metric Learning

  • Shuo Chen
  • Chen Gong
  • Jian Yang
  • Xiang Li
  • Yang Wei
  • Jun Li

In the past decades, intensive efforts have been put to design various loss functions and metric forms for metric learning problem. These improvements have shown promising results when the test data is similar to the training data. However, the trained models often fail to produce reliable distances on the ambiguous test pairs due to the different samplings between training set and test set. To address this problem, the Adversarial Metric Learning (AML) is proposed in this paper, which automatically generates adversarial pairs to remedy the sampling bias and facilitate robust metric learning. Specifically, AML consists of two adversarial stages, i. e. confusion and distinguishment. In confusion stage, the ambiguous but critical adversarial data pairs are adaptively generated to mislead the learned metric. In distinguishment stage, a metric is exhaustively learned to try its best to distinguish both adversarial pairs and original training pairs. Thanks to the challenges posed by the confusion stage in such competing process, the AML model is able to grasp plentiful difficult knowledge that has not been contained by the original training pairs, so the discriminability of AML can be significantly improved. The entire model is formulated into optimization framework, of which the global convergence is theoretically proved. The experimental results on toy data and practical datasets clearly demonstrate the superiority of AML to representative state-of-the-art metric learning models.

PDF Details

YNICL Journal 2018 Journal Article

Effect of rs1344706 in the ZNF804A gene on the brain network

  • Xiongying Chen
  • Zhifang Zhang
  • Qiumei Zhang
  • Wan Zhao
  • Jinguo Zhai
  • Min Chen
  • Boqi Du
  • Xiaoxiang Deng

ZNF804A rs1344706 (A/C) was the first SNP that reached genome-wide significance for schizophrenia. Recent studies have linked rs1344706 to functional connectivity among specific brain regions. However, no study thus far has examined the role of this SNP in the entire functional connectome. In this study, we used degree centrality to test the role of rs1344706 in the whole-brain voxel-wise functional connectome during the resting state. 52 schizophrenia patients and 128 healthy controls were included in the final analysis. In our whole-brain analysis, we found a significant interaction effect of genotype×diagnosis at the precuneus (PCU) (cluster size=52 voxels, peak voxel MNI coordinates: x=9, y=−69, z=63, F =32. 57, FWE corrected P <0. 001). When we subdivided the degree centrality network according to anatomical distance, the whole-brain analysis also found a significant interaction effect of genotype×diagnosis at the PCU with the same peak in the short-range degree centrality network (cluster size=72 voxels, F =37. 29, FWE corrected P <0. 001). No significant result was found in the long-range degree centrality network. Our results elucidated the contribution of rs1344706 to functional connectivity within the brain network, and may have important implications for our understanding of this risk gene's role in functional dysconnectivity in schizophrenia.

Details DOI

YNICL Journal 2018 Journal Article

Polymorphism in schizophrenia risk gene MIR137 is associated with the posterior cingulate Cortex's activation and functional and structural connectivity in healthy controls

  • Zhifang Zhang
  • Tongjun Yan
  • Yanyan Wang
  • Qiumei Zhang
  • Wan Zhao
  • Xiongying Chen
  • Jinguo Zhai
  • Min Chen

MIR137 gene has been repeatedly reported as a schizophrenia risk gene in genome-wide association studies (GWAS). A polymorphism (rs1625579) at the MIR137 gene has been associated with both neural activation and behavioral performance during a working memory task. This study examined MIR137's associations with task-related (N-back working memory) fMRI, resting state fMRI, and diffusion tensor images (DTI) data in 177 healthy adults. We found less deactivation of the PCC in risk allele homozygotes (TT) as compared to the GT heterozygotes (cluster size = 630 voxels, cluster level P FWE < 0. 001) during the N-back task, which replicated previous findings. Using the identified cluster within the PCC as the seed, we further found decreased functional connectivity between the PCC and the anterior cingulate cortex and its adjacent medial prefrontal cortex (ACC/MPFC) in risk allele homozygotes during both resting state (cluster size = 427 voxels, cluster level P FWE = 0. 001) and the N-back task (cluster size = 73 voxels, cluster level P FWE = 0. 05). Finally, an analysis of our DTI data showed decreased white matter integrity of the posterior cingulum in risk allele homozygotes (cluster size = 214 voxels, cluster level P FWE = 0. 03). Taken together, rs1625579 seems to play an important role in both functional and structural connectivity between the PCC and the ACC/MPFC, which may serve as the brain mechanisms for the link between rs1625579 and schizophrenia.

Details DOI

AAAI Conference 2018 Conference Paper

Predictive Coding Machine for Compressed Sensing and Image Denoising

  • Jun Li
  • Hongfu Liu
  • Yun Fu

Sparse and low rank coding has widely received much attention in machine learning, multimedia and computer vision. Unfortunately, expensive inference restricts the power of coding models in real-world applications, e. g. , compressed sensing and image deblurring. In order to avoid the expensive inference, we propose a predictive coding machine (PCM) which aims to train a deep neural network (DNN) encoder to approximate the codes. By this means, a test sample can be fast approximated by the well-trained DNN. However, DNN leads PCM to be a non-convex and non-smooth optimization problem, which is extremely hard to solve. To address this challenge, we extend accelerated proximal gradient for PCM by steering gradient descent of DNN. To the best of our knowledge, we are the first to propose a gradient descent algorithm guided by accelerated proximal gradient for solving the PCM problem. Besides, a sufficient condition is provided to ensure the convergence to a critical point. Moreover, when the coding models are convex in PCM, the convergence rate O(1/(m2 √ t)) can be held in which m is the iteration number of accelerated proximal gradient, and t is the epoch of training DNN. Numerical results verify the promising advantages of PCM in terms of effectiveness, efficiency and robustness.

PDF Details

IJCAI Conference 2018 Conference Paper

Progressive Generative Hashing for Image Retrieval

  • Yuqing Ma
  • Yue He
  • Fan Ding
  • Sheng Hu
  • Jun Li
  • Xianglong Liu

Recent years have witnessed the success of the emerging hashing techniques in large-scale image retrieval. Owing to the great learning capacity, deep hashing has become one of the most promising solutions, and achieved attractive performance in practice. However, without semantic label information, the unsupervised deep hashing still remains an open question. In this paper, we propose a novel progressive generative hashing (PGH) framework to help learn a discriminative hashing network in an unsupervised way. Very different from existing studies, it first treats the hash codes as a kind of semantic condition for the similar image generation, and simultaneously feeds the original image and its codes into the generative adversarial networks (GANs). The real images together with the synthetic ones can further help train a discriminative hashing network based on a triplet loss. By iteratively inputting the learnt codes into the hash conditioned GANs, we can progressively enable the hashing network to discover the semantic relations. Extensive experiments on the widely-used image datasets demonstrate that PGH can significantly outperforms state-of-the-art unsupervised hashing methods.

PDF Details

YNIMG Journal 2017 Journal Article

Functional and oxygen-metabolic photoacoustic microscopy of the awake mouse brain

  • Rui Cao
  • Jun Li
  • Bo Ning
  • Naidi Sun
  • Tianxiong Wang
  • Zhiyi Zuo
  • Song Hu

A long-standing challenge in optical neuroimaging has been the assessment of hemodynamics and oxygen metabolism in the awake rodent brain at the microscopic level. Here, we report first-of-a-kind head-restrained photoacoustic microscopy (PAM), which enables simultaneous imaging of the cerebrovascular anatomy, total concentration and oxygen saturation of hemoglobin, and blood flow in awake mice. Combining these hemodynamic measurements allows us to derive two key metabolic parameters—oxygen extraction fraction (OEF) and the cerebral metabolic rate of oxygen (CMRO2). This enabling technology offers the first opportunity to comprehensively and quantitatively characterize the hemodynamic and oxygen-metabolic responses of the mouse brain to isoflurane, a general anesthetic widely used in preclinical research and clinical practice. Side-by-side comparison of the awake and anesthetized brains reveals that isoflurane induces diameter-dependent arterial dilation, elevated blood flow, and reduced OEF in a dose-dependent manner. As a result of the combined effects, CMRO2 is significantly reduced in the anesthetized brain under both normoxia and hypoxia, which suggests a mechanism for anesthetic neuroprotection. The head-restrained functional and metabolic PAM opens a new avenue for basic and translational research on neurovascular coupling without the strong influence of anesthesia and on the neuroprotective effects of various interventions, including but not limited to volatile anesthetics, against cerebral hypoxia and ischemia.

Details DOI

EAAI Journal 2017 Journal Article

Grey wolf optimization evolving kernel extreme learning machine: Application to bankruptcy prediction

  • Mingjing Wang
  • Huiling Chen
  • Huaizhong Li
  • Zhennao Cai
  • Xuehua Zhao
  • Changfei Tong
  • Jun Li
  • Xin Xu

This study proposes a new kernel extreme learning machine (KELM) parameter tuning strategy using a novel swarm intelligence algorithm called grey wolf optimization (GWO). GWO, which simulates the social hierarchy and hunting behavior of grey wolves in nature, is adopted to construct an effective KELM model for bankruptcy prediction. The derived model GWO-KELM is rigorously compared with three competitive KELM methods, which are typical in a comprehensive set of methods including particle swarm optimization-based KELM, genetic algorithm-based KELM, grid-search technique-based KELM, extreme learning machine, improved extreme learning machine, support vector machines and random forest, on two real-life datasets via 10-fold cross validation analysis. Results obtained clearly confirm the superiority of the developed model in terms of classification accuracy (training, validation, test), Type I error, Type II error, area under the receiver operating characteristic curve (AUC) criterion as well as computational time. Therefore, the proposed GWO-KELM prediction model is promising to serve as a powerful early warning tool with excellent performance for bankruptcy prediction.

Details DOI

IJCAI Conference 2017 Conference Paper

Large-scale Subspace Clustering by Fast Regression Coding

  • Jun Li
  • Handong Zhao
  • Zhiqiang Tao
  • Yun Fu

Large-Scale Subspace Clustering (LSSC) is an interesting and important problem in big data era. However, most existing methods (i. e. , sparse or low-rank subspace clustering) cannot be directly used for solving LSSC because they suffer from the high time complexity-quadratic or cubic in n (the number of data points). To overcome this limitation, we propose a Fast Regression Coding (FRC) to optimize regression codes, and simultaneously train a non-linear function to approximate the codes. By using FRC, we develop an efficient Regression Coding Clustering (RCC) framework to solve the LSSC problem. It consists of sampling, FRC and clustering. RCC randomly samples a small number of data points, quickly calculates the codes of all data points by using the non-linear function learned from FRC, and employs a large-scale spectral clustering method to cluster the codes. Besides, we provide a theorem guarantee that the non-linear function has a first-order approximation ability and a group effect. The theorem manifests that the codes are easily used to construct a dividable similarity graph. Compared with the state-of-the-art LSSC methods, our model achieves better clustering results in large-scale datasets.

PDF Details

IJCAI Conference 2017 Conference Paper

Projective Low-rank Subspace Clustering via Learning Deep Encoder

  • Jun Li
  • Liu Hongfu
  • Handong Zhao
  • Yun Fu

Low-rank subspace clustering (LRSC) has been considered as the state-of-the-art method on small datasets. LRSC constructs a desired similarity graph by low-rank representation (LRR), and employs a spectral clustering to segment the data samples. However, effectively applying LRSC into clustering big data becomes a challenge because both LRR and spectral clustering suffer from high computational cost. To address this challenge, we create a projective low-rank subspace clustering (PLrSC) scheme for large scale clustering problem. First, a small dataset is randomly sampled from big dataset. Second, our proposed predictive low-rank decomposition (PLD) is applied to train a deep encoder by using the small dataset, and the deep encoder is used to fast compute the low-rank representations of all data samples. Third, fast spectral clustering is employed to segment the representations. As a non-trivial contribution, we theoretically prove the deep encoder can universally approximate to the exact (or bounded) recovery of the row space. Experiments verify that our scheme outperforms the related methods on large scale datasets in a small amount of time. We achieve the state-of-art clustering accuracy by 95. 8% on MNIST using scattering convolution features.

PDF Details

AAAI Conference 2017 Conference Paper

Sparse Subspace Clustering by Learning Approximation _0 Codes

  • Jun Li
  • Yu Kong
  • Yun Fu

Subspace clustering has been widely applied to detect meaningful clusters in high-dimensional data spaces. A main challenge in subspace clustering is to quickly calculate a “good” affinity matrix. 0, 1, 2 or nuclear norm regularization is used to construct the affinity matrix in many subspace clustering methods because of their theoretical guarantees and empirical success. However, they suffer from the following problems: (1) 2 and nuclear norm regularization require very strong assumptions to guarantee a subspace-preserving affinity; (2) although 1 regularization can be guaranteed to give a subspace-preserving affinity under certain conditions, it needs more time to solve a large-scale convex optimization problem; (3) 0 regularization can yield a tradeoff between computationally efficient and subspace-preserving affinity by using the orthogonal matching pursuit (OMP) algorithm, but this still takes more time to search the solution in OMP when the number of data points is large. In order to overcome these problems, we first propose a learned OMP (LOMP) algorithm to learn a single hidden neural network (SHNN) to fast approximate the 0 code. We then exploit a sparse subspace clustering method based on 0 code which is fast computed by SHNN. Two sufficient conditions are presented to guarantee that our method can give a subspace-preserving affinity. Experiments on handwritten digit and face clustering show that our method not only quickly computes the 0 code, but also outperforms the relevant subspace clustering methods in clustering results. In particular, our method achieves the state-of-the-art clustering accuracy (94. 32%) on MNIST.

PDF Details

AAAI Conference 2016 Conference Paper

Shakeout: A New Regularized Deep Neural Network Training Scheme

  • Guoliang Kang
  • Jun Li
  • Dacheng Tao

Recent years have witnessed the success of deep neural networks in dealing with a plenty of practical problems. The invention of effective training techniques largely contributes to this success. The so-called "Dropout" training scheme is one of the most powerful tool to reduce over-fitting. From the statistic point of view, Dropout works by implicitly imposing an L2 regularizer on the weights. In this paper, we present a new training scheme: Shakeout. Instead of randomly discarding units as Dropout does at the training stage, our method randomly chooses to enhance or inverse the contributions of each unit to the next layer. We show that our scheme leads to a combination of L1 regularization and L2 regularization imposed on the weights, which has been proved effective by the Elastic Net models in practice. We have empirically evaluated the Shakeout scheme and demonstrated that sparse network weights are obtained via Shakeout training. Our classification experiments on real-life image datasets MNIST and CIFAR- 10 show that Shakeout deals with over-fitting effectively.

PDF Details

TCS Journal 2015 Journal Article

A better approximation for constructing virtual backbone in 3D wireless ad-hoc networks

  • Xiaofeng Gao
  • Jun Li
  • Guihai Chen

Wireless ad hoc networks have been widely used in many areas. In order to improve network performance, we often select a connected dominating set (CDS) as its virtual backbone to deal with routing-related tasks. The problem of finding a minimum CDS (MCDS) for 2-dimensional networks has been widely studied, whereas finding an MCDS in 3-dimensional networks draws more attention recently, because it can formulate the network environment more precisely. Since MCDS problem is proved to be NP-complete, lots of approximations were proposed in literature. Among those, the best approximation for MCDS in 3D network is 14. 937 in [1]. However, their projection method during the approximation deduction process is incorrect, which overthrows its final bound completely. As a consequence, in this paper we will first propose a new projection method to overcome their problem, illustrate the cardinality upper bound of independent points in a graph (which will be used to analyze the approximation ratio), and then optimize the algorithms to select MCDS with prune techniques. The major technique we use is an adaptive jitter scheme, which solves the open question in this area.

Details DOI

AAAI Conference 2015 Conference Paper

Sparse Deep Stacking Network for Image Classification

  • Jun Li
  • Heyou Chang
  • Jian Yang

Sparse coding can learn good robust representation to noise and model more higher-order representation for image classification. However, the inference algorithm is computationally expensive even though the supervised signals are used to learn compact and discriminative dictionaries in sparse coding techniques. Luckily, a simplified neural network module (SNNM) has been proposed to directly learn the discriminative dictionaries for avoiding the expensive inference. But the SNNM module ignores the sparse representations. Therefore, we propose a sparse SNNM module by adding the mixed-norm regularization (l1/l2 norm). The sparse SNNM modules are further stacked to build a sparse deep stacking network (S-DSN). In the experiments, we evaluate S-DSN with four databases, including Extended YaleB, AR, 15 scene and Caltech101. Experimental results show that our model outperforms related classification methods with only a linear classifier. It is worth noting that we reach 98. 8% recognition accuracy on 15 scene.

PDF Details

YNIMG Journal 2014 Journal Article

Fiber connectivity between the striatum and cortical and subcortical regions is associated with temperaments in Chinese males

  • Xuemei Lei
  • Chuansheng Chen
  • Feng Xue
  • Qinghua He
  • Chunhui Chen
  • Qi Liu
  • Robert K. Moyzis
  • Gui Xue

The seven-factor biopsychosocial model of personality distinguished four biologically based temperaments and three psychosocially based characters. Previous studies have suggested that the four temperaments—novelty seeking (NS), reward dependence (RD), harm avoidance (HA), and persistence (P)—have their respective neurobiological correlates, especially in the striatum-connected subcortical and cortical networks. However, few studies have investigated their neurobiological basis in the form of fiber connectivity between brain regions. This study correlated temperaments with fiber connectivity between the striatum and subcortical and cortical hub regions in a sample of 50 Chinese adult males. Generally consistent with our hypotheses, results showed that: (1) NS was positively correlated with fiber connectivity from the medial and lateral orbitofrontal cortex (mOFC, lOFC) and amygdala to the striatum; (2) RD was positively correlated with fiber connectivity from the mOFC, posterior cingulate cortex/retrosplenial cortex (PCC), hippocampus, and amygdala to the striatum; (3) HA was positively linked to fiber connectivity from the dorsolateral prefrontal cortex (dlPFC) and PCC to the striatum; and (4) P was positively linked to fiber connectivity from the mOFC to the striatum. These results extended the research on the neurobiological basis of temperaments by identifying their anatomical fiber connectivity correlates within the subcortical–cortical neural networks.

Details DOI

IJCAI Conference 2013 Conference Paper

A Bayesian Factorised Covariance Model for Image Analysis

  • Jun Li
  • Dacheng Tao

This paper presents a specialised Bayesian model for analysing the covariance of data that are observed in the form of matrices, which is particularly suitable for images. Compared to existing generalpurpose covariance learning techniques, we exploit the fact that the variables are organised as an array with two sets of ordered indexes, which induces innate relationship between the variables. Specifically, we adopt a factorised structure for the covariance matrix. The covariance of two variables is represented by the product of the covariance of the two corresponding rows and that of the two columns. The factors, i. e. the row-wise and column-wise covariance matrices are estimated by Bayesian inference with sparse priors. Empirical study has been conducted on image analysis. The model first learns correlations between the rows and columns in an image plane. Then the correlations between individual pixels can be inferred by their locations. This scheme utilises the structural information of an image, and benefits the analysis when the data are damaged or insufficient.

PDF Details DOI

YNIMG Journal 2010 Journal Article

Haplotypes of catechol-O-methyltransferase modulate intelligence-related brain white matter integrity

  • Bing Liu
  • Jun Li
  • Chunshui Yu
  • Yonghui Li
  • Yong Liu
  • Ming Song
  • Ming Fan
  • Kuncheng Li

Twin studies have indicated a common genetic origin for intelligence and for variations in brain morphology. Our previous diffusion tensor imaging studies found an association between intelligence and white matter integrity of specific brain regions or tracts. However, specific genetic determinants of the white matter integrity of these brain regions and tracts are still unclear. In this study, we assess whether and how catechol-O-methyltransferase (COMT) gene polymorphisms affect brain white matter integrity. We genotyped twelve single nucleotide polymorphisms (SNPs) within the COMT gene and performed haplotype analyses on data from 79 healthy subjects. Our subjects had the same three major COMT haplotypes (termed the HPS, APS and LPS haplotypes) as previous studies have reported as regulating significantly different levels of enzymatic activity and dopamine. We used the mean fractional anisotropy (FA) values from four regions and five tracts of interest to assess the effect of COMT polymorphisms, including the well-studied val158met SNP and the three main haplotypes that we had identified, on intelligence-related white matter integrity. We identified an association between the mean FA values of two regions in the bilateral prefrontal lobes and the COMT haplotypes, rather than between them and val158met. The haplotype-FA value associations modulated nonlinearly and fit an inverted U-model. Our findings suggest that COMT haplotypes can nonlinearly modulate the intelligence-related white matter integrity of the prefrontal lobes by more significantly influencing prefrontal dopamine variations than does val158met.

Details DOI

YNIMG Journal 2008 Journal Article

Brain spontaneous functional connectivity and intelligence

  • Ming Song
  • Yuan Zhou
  • Jun Li
  • Yong Liu
  • Lixia Tian
  • Chunshui Yu
  • Tianzi Jiang

Many functional imaging studies have been performed to explore the neural basis of intelligence by detecting brain activity changes induced by intelligence-related tasks, such as reasoning or working memory. However, little is known about whether the spontaneous brain activity at rest is relevant to the differences in intelligence. Here, 59 healthy adult subjects (Wechsler Adult Intelligence Scale score, 90–138) were studied with resting state fMRI. We took the bilateral dorsolateral prefrontal cortices (DLPFC) as the seed regions and investigated the correlations across subjects between individual intelligence scores and the strength of the functional connectivity (FC) between the seed regions and other brain regions. We found that the brain regions in which the strength of the FC significantly correlated with intelligence scores were distributed in the frontal, parietal, occipital and limbic lobes. Stepwise linear regression analysis also revealed that the FCs within the frontal lobe and between the frontal and posterior brain regions were both important predictive factors for the differences in intelligence. These findings support a network view of intelligence, as suggested in previous studies. More importantly, our findings suggest that brain activity may be relevant to the differences in intelligence even in the resting state and in the absence of an explicit cognitive demand. This could provide a new perspective for understanding the neural basis of intelligence.

Details DOI

YNIMG Journal 2008 Journal Article

White matter tract integrity and intelligence in patients with mental retardation and healthy adults

  • Chunshui Yu
  • Jun Li
  • Yong Liu
  • Wen Qin
  • Yonghui Li
  • Ni Shu
  • Tianzi Jiang
  • Kuncheng Li

It is well known that brain structures correlate with intelligence but the association between the integrity of brain white matter tracts and intelligence in patients with mental retardation (MR) and healthy adults remains unknown. The aims of this study are to investigate whether the integrity of corpus callosum (CC), cingulum, uncinate fasciculus (UF), optic radiation (OR) and corticospinal tract (CST) are damaged in patients with MR, and to determine the correlations between the integrity of these tracts and full scale intelligence quotient (FSIQ) in both patients and controls. Fifteen MR patients and 79 healthy controls underwent intelligence tests and diffusion tensor imaging examinations. According to the FSIQ, all healthy controls were divided into general intelligence (GI: FSIQ<120; n=42) and high intelligence (HI: FSIQ≥120; n=37) groups. Intelligence was assessed by Chinese Revised Wechsler Adult Intelligence Scale, and white matter tract integrity was assessed by fractional anisotropy (FA). MR patients showed significantly lower FA than healthy controls in the CC, UF, OR and CST. However, GI subjects only demonstrated lower FA than HI subjects in the right UF. Partial correlation analysis controlling for age and sex showed that FSIQ scores were significantly correlated with the FA of the bilateral UF, genu and truncus of CC, bilateral OR and left CST. While FSIQ scores were only significantly correlated with the FA of the right UF when further controlling for group. This study indicate that MR patients show extensive damage in the integrity of the brain white matter tracts, and the right UF is an important neural basis of human intelligence.

Details DOI

YNIMG Journal 2007 Journal Article

Plasticity of the corticospinal tract in early blindness revealed by quantitative analysis of fractional anisotropy based on diffusion tensor tractography

  • Chunshui Yu
  • Ni Shu
  • Jun Li
  • Wen Qin
  • Tianzi Jiang
  • Kuncheng Li

Early visual deprivation may induce plastic changes, not only in the visual system, but also in the remaining sensory systems, secondary to altered experience in these spared modalities. Most of previous studies were focused on the plasticity of cortical areas of sensory modalities, but little attention was paid to the plasticity of motor system and white matter fiber tracts. Our purpose is to investigate the plasticity of the corticospinal tract (CST) in early blindness by tract-based quantitative analysis of fractional anisotropy (FA). Diffusion tensor imaging was performed in 17 early blind and 17 gender- and age-matched sighted subjects. The entire CST of each subject was reconstructed and the average FA of the tract was analyzed. To validate the results derived from the entire CST, we further analyzed a segment of the CST between the lowest slice of the cerebral peduncle and the uppermost slice of the lateral ventricle, in which the fibers are coherently arranged and the anatomical correspondence of the CST across subjects is established. On comparison with matched sighted participants, the average FA of the CST was significantly increased in the early blind men, but not in the early blind women. In conclusion, the plasticity of the CST is present in the early blind men, which might be related to the changes of motor experience during critical developmental period of the CST. This study also supports the perspective that experience-dependent plasticity occurs not only in the cortical areas but also in the white matter fiber tracts.

Details DOI

IROS Conference 2006 Conference Paper

Q-RAN: A Constructive Reinforcement Learning Approach for Robot Behavior Learning

  • Jun Li
  • Achim J. Lilienthal
  • Tomás Martínez-Marín
  • Tom Duckett

This paper presents a learning system that uses Q-learning with a resource allocating network (RAN) for behavior learning in mobile robotics. The RAN is used as a function approximator, and Q-learning is used to learn the control policy in 'off-policy' fashion that enables learning to be bootstrapped by a prior knowledge controller, thus speeding up the reinforcement learning. Our approach is verified on a PeopleBot robot executing a visual servoing based docking behavior in which the robot is required to reach a goal pose. Further experiments show that the RAN network can also be used for supervised learning prior to reinforcement learning in a layered architecture, thus further improving the performance of the docking behavior

Details
v2026.09.13