Author name cluster

Dong Nie

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

1 author row

JBHI Journal 2026 Journal Article

A Knowledge-Guided Bi-Modal Network for the Classification of Anterior Chamber Angle Images

Xiping Jia
Jingqi Huang
Dong Nie
Linan Guan
Jianying Qiu

Glaucoma is a leading cause of irreversible blindness globally. When glaucoma is diagnosed, Anterior Chamber Angle (ACA) evaluation is the necessary step for the prognosis and treatment of glaucoma. However, current clinical evaluation methods are labor intensive and rely on expert judgment, which makes them inefficient. Automating ACA classification based on images using machine learning, especially deep neural networks, holds promise. Yet, image samples alone can't provide sufficient high-level semantic information on ACA, resulting in suboptimal classification performance. This paper proposes a novel end-to-end knowledge-guided bi-modal network (KGNet) for ACA evaluation. Specifically, we consider two modalities of ACA data: textual domain knowledge and images. We first design a new strategy to refine class-based knowledge into textual descriptions, thereby increasing the diversity of features learned by the model. We then extract two types of representations using two distinct components: 1) a supervised loss is applied to learn modality-specific representations by incorporating domain knowledge; 2) a fusion module that uses knowledge-guided learning to highlight key clinical structures in ACA images leveraging bimodal correlations. Experimental results on an ACA dataset and three public datasets show that our method outperforms several state-of-the-art deep learning models in eye image evaluation, indicating the potential medical interest of our method. Furthermore, our approach improves interpretability by explicitly aligning visual representations with structured clinical knowledge, enabling more structured and clinically grounded explanations than conventional models.

Details DOI

AAAI Conference 2026 Conference Paper

Optimizing LoRA Allocation of MoE with the Alignment of Topic Correlation

Hengyuan Xu
Wenjun Ke
Yao He
Jiajun Liu
Dong Nie
Peng Wang
Ziyu Shang
Zijie Xu

Mixture of experts (MoE) dynamically routes inputs to specialized expert networks to scale model capacity with low inference overhead. However, the excessive parameter growth in MoE models poses challenges in low-resource settings. To address these issues, MoE with parameter-efficient fine-tuning (PEFT) methods have emerged as a lightweight adaptation paradigm that distributes knowledge among experts via multiple LoRA blocks. Existing MoE-PEFT methods can be broadly categorized into External and Internal PEFT methods. External PEFT methods incorporate lightweight models into existing MoE architectures without modifying their routing, which limits the model’s parameter efficiency. To overcome these issues, Internal PEFT methods integrate MoE architectures into PEFT, enabling minimal parameter overhead. However, they still face two major challenges: (1) lack of expert functional differentiation, resulting in overlapping specialization across modules, and (2) absence of a structured attribution mechanism to guide expert selection based on semantic relevance. To alleviate these challenges, we propose TopicLoRA, a novel three-stage framework that leverages topic knowledge as semantic anchors to guide expert allocation. Specifically, (1) to address expert redundancy, we construct a topic-level prior graph using Graph Neural Network-enhanced representation learning over Big-Bench categories, enforcing structural separation among expert embeddings, and (2) to introduce semantic attribution, we design a dual-loss training mechanism that softly aligns input-query relevance with topic-guided routing distributions via KL divergence. Extensive experiments on representative datasets (e.g., MMLU, GSM8K, Flanv2) demonstrate that TopicLoRA outperforms state-of-the-art PEFT baselines by 2.40% on average in accuracy. Notably, the maximum improvement is 4.21%. Furthermore, ablation studies demonstrate that our framework's robustness to intricate topics and input sequence variations, which stems from the dual-loss training mechanism.

PDF Details DOI

NeurIPS Conference 2025 Conference Paper

Brain-Inspired fMRI-to-Text Decoding via Incremental and Wrap-Up Language Modeling

Wentao Lu
Dong Nie
Pengcheng Xue
Zheng Cui
Piji Li
Daoqiang Zhang
Xuyun Wen

Decoding natural language text from non-invasive brain signals, such as functional magnetic resonance imaging (fMRI), remains a central challenge in brain-computer interface research. While recent advances in large language models (LLMs) have enabled open-vocabulary fMRI-to-text decoding, existing frameworks typically process the entire fMRI sequence in a single step, leading to performance degradation when handling long input sequences due to memory overload and semantic drift. To address this limitation, we propose a brain-inspired sequential fMRI-to-text decoding framework that mimics the human cognitive strategy of segmented and inductive language processing. Specifically, we divide long fMRI time series into consecutive segments aligned with optimal language comprehension length. Each segment is decoded incrementally, followed by a wrap-up mechanism that summarizes the semantic content and incorporates it as prior knowledge into subsequent decoding steps. This sequence-wise approach alleviates memory burden and ensures semantic continuity across segments. In addition, we introduce a text-guided masking strategy integrated with a masked autoencoder (MAE) framework for fMRI representation learning. This method leverages attention distributions over key semantic tokens to selectively mask the corresponding fMRI time points, and employs MAE to guide the model toward focusing on neural activity at semantically salient moments, thereby enhancing the capability of fMRI embeddings to represent textual information. Experimental results on the two datasets demonstrate that our method significantly outperforms state-of-the-art approaches, with performance gains increasing as decoding length grows.

PDF Details

JBHI Journal 2025 Journal Article

Structure-Aware Brain Tissue Segmentation for Isointense Infant MRI Data Using Multi-Phase Multi-Scale Assistance Network

Jiameng Liu
Feihong Liu
Dong Nie
Yuning Gu
Yuhang Sun
Dinggang Shen

Accurate and automatic brain tissue segmentation is crucial for tracking brain development and diagnosing brain disorders. However, due to inherently ongoing myelination and maturation during the first postnatal year, the intensity distributions of gray matter and white matter in the infant brain MRI at the age of around 6 months old ( a. k. a. isointense phase) are highly overlapped, which makes tissue segmentation very challenging, even for experts. To address this issue, in this study, we propose a multi-phase multi-scale assistance segmentation framework, which comprises a structure-preserved generative adversarial network (SPGAN) and a multi-phase multi-scale assisted segmentation network (M $^{2}$ ASN). SPGAN bi-directionally synthesizes isointense and adult-like data. The synthetic isointense data essentially augment the training dataset, combined with high-quality annotations transferred from its adult-like counterpart. By contrast, the synthetic adult-like data offers clear tissue structures and is concatenated with isointense data to serve as the input of M $^{2}$ ASN. In particular, M $^{2}$ ASN is designed with two-branch networks, which simultaneously segment tissues with two phases (isointense and adult-like) and two scales by also preserving their correspondences. We further propose a boundary refinement module to extract maximum gradients from local feature maps to indicate tissue boundaries, prompting M $^{2}$ ASN to focus more on boundaries where segmentation errors are prone to occur. Extensive experiments on the National Database for Autism Research and Baby Connectome Project datasets quantitatively and qualitatively demonstrate the superiority of our proposed framework compared with seven state-of-the-art methods.

Details DOI

JBHI Journal 2024 Journal Article

A New Multi-Atlas Based Deep Learning Segmentation Framework With Differentiable Atlas Feature Warping

Huabing Liu
Dong Nie
Jian Yang
Jinda Wang
Zhenyu Tang

Deep learning based multi-atlas segmentation (DL-MA) has achieved the state-of-the-art performance in many medical image segmentation tasks, e. g. , brain parcellation. In DL-MA methods, atlas-target correspondence is the key for accurate segmentation. In most existing DL-MA methods, such correspondence is usually established using traditional or deep learning based registration methods at image level with no further feature level adaption. This could cause possible atlas-target feature inconsistency. As a result, the information from atlases often has limited positive and even counteractive impact on the final segmentation results. To tackle this issue, in this paper, we propose a new DL-MA framework, where a novel differentiable atlas feature warping module with a new smooth regularization term is presented to establish feature level atlas-target correspondence. Comparing with the existing DL-MA methods, in our framework, atlas features containing anatomical prior knowledge are more relevant to the target image feature, leading the final segmentation results to a high accuracy level. We evaluate our framework in the context of brain parcellation using two public MR brain image datasets: LPBA40 and NIREP-NA0. The experimental results demonstrate that our framework outperforms both traditional multi-atlas segmentation (MAS) and state-of-the-art DL-MA methods with statistical significance. Further ablation studies confirm the effectiveness of the proposed differentiable atlas feature warping module.

Details DOI

NeurIPS Conference 2024 Conference Paper

Unveiling LoRA Intrinsic Ranks via Salience Analysis

Wenjun Ke
Jiahao Wang
Peng Wang
Jiajun Liu
Dong Nie
Guozheng Li
Yining Li

The immense parameter scale of large language models underscores the necessity for parameter-efficient fine-tuning methods. Methods based on Low-Rank Adaptation (LoRA) assume the low-rank characteristics of the incremental matrix and optimize the matrix obtained from low-rank decomposition. Although effective, these methods are constrained by a fixed and unalterable intrinsic rank, neglecting the variable importance of matrices. Consequently, methods for adaptive rank allocation are proposed, among which AdaLoRA demonstrates excellent fine-tuning performance. AdaLoRA conducts adaptation based on singular value decomposition (SVD), dynamically allocating intrinsic ranks according to importance. However, it still struggles to achieve a balance between fine-tuning effectiveness and efficiency, leading to limited rank allocation space. Additionally, the importance measurement focuses only on parameters with minimal impact on the loss, neglecting the dominant role of singular values in SVD-based matrices and the fluctuations during training. To address these issues, we propose SalientLoRA, which adaptively optimizes intrinsic ranks of LoRA via salience measurement. Firstly, during rank allocation, the salience measurement analyses the variation of singular value magnitudes across multiple time steps and establishes their inter-dependency relationships to assess the matrix importance. This measurement mitigates instability and randomness that may arise during importance assessment. Secondly, to achieve a balance between fine-tuning performance and efficiency, we propose an adaptive adjustment of time-series window, which adaptively controls the size of time-series for significance measurement and rank reduction during training, allowing for rapid rank allocation while maintaining training stability. This mechanism enables matrics to set a higher initial rank, thus expanding the allocation space for ranks. To evaluate the generality of our method across various tasks, we conduct experiments on natural language understanding (NLU), natural language generation (NLG), and large model instruction tuning tasks. Experimental results demonstrate the superiority of SalientLoRA, which outperforms state-of-the-art methods by 0. 96\%-3. 56\% on multiple datasets. Furthermore, as the rank allocation space expands, our method ensures fine-tuning efficiency, achieving a speed improvement of 94. 5\% compared to AdaLoRA. The code is publicly available at https: //github. com/Heyest/SalientLoRA.

PDF Details DOI

AAAI Conference 2020 Conference Paper

Hybrid Graph Neural Networks for Crowd Counting

Ao Luo
Fan Yang
Xin Li
Dong Nie
Zhicheng Jiao
Shangchen Zhou
Hong Cheng

Crowd counting is an important yet challenging task due to the large scale and density variation. Recent investigations have shown that distilling rich relations among multi-scale features and exploiting useful information from the auxiliary task, i. e. , localization, are vital for this task. Nevertheless, how to comprehensively leverage these relations within a uni- ﬁed network architecture is still a challenging problem. In this paper, we present a novel network structure called Hybrid Graph Neural Network (HyGnn) which targets to relieve the problem by interweaving the multi-scale features for crowd density as well as its auxiliary task (localization) together and performing joint reasoning over a graph. Speciﬁcally, HyGnn integrates a hybrid graph to jointly represent the task-speciﬁc feature maps of diﬀerent scales as nodes, and two types of relations as edges: (i) multi-scale relations capturing the feature dependencies across scales and (ii) mutual beneﬁcial relations building bridges for the cooperation between counting and localization. Thus, through message passing, HyGnn can capture and distill richer relations between nodes to obtain more powerful representations, providing robust and accurate results. Our HyGnn performs signiﬁcantly well on four challenging datasets: ShanghaiTech Part A, ShanghaiTech Part B, UCF CC 50 and UCF QNRF, outperforming the state-ofthe-art algorithms by a large margin.

PDF Details

AAAI Conference 2019 Conference Paper

Difficulty-Aware Attention Network with Confidence Learning for Medical Image Segmentation

Dong Nie
Li Wang
Lei Xiang
Sihang Zhou
Ehsan Adeli
Dinggang Shen

Medical image segmentation is a key step for various applications, such as image-guided radiation therapy and diagnosis. Recently, deep neural networks provided promising solutions for automatic image segmentation; however, they often perform good on regular samples (i. e. , easy-to-segment samples), since the datasets are dominated by easy and regular samples. For medical images, due to huge inter-subject variations or disease-specific effects on subjects, there exist several difficult-to-segment cases that are often overlooked by the previous works. To address this challenge, we propose a difficulty-aware deep segmentation network with confidence learning for end-to-end segmentation. The proposed framework has two main contributions: 1) Besides the segmentation network, we also propose a fully convolutional adversarial network for confidence learning to provide voxel-wise and region-wise confidence information for the segmentation network. We relax the adversarial learning to confidence learning by decreasing the priority of adversarial learning, so that we can avoid the training imbalance between generator and discriminator. 2) We propose a difficulty-aware attention mechanism to properly handle hard samples or hard regions considering structural information, which may go beyond the shortcomings of focal loss. We further propose a fusion module to selectively fuse the concatenated feature maps in encoder-decoder architectures. Experimental results on clinical and challenge datasets show that our proposed network can achieve state-of-the-art segmentation accuracy. Further analysis also indicates that each individual component of our proposed network contributes to the overall performance improvement.

PDF Details

JBHI Journal 2018 Journal Article

Anatomical Landmark Based Deep Feature Representation for MR Images in Brain Disease Diagnosis

Mingxia Liu
Jun Zhang
Dong Nie
Pew-Thian Yap
Dinggang Shen

Most automated techniques for brain disease diagnosis utilize hand-crafted (e. g. , voxel-based or region-based) biomarkers from structural magnetic resonance (MR) images as feature representations. However, these hand-crafted features are usually high-dimensional or require regions-of-interest defined by experts. Also, because of possibly heterogeneous property between the hand-crafted features and the subsequent model, existing methods may lead to sub-optimal performances in brain disease diagnosis. In this paper, we propose a landmark-based deep feature learning (LDFL) framework to automatically extract patch-based representation from MRI for automatic diagnosis of Alzheimer's disease. We first identify discriminative anatomical landmarks from MR images in a data-driven manner, and then propose a convolutional neural network for patch-based deep feature learning. We have evaluated the proposed method on subjects from three public datasets, including the Alzheimer's disease neuroimaging initiative (ADNI-1), ADNI-2, and the minimal interval resonance imaging in alzheimer's disease (MIRIAD) dataset. Experimental results of both tasks of brain disease classification and MR image retrieval demonstrate that the proposed LDFL method improves the performance of disease classification and MR image retrieval.

Details DOI